This post is the next in the series on how I overhauled my personal infrastructure to make it easier to manage, make changes and integrate new applications.

Previous posts in the series are: -

Now that we have a better Ansible and Salt codebase, and have Drone working, we can start to look at some Drone pipelines! In this post, I will cover using Ansible with Drone.

Background

As mentioned in previous posts, I currently use Ansible to manage bootstrapping machines in my home infrastructure, as well as managing DNS, DHCP, IP address management, and adding monitoring/Prometheus checks for hosts not managed by Salt.

In addition, I have also recently started managing my lab environments (usually for Proof-of-Concepts, or even building infrastructure to use in posts on this site) using Ansible too.

To manage all of this before, it required manual updates, running multiple playbooks from the command line, all while logged in to a single server.

By creating pipelines in Drone, I can now commit changes directly to Git (from any machine, or even a phone/tablet) and the changes will roll out automatically.

Steps

The flow I want to use for making changes is as follows: -

  1. Create a branch in Git
  2. Make the relevant changes
  3. Commit the branch to Git
  4. Raise a Pull Request
  5. Before merging, have the Ansible playbooks go through a syntax check and linting, as well as a dry run
  6. Send a notification of success or failure
  7. Show the results of the dry run
  8. Merge the Pull Request
  9. Apply the changes
  10. Send a notification of success or failure

This seems like a lot of steps, but this flow is how I tend to work anyway (personally and professionally). Drone adds the automated testing and applying the changes.

Full Drone Pipeline

The below is the full pipeline: -

kind: pipeline
name: default
type: docker

trigger:
  branch:
    - main

steps:
- name: Syntax Check 
  image: plugins/ansible
  settings:
    become: true
    playbook: playbook.yml
    inventory: inventory
    private_key:
      from_secret: drone_ssh_priv
    ssh_extra_args: "-o StrictHostKeyChecking=no"
    requirements: requirements.txt
    galaxy: requirements.yml
    syntax_check: true
  when:
    event:
    - pull_request

- name: Lint
  image: cytopia/ansible-lint
  commands:
    - ansible-lint playbook.yml --force-color
  when:
    event:
    - pull_request

- name: Show Diff and Check
  image: plugins/ansible
  settings:
    become: true
    playbook: playbook.yml
    inventory: inventory
    private_key:
      from_secret: drone_ssh_priv
    diff: true
    check: true
    ssh_extra_args: "-o StrictHostKeyChecking=no"
    requirements: requirements.txt
    galaxy: requirements.yml
  when:
    event:
    - pull_request

- name: slack-pr
  image: plugins/slack
  settings:
    webhook:
      from_secret: drone_builds_slack_webhook 
    channel: builds
    template: >
      {{#success build.status}}
        {{repo.name}} PR build passed. 
        Merge in to apply.
        PR: https://git.noisepalace.co.uk/YetiOps/{{repo.name}}/pulls/{{build.pull}}
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{else}}
        {{repo.name}} PR build failed. 
        Please investigate. 
        PR: https://git.noisepalace.co.uk/YetiOps/{{repo.name}}/pulls/{{build.pull}}
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{/success}}      
  when:
    status:
    - failure
    - success
    event:
      - pull_request

- name: slack-push-start
  image: plugins/slack
  settings:
    webhook:
      from_secret: drone_builds_slack_webhook 
    channel: builds
    template: >
      {{repo.name}} build is starting.
      Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
  when:
    branch:
    - main
    event:
    - push
    - tag

- name: Apply Playbook 
  image: plugins/ansible
  settings:
    become: true
    playbook: playbook.yml
    inventory: inventory
    private_key:
      from_secret: drone_ssh_priv
    ssh_extra_args: "-o StrictHostKeyChecking=no"
    requirements: requirements.txt
    galaxy: requirements.yml
    diff: true
  when:
    branch:
    - main
    event:
    - push
    - tag

- name: slack-push
  image: plugins/slack
  settings:
    webhook:
      from_secret: drone_builds_slack_webhook 
    channel: builds
    template: >
      {{#success build.status}}
        {{repo.name}} build passed.
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{else}}
        {{repo.name}} build {{build.number}} failed. Please investigate. 
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{/success}}      
  when:
    status:
    - failure
    - success
    branch:
    - main
    event:
    - push
    - tag

To cover a few of the terms: -

  • kind - This defines the kind of Drone execution. This is almost always going to be pipeline (although template is another option)
  • type - This is the kind of execution. This could be docker, exec, ssh, kubernetes and many more
  • trigger - This defines what conditions need to be met for a pipeline to start
  • steps - These are the steps that the pipeline will take

The trigger step in this pipeline specifies that it will only run if the destination branch of the Pull Request is main, or commits directly to main. Pull Requests between other branches (e.g. dev to prod) would not be covered by this.

Pipeline Steps

I’ll now go through each step to show what they do.

Syntax Check

steps:
- name: Syntax Check 
  image: plugins/ansible
  settings:
    become: true
    playbook: playbook.yml
    inventory: inventory
    private_key:
      from_secret: drone_ssh_priv
    ssh_extra_args: "-o StrictHostKeyChecking=no"
    requirements: requirements.txt
    galaxy: requirements.yml
    syntax_check: true
  when:
    event:
    - pull_request

This step details the following: -

  • The name of the step
  • The Plugin (a.k.a. Docker Image) used to execute the step
  • Any settings specific to the plugin
  • When to execute the step

All available Plugins for Drone can be found here. In our case, we use a plugin (Docker Image) that will run Ansible. We also specify a few settings: -

  • become: true - This states that we will use privilege escalation (e.g. sudo, doas) for tasks
    • This is often a requirement when installing packages and/or updating configuration of system daemons
  • playbook: playbook.yml - This specifies the path for the playbook. In this repository, it is in the base path
  • inventory: inventory - This specifies the inventory file for Ansible, which in this case is a file in the base path called inventory
  • ssh_extra_args - This specifies any extra arguments to the Ansible SSH client process
  • requirements: requirements.txt - This specifies any Python dependencies required for the pipeline to run
  • galaxy: requirements.yml - This specifies any Ansible Galaxy roles (and now Collections) to install before running the Playbooks
  • private_key - This section specifies the SSH private key used by Ansible/Drone to connect to machines in the inventory
  • syntax_check: true - This tells Ansible to only check that the syntax of the roles and playbooks are correct, not actual running any steps

Drone Ansible Syntax Check

SSH Extra Arguments

The extra argument supplied here is -o StrictHostKeyChecking=no. This tells Ansible to ignore if the host SSH key is unknown.

In most cases, you don’t want to enable this as a host key changing may indicate a machine has been compromised. However because the Docker containers that apply the changes are brand new on every run, they would have no previous knowledge of any hosts. For Ansible to run from a container, we must ignore strict host key checking.

Private Key

The private key is created on another machine using ssh-keygen -t rsa. We add the public key as an authorized keys on all nodes we want to manage. In my case, this is done using an Ansible role (that Drone also controls the execution of).

The contents of the private key are then added to a secret within Drone.

Drone Secrets can be repository specific, or common across a Git organisation. The below shows this: -

Drone Secrets

We can then source the contents of this secret within a Drone pipeline.

When clause

The when field says when this action will take place. In this task, it says that it will take action on Pull Requests. Combined with the trigger on the main branch set across all steps in the Pipeline, this means that this will only take effect on Pull Requests into the main branch.

Lint

The lint step is below: -

- name: Lint
  image: cytopia/ansible-lint
  commands:
    - ansible-lint playbook.yml --force-color
  when:
    event:
    - pull_request

This uses the ansible-lint package to check that we have used the correct syntax, loops are defined correctly, variables are defined correctly and many other rules too.

Rather than supplying configuration settings (like in the previous step), this just runs a command in a Docker image against our Playbook.

Drone Ansible Lint

Show Diff and Check

- name: Show Diff and Check
  image: plugins/ansible
  settings:
    become: true
    playbook: playbook.yml
    inventory: inventory
    private_key:
      from_secret: drone_ssh_priv
    diff: true
    check: true
    ssh_extra_args: "-o StrictHostKeyChecking=no"
    requirements: requirements.txt
    galaxy: requirements.yml
  when:
    event:
    - pull_request

This step is almost identical to the Syntax Check stage, but with some minor differences: -

  • We do not set syntax_check to true
  • We add the diff: true and check: true flags

This will do a Dry Run of the playbook (i.e. shows what actions would take place) and will also show a diff (i.e. the changes) between what already exists and what would change.

This allows us to see what actions would take place, without them actually being applied. We can then make a judgement call on whether the changes are correct before merging and applying the code.

Drone Ansible Diff and Check

Slack PR

- name: slack-pr
  image: plugins/slack
  settings:
    webhook:
      from_secret: drone_builds_slack_webhook 
    channel: builds
    template: >
      {{#success build.status}}
        {{repo.name}} PR build passed. 
        Merge in to apply.
        PR: https://git.noisepalace.co.uk/YetiOps/{{repo.name}}/pulls/{{build.pull}}
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{else}}
        {{repo.name}} PR build failed. 
        Please investigate. 
        PR: https://git.noisepalace.co.uk/YetiOps/{{repo.name}}/pulls/{{build.pull}}
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{/success}}            
  when:
    status:
    - failure
    - success
    event:
      - pull_request

This task is used to send notifications to a Slack instance. We have a template which includes relevant details (i.e. links to the build job and the pull request), and also shows a slightly different message based upon whether the job was successful or not.

One point to note here is that in the when section, we match on a status of success or failure. Most pipeline steps will not execute if a previous step has failed, which would mean we wouldn’t receive notifications on failure (only success). Adding this condition means it will execute even if a previous step failed.

Slack Push Start

- name: slack-push-start
  image: plugins/slack
  settings:
    webhook:
      from_secret: drone_builds_slack_webhook 
    channel: builds
    template: >
      {{repo.name}} build is starting.
      Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}      
  when:
    branch:
    - main
    event:
    - push
    - tag

This step is almost identical to the previous step, except that it matches upon a push to the main branch, or a tag being applied to a commit in the main branch.

This allows us to notify that a build has started.

Drone Ansible Slack Notification

Apply Playbook

- name: Apply Playbook 
  image: plugins/ansible
  settings:
    become: true
    playbook: playbook.yml
    inventory: inventory
    private_key:
      from_secret: drone_ssh_priv
    ssh_extra_args: "-o StrictHostKeyChecking=no"
    requirements: requirements.txt
    galaxy: requirements.yml
    diff: true
  when:
    branch:
    - main
    event:
    - push
    - tag

This step is identical to the Show Diff and Check step, except that we do not use the check field. This will apply all the changes, as well as showing a diff of all changes (rather than only showing that tasks were executed).

We have also changed when this will run, which again is based upon a push to the main branch, or a tagged commit.

Slack Push

- name: slack-push
  image: plugins/slack
  settings:
    webhook:
      from_secret: drone_builds_slack_webhook 
    channel: builds
    template: >
      {{#success build.status}}
        {{repo.name}} build passed.
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{else}}
        {{repo.name}} build {{build.number}} failed. Please investigate. 
        Build: https://drone.noisepalace.co.uk/YetiOps/{{repo.name}}/{{build.number}}
      {{/success}}            
  when:
    status:
    - failure
    - success
    branch:
    - main
    event:
    - push
    - tag

This final step sends a notification that the Playbook apply succeeded or failed. Again, this is based upon a push to the main branch (i.e. a merged pull request, or a direct push to the branch) or a tagged commit.

Demonstration

The below video is a demonstration of making a simple change (adding a CNAME to DNS), and seeing Gitea and Drone work together to apply the changes: -

Summary

This has demonstrated how to use Ansible with Drone. I currently use this for multiple repositories, and will probably add more to it in future as well.

The next post will cover how I use Drone CI with SaltStack, which involves the Docker runner and the Exec runner.