Lots of our infrastructure is managed by Puppet, the open source configuration management tool that uses its own domain specific language (DSL) to describe how services, files and systems should look.

The DSL was inspired by Nagios configuration files, so you’ll find lots of nested blocks under curly braces, indentation, groups of parameters assigned with => separators, all of which can begin to look messy in a large manifest, as Puppet calls them. Keeping the syntax and style clean and consistent with the style guide helps team members read and understand these manifests quickly, so it’s vital to keep a large repository navigable.

The community has developed some great tools that help you keep your code correct and the style consistent and readable.

• puppet-syntax was originally written by the talented GOV.UK team, and is now maintained by volunteers in the Vox Pupuli community along with other vital tools
• puppet-lint with its own DSL parser is maintained by Tim Sharpe and also has a community of plugins to customise it
• puppetlabs_spec_helper integrates these tools into your project, making them easy to run and use

Keeping lint-free with no effort

Our repo receives a few contributions from outside the ops team at FreeAgent, such as our corporate IT team maintaining VPN access, and engineers in other teams adding and changing configuration files or entire services. So when it came to adding puppet-lint recently, I wanted to minimise the effect of failing lint tests on other contributors and keep it easy to accept changes while enforcing a consistent style.

While some editors make it easy to run linters for you, not everybody inside and outside of our team has this set up — this meant having Jenkins, our test/continuous integration server, also help out by automatically fixing issues where it could.

The idea here is to have Jenkins run puppet-lint in fix mode to correct style issues in manifests, then push any fixes it can to the user’s branch — which is possible because our branches are in a shared Git repository. This causes a little feedback loop that results in the branch passing all of the lint tests, or for any unfixable issues to be left to the user. They can then check the output of Jenkins to see the remaining errors, run git pull, fix the issues and push the last changes.

De-linting pipeline with Jenkins

This new step is added into the Jenkins build pipeline, configured in the Jenkinsfile for the project, and looks like this:

/* Utility to run a command and return stdout */
def capture(String script) {
  return sh(returnStdout: true, script: script).trim()
}

stage('Lint check') {
  try {
    sh('chruby system && bundle exec rake lint')
  } catch(err) {
    if (env.BRANCH_NAME == 'production') {
      echo 'Unable to help on production branch, please open a PR!'
      throw err
    }

    /*
    * Attempt to autofix any lint issues and push another commit.
    * If it works, Jenkins will re-test the new commit and then pass.
    */
    echo 'Attempting to run auto-fixer..'
    fixedOutput = capture('chruby system && find . ! -path ./.bundle/\ -name \.pp -exec bundle exec puppet-lint --fix {} + || true')
    /* If issues were fixed, push the changes */
    if (capture('git diff-index HEAD --') != '') {
      echo 'Found some automatic fixes, preparing to commit them'
      commitMsg = fixedOutput.split("n").findAll {
        it.contains('FIXED:')
      }.inject("Fix lint issuesnn") {
        result, line -> result + " " + line + "n"
      }

      sh('git config user.email jenkins@example.com')
      sh('git commit -a -m "' + commitMsg + '"')
      sh('git show -s | cat')
      sh('git push origin HEAD:' + env.BRANCH_NAME)
    }

    throw err
  }
}

Now, each time a branch is tested by Jenkins, it can run through these steps:

• Check if the lint checks are passing
• If not, try to run puppet-lint with the fix mode enabled
• If it was able to fix issues, construct a new git commit message with the output of the command, commits and pushes
• It raises an error as the branch (as it was) failed the lint test
• The new commit on the branch triggers the test to run again with the fixes, and it hopefully passes

And the end result looks like this, with our butler doing the dull work!

This is just one of the ways we aim to make the development process easier and smoother for engineers in FreeAgent. If this sort of thing interests you, look out for jobs in our Development Platform team who come up with tools to power our growing engineering department.