Embracing automation with GitHub Actions

Alvin Tang profile image Alvin Tang 2023-11-30

Actions is a feature of GitHub for automating software workflows. It includes Continuous Integration and Delivery (CI/CD) so it allows us to build, test, and deploy our code. In this post, I’ll discuss some of the things we love about it and what we think it can improve on.

Things we love about Actions

Easy integration with GitHub

One major reason for using Actions is its built on top of GitHub. If a repository is already in GitHub, using Actions is as simple as pushing a workflow file in the correct directory. Workflow files are YAML files defining the steps to be done when an event is triggered. There is no need to run a separate application and/or server since GitHub provides it already.

In our organization, we have non-technical users working on content and configuration files. We moved these files into a git repository so they are version controlled. We taught users git basics including pushing commits and creating a pull request for their changes. With this move, we were also able to add workflows to make everyone’s lives easier. The workflows do several things like running tests, generating files, and verifying protected content; several workflows build updated assets and deploy them to the correct environment. The workflows also provide feedback by generating PR comments and Slack messages. All of these workflows run automatically or can be manually triggered. Over time, our users have come to understand what happens in these workflows and how to respond to different types of failures. And they are able to do this all within the familiar surrounds of Github.

Event triggers

CI/CD workflows are usually triggered by events. Actions provide a wide array of event triggers. The push and pull request events are the most common; you can use these to trigger builds and run tests on a repository.

We’ve found uses for other event triggers such as:

  • building a package for a draft release, triggered by the release event
  • running a workflow from an external script, triggered by the repository_dispatch event
  • merging a pull request upon approval, triggered by the pull_request_review event

Marketplace

There are workflow actions that are common to many repositories. These include actions like checking out a repository, building a Docker image, or setting up tools like node. Actions has a marketplace containing actions that other developers have published. GitHub also provides its own actions such as checkout, upload-artifact, or download-artifact. Aside from GitHub, a lot of third-party providers have published actions to support their products. If we want to integrate actions with other tools like Docker or Slack, we try searching the marketplace first!

Here are some of the third-party actions we use:

  • github/checkout - This action checks out a repository so we can work with its contents. This allows you to checkout multiple repositories. For example, we have a workflow that works with two other repositories. We can check out each of those repositories in their own directories so we can work with the files inside them.
  • docker/build-push-action - We use Docker containers for building GitHub codespaces. This Docker action builds a Docker image for our codespaces and pushes it to a registry.
  • slackapi/slack-github-action - We post workflow updates in Slack. This action allows us to send messages to Slack. It has formatting and even support for threaded messages!
  • cachix/install-nix-action - We use Nix in most of our repositories. This action installs Nix in GitHub-hosted runners so we can run code in our defined Nix environment.
  • aws-actions/configure-aws-credentials - This action configures a runner to access AWS. We use this to generate OIDC tokens so workflows can access AWS resources during build, test, and deployment.

Composite actions and reusable workflows

A few months into our Actions journey, workflows were getting repetitive. We started exploring how to DRY them up. Composite actions and reusable workflows address this problem. These allow us to group repetitive steps or jobs and call them in one step, instead of writing the steps all over again.

A sample composite action we have is called slack-update. This is a wrapper around slackapi/slack-github-action. We have predefined messages for certain events such as deployment success or failure messages. Using a composite action for this keeps our messaging consistent across different repositories. Here is a code snippet of this composite action:

description: Pinned slackapi/slack-github-action. Posts a message to the specified slack channel.
using: composite
name: slack update
inputs:
  authors:
    description: authors of the change
    required: true
  branch:
    description: branch name to put in messages
    required: true
  channel-id:
    description: slack channel ID where message will be posted
    required: true
  message-type:
    description: "type of message to be posted. Options are: `build-start`, `build-fail`, `build-success`, `open-pr`"
    required: true
  pr-url:
    default: ''
  ...
runs:
  steps:
    - if: "contains( inputs.message-type, 'open-pr' ) && inputs.branch == 'master'"
      name: Slack Notifications - Create PR payload
      run: |
        echo "PAYLOAD={\"text\":\":*${{ input.authors }}* has opened a PR: ${{ inputs.pr-url }}\"}" >> $GITHUB_ENV
      shell: bash
    - env:
        SLACK_BOT_TOKEN: "${{ inputs.slack-bot-token }}"
      name: Slack - Notifications - Post to slack
      uses: "slackapi/slack-github-action@e28cf165c92ffef168d23c5c9000cffc8a25e117" # see "Managing workflows" for more on why we use a SHA here
      with:
        channel-id: "${{ inputs.channel-id }}"
        payload: "${{ env.PAYLOAD }}"

And then we call the composite action as a step in workflows:

steps:
...
- if: "github.ref == 'refs/heads/master'"
  name: Post pull request link to Slack
  uses: "bellroy/workflows/composite/slack-update@master"
  with:
    branch: "${{ env.BRANCH }}"
    channel-id: "${{ env.SLACK_CHANNEL_ID }}"
    message-type: open-pr
    pr-url: "${{ env.PR_URL }}"
    slack-bot-token: "${{ secrets.SLACK_TOKEN }}"

Other examples of our composite actions:

  • get-tool - We build internal tools that are released through GitHub. These tools are used within our workflows so they have to be retrieved for the workflow to run successfully. We wrote an action to get the latest release of a tool, or we can retrieve a specific version.
  • install-nix - This is a wrapper around cachix/install-nix-action to customize Nix installations. We have predefined setups for different access levels to our private Nix cache.

Self-hosted runners

Self-hosted runners can be set up to run actions in our own server. We do this when workflows need access to protected resources. Our self-hosted runners run within our private network. Aside from that, we also customize the runner by adding the tools it needs to do its job. For example, one of our workflows need to produce messages to a Kafka topic in AWS Managed Streaming for Apache Kafka (MSK). We have provisioned a self-hosted runner in the same network as the MSK cluster and installed (kcat)[https://github.com/edenhill/kcat] as the Kafka client.

If all else fails, use bash scripts

The ability to use bash scripts in the workflows provides endless possibilities. We have scripts for output formatting, file manipulations, running tests, cleaning up after builds, etc. Combining this with a controlled environment (such as a self-hosted runner), we can do most of the automation we need.

Things we don’t love so much about Actions

Developing and testing workflows

The process of developing and testing actions is not the best. Some event triggers need the workflow change to be on the default branch to be executed. There are ways to work around this such as adding an event that can trigger even if it’s not on the default branch. But this still has limitations if we want to test the actual event, instead of the steps.

Workflows can also take a long time to run. A minor mistake such as a typographical error can cost us minutes before we get feedback from actions. This gets compounded if testing in self-hosted runners since the actions are queued.

Managing workflows

As the number of repositories grows, so does the number of workflows. Although there are mechanisms to DRY them up, it’s better to manage these workflows in one place. One benefit of this approach is that it makes managing our third-party actions much easier. As best practice, we pin third-party actions to commit SHAs (which can’t change), rather than release tags (which can), so we don’t get surprised by updates that can break our workflows or get exposed to supply chain attacks. This means we have to update these SHAs every so often. This becomes painful when there are a lot of repositories involved.

Overall, GitHub Actions is a great tool for automation and CI/CD. If you are already using GitHub, it provides easy integration and a wide variety of actions to use. It is also versatile enough to support custom actions specific to your needs.

Next time, I will discuss how we use Dhall to manage our workflow files for Github Actions.