CI/CD for Amiberry using Github Actions (pt 1)

The Amiberry emulator supports multiple platforms, so if you want to provide binaries on new releases (like I do), you’ll have a lot of compiling to do. Now, you could just compile a binary manually for each platform you have available, but that would take (very) a long time (especially if you have some slow devices). And what happens if you want to have binaries compiled for testing before a major release? Like for example, on each new commit or pull request? That’s where we need automation, and specifically what’s called Continuous Integration and Continuous Deployment – or CI/CD for short.

I have been using Azure DevOps for many years, and the Azure Pipelines specifically, for setting up similar scenarios for CI/CD. However, since everything else regarding the Amiberry project is hosted on Github, and since there is a similar feature there, named Actions, I thought I’d give it a try and move over my pipelines to Github as well.

But let’s set the goals fist:

• I want an automated job, that would compile the latest version of Amiberry every time I commit something new in the repository (let’s say the “master” branch only, for now).
• After it compiles each binary, it should compress it with ZIP and upload it somewhere testers can access it. The ZIP archive should contain everything that’s needed to run Amiberry, not just the executable (so data files and directories should be included). If you have the requirements to run Amiberry (e.g. SDL2), the archive should be portable – just extract it and run Amiberry from it.
• The ZIP filename should indicate the platform and OS it was compiled for. For example, amiberry-rpi4-sdl2-64bit-debian.zip would be such a filename.
• When the time comes to publish a new release, I want the automated job to do the publishing for me. I will use git tags to mark a new release, with the following example format: v5.2. So basically, if I set a git tag on a certain commit, with that version format, a new release should be published automatically with that version number.
• A new release should also include the ZIP archives, which besides the platform and OS we had before, should also include the version in the filename. For example, amiberry-v5.2-rpi4-sdl2-64bit-debian.zip
• A new release should also contain a Changelog, containing what changed since the previous release. The information for that should come from the commits themselves and should be fully automated also. I don’t want to maintain a file manually if I can avoid it.

Looks like quite the list, doesn’t it? Let’s see what we can achieve!

First things first: Let’s read the docs: https://docs.github.com/en/actions (I like doing things by the book, from time to time).

Github Actions uses so-called “workflows“, which are YAML based files that dictate what should happen. It’s basically the same thing as Azure DevOps’ “Pipelines”, with slightly different syntax in some parts, so if you’re familiar with one, the other shouldn’t be too hard to get into either.

Those workflows are saved in your repository (under .github/workflows), so you can include them in the git history as you make changes to them. Nice!

Here’s an example of such a workflow file, from the Quickstart:

name: GitHub Actions Demo
on: [push]
jobs:
  Explore-GitHub-Actions:
    runs-on: ubuntu-latest
    steps:
      - run: echo "🎉 The job was automatically triggered by a ${{ github.event_name }} event."
      - run: echo "🐧 This job is now running on a ${{ runner.os }} server hosted by GitHub!"
      - run: echo "🔎 The name of your branch is ${{ github.ref }} and your repository is ${{ github.repository }}."
      - name: Check out repository code
        uses: actions/checkout@v3
      - run: echo "💡 The ${{ github.repository }} repository has been cloned to the runner."
      - run: echo "🖥️ The workflow is now ready to test your code on the runner."
      - name: List files in the repository
        run: |
          ls ${{ github.workspace }}
      - run: echo "🍏 This job's status is ${{ job.status }}."

The workflows need to run somewhere, and you can either use Github-hosted VMs, or use your own (self-hosted runners). To get started, I chose the self-hosted option for now.

If you choose to use self-hosted runners, you will need some device(s) to run the jobs on, obviously. I had several Raspberry Pi 4s around, a RockPro64 and a Mac-Mini which I could use. That should be good enough!

You can easily install the required software to make each of those machines a “self-hosted runner”, by going in your Repository settings, under Actions -> Runners:

Then click on “New self-hosted runner“:

Select the type of platform you will be installing the runner in, and it will give you a list of commands you can use, to download and configure it. Sweet!

I used that process to install 5 separate self-hosted runners:

• A Raspberry Pi 4, running Manjaro Linux 64-bit
• A Raspberry Pi 4, running RPI-OS 32-bit
• A Raspberry Pi 4, running RPI-OS 64-bit
• A RockPro64, running Manjaro Linux 64-bit
• An Intel Mac-Mini, running macOS Catalina

All devices already had the Amiberry requirements installed, to make sure they can compile it without problems.

I installed the runners, configured them to talk to my Github repository, set them up to run automatically as services and checked that they showed “Green” from Github’s side once they were up and running.

Coming in the next chapter: Time to write some content for those workflows!

Note: if you just want to skip ahead and see the result, the repository already contains everything: https://github.com/midwan/amiberry/blob/master/.github/workflows/c-cpp.yml