Github Actions

Motivation

Why I’m writing a short post about Github Actions? Because I wanted to use it for my latest Software-project, QR-codengrave. When at some distant point in the future, I have a new computer, don’t remember which IDE I used to build and run, test, and create assets with. Or, say, I have a corrupted virtual environment or launch.json, I still want to be able to deploy that application, publish bugfixes or bump a release.

Although that’s not the usual argument for using CI/CD - typically the first reason being that collaboration on a piece of software becomes easier - still the appeal is strong enough to try Github’s automation called Github Actions.

I won’t try to run nightlys or have a continuous deployment pipeline as an end in itself, though, and only want an automated build, lint, test run, and deploy stuff when merging back to the production branch.

Got it, but why a dedicated post?

Because it was such a pain to get it right. I spent hours and hours pushing and hoping that, this time, the goddess of Github Actions would actually have my solution built without errors and at least run some of the tests.

Note: If I had been more diligent, I would have installed yet another tool like this to have my actions run locally, both saving time and the embarassment of tens of failed builds in a row. But I wasn’t.

I had quite some errors that appeared on the way of making QR-codengrave. Some I wasn’t even able to properly resolve so I had to utilize work-arounds. But let’s get to that later.

The script

Github Actions uses YAML to take orders for its pipelines. It is very well documented and provides many readily-working scripts for most scenarios and programming languages.

The so-called “Workflow file” looks like this:

# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: python_integrate

on:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]

permissions:
  contents: read

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python 3.10
      uses: actions/setup-python@v3
      with:
        python-version: "3.10"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install flake8 pytest
        pip install qrcodegen  # Dependency install of qrcodegen
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with flake8
      run: |
        # stop the build if there are Python syntax errors or undefined names
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
    - name: Test with pytest
      uses: GabrielBB/xvfb-action@v1  # Diverts tkinter GUI to a virtual frame buffer (VFB)
      with:
        run: |
          pytest

And it does not divert too much from the standard template that I was using first. The only things I added are the run pytest entry and the usage of a virtual frame buffer to cirumvent issues with my GUI which makes the tests fail when the GUI tries to fire up and there’s no screen to show it on.

Github Action runner errors

The following list of fails describe the errors and resolutions that I faced until I had a stable CI at around action run #50.

Directory mismatch?

Output:

/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_machinify_vector.py:4: in <module>
    from bin.platform.machinify_vector import MachinifyVector, Tool, EngraveParams
E   ModuleNotFoundError: No module named 'bin'

Reason: I placed my source files under /bin and made it available to a local instance of PyInstaller (the tool I chose to render my python sources into an executable under Windows). Upon upload, Github Actions couldn’t relate to the paths I chose and had that error prepared for me.

Resolution: Add an empty file with name __init__.py into the /bin folder. This will flag anything in that folder as package, thus making it available to GH actions. So what looked like a directory mismatch actually was a package-not-found error that my IDE didn’t have as it knew which files I had created.

YAML syntax errors

This error occurs because I was trying to put two workflows into one file. Somehow Github Actions seems to only accept a single one per file.

Path-not-found errors

I had a lot of those and they were painful to resolve. The folder structure that I was targeting looked like:

- assets  # images, persistence file, etc.
- src  # source files
- test  # pytest files for unit and integration testing
- dist  # build artifacts
- build  # build process files

But somehow, it was super hard to make my local IDE build environment, Pyinstaller, and the Github Actions Worker and its trigger of Pyinstaller to cooperate. One of those four parties would always complain that a path is missing or something else was wrong. That’s why I decided to abandon relative paths and instead use python’s importlib_resources. But in the end, this turned out to be problematic with Pyinstuller on remote Github Actions.

What finally solved the problem was to move assets into src. This way, the ./ command couldn’t go wrong anywhere and although I don’t like the construct very much, I was tired of putting more time into that for a clean fix (If you know how to handle path pointing in Python for both local and remote, let me know in the discussions).

Tkinter headless testing

Later in the development process, I decided to add some integration tests into my solution. This way I wanted to make sure that child windows would actually perform the callbacks to update the main application, and vice versa e.g. that the persisted tool list is cascaded into the tool configuration window.

These tests were running locally but with a disadvantage: When I simulated an error or warning case, the corresponding messageBox would insist to be closed manually by the user before continuing the tests. Which was just a bit annoying for me in the IDE to do, but would just be impossible to do from within Github Action’s CI pipeline.

So I bit the bullet and added a wrapper class to tkinter’s messageBox so I could inject a mock that wouldn’t actually trigger the popup:

class MsgBox:
    """Re-implementation due to testing purposes: With this trick, we are able to mock these windows
    so we do not have to wait for users to manually close the dialog, unblocking the application again."""
    def showinfo(self, title, message):
        showinfo(title=title, message=message)

    def error(self, title, message):
        showerror(title=title, message=message)

Although in theory it should be possible also to invoke the popup’s “OK button” from within the test, I wasn’t able to have that automated reliably.

When this worked, I pushed to Github with high expectations - and got another beautiful error:

_tkinter.TclError: no display name and no $DISPLAY environment variable

At least that was easy to understand: Tkinter just didn’t know where to draw the windows - seems legit when there’s no display connected.

So I searched the internet and found a single line of code that miraculously solved my problem by introducing a virtual frame buffer:

uses: GabrielBB/xvfb-action@v1 # Diverts tkinter GUI to a virtual frame buffer (VFB)

Now, on Push, Github Actions rained green ticks down on me which felt really good for a change.