There is a Github action defined to auto-execute the "can-actually-be-used" test suite on a Debian based system. It's not perfect, and doesn't test every thing possible - but, it's the best we have for now. It verifies the config tools are working and result in a working authentication as well as agent events are triggered. It's intended to be run on a VM you could throw away, not on your work system.

It uses four Github secrets as configuration values to provide host, port, user and password for executing. Your user should use a really good password, since we gonna allow it quite some passwordless sudo requests. The runner is expected to be Debian based, since the scripts rely on apt stuff.

The test suite can't be executed directly on Githubs runners because their kernel doesn't have USB_GADGET support which is required for dummy-hcd/faking removable media. That's why you need to provide your own environment having that kernel option.

The secrets you need to provide, either at personal or organization level (dont use repository, they are contained in forks!), are:

• TEST_RUNNER_HOST_NEW
• TEST_RUNNER_PORT_NEW
• TEST_RUNNER_USER_NEW
• TEST_RUNNER_PASS_2025

(Note that these may change with future commits in case my VM breaks again xD)

CI pipeline notes:

A Gate job runs first on GitHub's free runner (make + make test). The expensive packaging and cross-arch functional test jobs only start if the Gate passes — so a broken build never consumes custom runner resources.

The packaging workflow also runs QEMU cross-arch functional tests for arm64 and armhf, but only on pushes to master (not on PRs). They are non-blocking (continue-on-error: true) — a QEMU failure shows as a warning in the GitHub UI but does not fail the overall workflow. These tests require pre-provisioned golden images on your CI runner (see Cross-architecture QEMU functional tests below). If the images are missing, those jobs fail fast (< 30 s).

Additionally the environment used to run the testsuite should have a nopasswd sudoers config to allow the user used to execute the tests can do the required actions. I'm using this config, saved as /etc/sudoers.d/somefilename:

Defaults  env_keep += "DEBIAN_FRONTEND"

$username $hostname = (root) NOPASSWD: /usr/bin/apt
$username $hostname = (root) NOPASSWD: /usr/bin/pamusb-conf
$username $hostname = (root) NOPASSWD: /usr/bin/pamusb-check
$username $hostname = (root) NOPASSWD: /usr/sbin/modprobe
$username $hostname = (root) NOPASSWD: /usr/sbin/sfdisk
$username $hostname = (root) NOPASSWD: /usr/bin/mount
$username $hostname = (root) NOPASSWD: /usr/bin/umount
$username $hostname = (root) NOPASSWD: /usr/bin/chmod
$username $hostname = (root) NOPASSWD: /usr/bin/sed
$username $hostname = (root) NOPASSWD: /usr/bin/debconf-set-selections
$username $hostname = (root) NOPASSWD: /usr/sbin/pam-auth-update
$username $hostname = (root) NOPASSWD: /usr/bin/systemctl
$username $hostname = (root) NOPASSWD: /usr/bin/tail
$username $hostname = (root) NOPASSWD: /usr/bin/rm
$username $hostname = (root) NOPASSWD: /usr/sbin/mkfs.vfat
$username $hostname = (root) NOPASSWD: /usr/sbin/mkfs.fat
$username $hostname = (root) NOPASSWD: /usr/sbin/mkfs.ext4
$username $hostname = (root) NOPASSWD: /usr/sbin/mke2fs
$username $hostname = (root) NOPASSWD: /usr/sbin/mkfs.exfat
$username $hostname = (root) NOPASSWD: /usr/bin/udevadm

$username and $hostname are placeholders, adjust them to the VM/System you use for testing.

Note on udevadm path: On Ubuntu 22.04+, udevadm lives at /usr/bin/udevadm, not /usr/sbin/udevadm. Using the wrong path means the NOPASSWD rule silently fails to match and sudo falls through to PAM authentication. Verify with which udevadm on your runner.

The test suite has three independent tiers with very different requirements:

Always run unit tests first — they catch most regressions without any special setup.

From the repository root:

make test

This compiles and runs all C unit tests (via cmocka) and all Python unit tests (via pytest) in one shot. To run only one tier:

make test-c       # C unit tests only
make test-python  # Python unit tests only

Prerequisites:

• gcc, make, pkg-config
• libcmocka-dev (or distro equivalent)
• libxml2-dev
• python3 and pytest (python3-pytest or pip install pytest)

No root, no hardware, no kernel modules needed.

The integration tests exercise the full tool chain against a real (emulated) USB device. They require a dedicated user and a throwaway VM or spare machine — do not run them on your daily workstation.

Prerequisites:

• Kernel compiled with CONFIG_USB_GADGET. Verify with:

  grep USB_GADGET /boot/config-$(uname -r)
  

• dkms and kernel headers (to build the dummy-hcd module if not already in your kernel)
• git (for cloning the dummy-hcd source)
• dosfstools (mkfs.vfat, mkfs.fat)
• e2fsprogs (mkfs.ext4)
• exfatprogs (mkfs.exfat) — optional; the exFAT iteration is skipped automatically if the exfat kernel module is unavailable
• systemd (pamusb-agent is managed as a service during tests)
• The sudoers configuration from the Using the test suite when forking section above, with $username and $hostname set to your test user and machine

One-time setup:

cd tests/can-actually-be-used
sudo bash setup-test-requirements.sh

This installs the dummy-hcd kernel module via DKMS, creates two 16 MB virtual USB images, and mounts them. If dummy-hcd is already built into your kernel you can skip the DKMS step and run prepare-mounting.sh, create-image.sh, and mount-image.sh directly instead.

Running the tests:

cd tests/can-actually-be-used
bash run-tests.sh

Each test prints PASSED! on success. The suite is stateful — tests run in a fixed order and each one builds on the configuration created by the previous step.

Cleaning up between runs:

Tests are intended to run in a clean environment. The GitHub Actions workflow handles cleanup automatically; when running locally you must do it yourself. Refer to the action workflow for the exact cleanup steps, or simply revert to a clean VM snapshot before each run.

Availability: Cross-architecture builds and QEMU functional tests are available from v0.9.3 onwards.

The QEMU functional tests boot a full Ubuntu 22.04 Jammy cloud image under QEMU system emulation, install the built .deb, and run the complete run-tests.sh suite (vfat / ext4 / exfat iterations) inside the VM. They cover arm64 and armhf.

In CI these run as FunctionalTest-{arch} jobs inside the packaging workflow, gated on the corresponding Debian-{arch} job passing and reusing its already-built .deb artifact. They only trigger on pushes to master (not PRs) and carry continue-on-error: true, so a QEMU instability does not fail the overall workflow.

Install the required QEMU packages:

sudo apt install \
  qemu-system-arm qemu-efi-aarch64 \
  qemu-efi-arm u-boot-qemu \
  cloud-image-utils

Tests boot from a pre-provisioned golden image rather than a live cloud-init run. This eliminates 10+ minutes of per-run package installation and kernel module compilation.

Key facts:

• Images live in ~/.cache/pam_usb-qemu/ on the runner as jammy-{arch}-provisioned-v{N}.qcow2
• PROVISION_VERSION in tests/can-actually-be-used/run-tests-in-qemu.sh identifies which version is expected
• Images are never auto-provisioned in CI — they must be built manually and committed before pushing
• If the versioned image is absent, the CI job fails within 30 seconds (before any Docker build)

make provision-qemu-images

Runs arm64 then armhf sequentially (parallel provisioning causes QEMU OOM on the runner). Already-current images are skipped instantly (< 1 s each). A full first-time provision takes ~60–90 min per arch; subsequent test-run boots take < 2 min via a throw-away CoW overlay.

If QEMU is killed mid-provision it may leave a partial .qcow2 that the early-exit check treats as valid:

make clean-qemu-images     # removes *.qcow2 + key files; keeps base .img downloads
make provision-qemu-images # reprovisioned from scratch

Each arch has its own PROVISION_VERSION set at the top of its case block in run-tests-in-qemu.sh. Bump only the arch(es) whose environment changed (packages, kernel modules, SSH key, etc.). After bumping:

1. Run make provision-qemu-images on the CI runner
2. Verify tests/can-actually-be-used/run-tests-in-qemu.sh arm64 ... passes locally or on the runner
3. Push

# Build the .deb for the target arch
make build-debian-arm64    # or armhf

# Boot QEMU VM and run full test suite inside it
tests/can-actually-be-used/run-tests-in-qemu.sh arm64 .build/libpam-usb_*_arm64.deb

If the required QEMU binary is not installed the script exits immediately with a clear error and the exact apt install command for that arch.

This is mainly a checklist for myself

• Create issue & PR to prepare release
• Check if Wiki needs update, if yes: do so. When done run make update-other-docs
• If the provisioned QEMU environment changed (packages, kernel modules): bump PROVISION_VERSION in tests/can-actually-be-used/run-tests-in-qemu.sh, then run make provision-qemu-images on the CI runner before merging
• Check & Update Manpages, no need to gz them - release process does that
• Update AUTHORS
• Update CHANGELOG
• Update Debian changelog
• Update Fedora changelog
• Update Arch PKGBUILD versions
• Update version.h
• Update version in pamusb-conf
• Update SECURITY.md version numbers
• Add tag "major.minor.patch" after merging prep PR

• make build-debian
• make deb-sign (APT_SIGNING_KEY is defined in ~/.bashrc)
• Copy the release files to local repo dir
• Run update-repo.sh after switching to repo dir