Testing

The core mise run e2e task runs 9 NixOS VM integration tests that validate the RAUC update lifecycle, network security, and rollback behavior. Additional provisioning and forensics checks are also available directly under the flake checks.* outputs. Tests run on both Linux (TCG software emulation) and macOS (Apple Virtualization Framework).

Running Tests

Provisioning package

cd scripts/atomixos_provision
uv run --extra dev pytest
uv run --extra dev ruff check .

These tests cover the Litestar API, SSH-signature auth helpers, config parsing, bundle import, Quadlet rendering/sync, activation, job tracking, and service foundation modules.

All tests

mise run e2e

# Run all tests inside a Lima VM
mise run e2e --lima
mise run e2e --lima --vm my-builder

Individual tests

mise run e2e:rauc-slots          # RAUC sees all 4 A/B slots after boot
mise run e2e:rauc-update         # Bundle install writes to inactive slot pair, slot switches A->B
mise run e2e:rauc-rollback       # Install to B, mark bad, verify rollback to A
mise run e2e:rauc-confirm        # os-verification health checks pass, slot marked good (~3 min)
mise run e2e:rauc-power-loss     # Crash VM mid-install, verify slot A intact after reboot
mise run e2e:rauc-watchdog       # Freeze systemd to trigger watchdog, verify boot-count rollback
mise run e2e:firewall            # 2-node test: WAN allows HTTPS/VPN, LAN allows SSH/DHCP/NTP
mise run e2e:network-isolation   # 2-node test: LAN gets DHCP/NTP, cannot reach WAN
mise run e2e:ssh-wan-toggle      # Flag file enables/disables SSH on WAN via nftables reload

# Run an individual test inside Lima
mise run e2e:rauc-slots --lima

Test Descriptions

Platform Performance

The mise task wrappers auto-detect the platform and select the correct flake output.

The rauc-confirm test has the smallest speedup because most of its runtime is a fixed 60-second sustained health check timer.

Interactive Debugging

Bundle Test VM

Use vm:bundle-test to boot an interactive AtomixOS VM for exercising real config.toml bundles without physical hardware:

mise run vm:bundle-test

Each launch uses a fresh temporary VM disk image. The VM runner is still built through Nix and reused from the store when inputs have not changed, but runtime state from previous bundle tests is discarded when the VM exits.

The VM uses the QEMU hardware profile with eth0 as WAN and a second virtio NIC as LAN. Host ports are forwarded for common operator workflows:

Build the runner without launching it:

mise run vm:bundle-test --build-only

Apply a bundle from the host:

tar --zstd -cvf config.tar.zst -C example/caddy-oidc .
curl -F config_file=@config.tar.zst http://127.0.0.1:8080/apply

For domain-based Caddy examples, map the example domain to localhost while testing from the host:

curl -k --resolve gateway.example.com:8443:127.0.0.1 \
  https://gateway.example.com:8443/cockpit/

For browser testing, add a local hosts entry and include the forwarded HTTPS port in the URL:

127.0.0.1 gateway.example.com

Then browse to https://gateway.example.com:8443/.

The Cockpit container generates both the real device origin and the VM forwarded HTTPS origin from GATEWAY_DOMAIN because Cockpit accepts space-separated origins.

SSH is key-only and uses the provisioned /data/config/ssh-authorized-keys state. On a fresh VM, use the serial console or apply a bundle containing an admin key before expecting ssh -p 10022 admin@127.0.0.1 to succeed.

Launch an interactive QEMU VM with a Python REPL:

# Debug the default test (rauc-slots)
mise run e2e:debug

# Debug a specific test
mise run e2e:debug -t update
mise run e2e:debug -t confirm
mise run e2e:debug -t watchdog

# Keep VM state between runs
mise run e2e:debug -t slots --keep

Available test short names: slots, update, rollback, confirm, power-loss, watchdog, firewall, net-iso, ssh-toggle.

Inside the REPL:

gateway.start()                          # boot the VM
gateway.wait_for_unit("multi-user.target")
gateway.succeed("rauc status")           # run a command
gateway.shell_interact()                 # drop into a root shell
gateway.screenshot("name")              # save a screenshot
# Ctrl+D to exit

Running Tests with Nix Directly

# Linux (TCG, no KVM required)
nix build .#checks.aarch64-linux.rauc-slots --no-link -L

# macOS (requires nix-darwin with linux-builder enabled)
nix build .#checks.aarch64-darwin.rauc-slots --no-link -L

# Local Darwin eval/builds that depend on nix/tests/rauc-qemu-config.nix should
# use a path flake ref so local files remain visible even if they are untracked.
nix build "path:$PWD#checks.aarch64-darwin.rauc-slots" --no-link -L

When iterating on a single Darwin check locally, evaluate and build the exact derivation with the same path: flake ref:

drv=$(nix eval --raw "path:$PWD#checks.aarch64-darwin.rauc-slots.drvPath")
nix-store -r "$drv"

Test Architecture

Tests use the NixOS test framework (nixos-lib.runTest). Each test:

1. Defines one or two virtual machines with the full AtomixOS service stack (using hardware-qemu.nix instead of hardware-rock64.nix)
2. Boots the VM(s) and runs a Python test script that interacts via QEMU’s monitor interface
3. Asserts on command output, service states, and network behavior

The QEMU target uses a custom RAUC backend that simulates U-Boot’s slot selection using files instead of environment variables, allowing the full A/B update lifecycle to be tested without real hardware. The shared slot mapping for the RAUC tests lives in nix/tests/rauc-qemu-config.nix.