[clhodapp]

Step 5 of the checks work: the nix-unit integration.

The integration

A nix-unit configuration names a file of unit tests and the nix-unit program that runs them:

# configs/flake/checks/default.nix, a flake-parts configuration
caisson.nix-unit.configurations.lib = lib.caisson.nix-unit.mkConfiguration { };

# configs/nixUnit/lib/default.nix, the module registered under that name
{ ... }:
{ ... }:
{
  tests = ../../../tests/unit;
}

The file, or the default.nix of a directory, is a function of { lib, manifest, dependencies } that returns nix-unit's suite. lib is the library that a module of the configuration that declares the nix-unit configuration receives. manifest is the manifest of that library, and dependencies are the pinned dependencies of the nix-unit configuration. package defaults to nix-unit of the package set the configuration runs on.

A nix-unit configuration is evaluated per system, once for each system its parent evaluates its children for, and is published in no output attribute set. Its default check unit, of the category unit, is published as <names>-unit, so the configuration above gives the check lib-unit.

How the check runs nix-unit

The derivation runs nix-unit in file mode over a generated file, with a local store under the build directory as nix-unit's flake-parts module sets it up. The file imports lib-overlays/nix-unit/sandbox.nix with a JSON record written at evaluation:

• the store path of every source that the library and the configuration reach (dependencies, projects and the inputs of each flake, recursively), with the source information of each;
• the file every registry entry of the library was registered from (lib overlays, modules, configs, package overlays), and the lib overlays the library applied, in order;
• the name, systems, root and defaultEcosystemSrc names of the library.

In the sandbox, sandbox.nix evaluates each flake again with caisson's callFlake over its inputs. It then calls mkLib of the caisson-core that composed the library, over that record, and calls the test file. No pin reader runs in the sandbox.

Limits of the composition in the sandbox

• The record holds no declaration of a package config, so the library composed in the sandbox keeps the names of its package configs, and a test that reads a package config fails with a message that says so.
• A registry entry given as a value, with no file, and a file outside the Nix store make the check fail at evaluation, with a message that names the entry. A caisson top evaluated as a flake has every file in the store.

caisson's unit tests

They stay on the nix-unit flake-parts module. The published name of a check of this integration is <configuration>-unit, so keeping the name nix-unit would mean naming the configuration nix. The suite also reads a library that tests/unit/flake.nix composes from caisson's source with no name, configs or root, and the input parent of that flake, caisson's source. The library composed in the sandbox would be the library of the caisson top of caisson. Porting therefore changes the name and what the cases evaluate against.

Deviations from the design

• The design hands the integration a file through caisson.checks.<name>.tests. This PR makes the file an option of a nix-unit configuration, caisson.nix-unit.configurations.<name>, as the nixos-test integration does for VM tests.
• Configurations of the class are registered under configs/nixUnit/<name>, because the registry is still keyed by class.
• Package configs are not composed again in the sandbox (above).

Checks

nix-unit-integration (tests/nix-unit) evaluates a flake-parts caisson top for x86_64-linux and aarch64-linux with the nix-unit configurations passing and failing. It asserts that the check record holds passing-unit and failing-unit with the category unit for both systems. It builds passing-unit, whose cases read the library composed in the sandbox, its manifest and a dependency flake. It builds failing-unit with testers.testBuildFailure and finds in its log that the failing case failed and the passing case passed.

Eval weight

A flake that uses caisson composes the nix-unit overlay, and every module evaluation declares caisson.nix-unit.configurations. On main 3b28e96 the caisson-overhead gate measures 5055440 allocated bytes, and on this branch 5114064, above the ceiling of 5084142 (10% over the baseline of 4618224). The last commit records the values measured on this branch as the baseline. The integration adds 58624 bytes (1.2%); the rest of the growth since the previous baseline came from earlier integrations.

Results

Built locally before the rebase onto 3b28e96: every check of checks.x86_64-linux, the VM checks and nix-unit (232/232) among them. After the rebase: nix-unit, nix-unit-integration, pinned-world, registries, eval-weight and treefmt. The check darwin-system of checks.aarch64-darwin evaluates; building it needs a macOS builder, which the darwin job of the workflow has.

🤖 Generated with Claude Code