[clhodapp]

A check of a volatile category depends on something outside the build, such as a service reached over the network. This PR adds the function that builds the derivation of such a check, and it makes a caisson top refuse a check of a volatile category whose derivation cannot reach the network.

lib.caisson.mkVolatileCheck

caisson.checks.mirror-answers = {
  category = "online";
  derivation = lib.caisson.mkVolatileCheck {
    inherit pkgs;
    name = "mirror-answers";
    program = lib.getExe mirrorProbe; # a pkgs.writeShellApplication, for instance
    # impureEnvVars = [ "MIRROR_TOKEN" ];  (optional)
  };
};

The function returns a fixed-output derivation, the kind of derivation Nix lets reach the network.

• The builder is the bash of pkgs, run with no PATH. It runs the program. When the program exits with status 0, the builder writes the pass marker, a file holding the line pass. The output hash is the flat sha256 of that file, so every marker declares the same hash.
• The name is <name>-<hash>. <hash> is the 32-character hash part of the store path of the program. A changed program therefore gets a new marker path, and the same program keeps its marker path. impureEnvVars leave the marker path unchanged.
• program is an executable in the Nix store. It can be a derivation that is an executable file, a string naming an executable inside a store path (lib.getExe), or a path. A string outside the store is refused. The builder sets no PATH, so the program has to carry the commands it runs, as writeShellApplication does with runtimeInputs.
• The function returns a plain derivation, so it keeps drvAttrs. topside reads the inputs of the marker from builtins.getContext (builtins.toJSON drv.drvAttrs). The context holds the program and the bash.

A pass leaves the marker in the store, and a failure leaves nothing.

Refusal

The check record of a caisson top refuses a check of a volatile category when the drvAttrs of its derivation declare no outputHash: reading the volatile field of its entry fails, and the other fields still read. topside check reads that field for every check it runs. The refusal accepts any fixed-output derivation, not only those mkVolatileCheck returns.

The refusal lives in the record and not in the checks output, because of its evaluation cost. Reading whether a category is volatile evaluates the checkCategories submodules that declare it, and on main the checks output reads no property of a category. Refusing in the checks output made the eval-weight check fail locally: moduleSystemEvals rose from 28 to 30 in minimal-consumer-total and from 11 to 13 in caisson-overhead. With the refusal in the record, eval-weight passes at the committed baseline. The probes in the pinned-world suite whose categories are volatile (nas-1-boots and the artifact check of the stub integration) are now fixed-output derivations.

Tests

• pinned-world volatileCheckNamesItsMarkerForTheProgram: the name and output path change with the program and stay the same for the same program, with or without impureEnvVars. The test also covers the lib.getExe form, the fixed-output attributes, system, the context of drvAttrs, and the refusal of a program outside the store.
• pinned-world volatileChecksAreFixedOutputDerivations: a hermetic derivation under a volatile category is refused in the volatile field of its record entry, while its category, its requiredFeatures and the checks output still read. The derivation mkVolatileCheck returns is accepted. The unit test of a declared volatile category now gives its check a fixed-output drvAttrs.
• tests/volatile-checks/check.sh, run by the new volatile-checks job of the check workflow, builds checks in the real store. A passing check writes its marker at the declared path, holding pass, and nix path-info finds it. Building it again with --rebuild keeps the marker. A failing check fails to build, and nix path-info finds no marker at its output path. The script runs outside the sandbox, because a check of nix flake check cannot run nix build.

Locally: the pinned-world check evaluates, tests/volatile-checks/check.sh passes, and nix flake check passes.

tests/check.sh of topside#14 passes against this branch, with the hand-written passMarker of its fixture replaced by mkVolatileCheck. The fixture adds its probe builder variable to the drvAttrs the helper returns. The topside script ran in a local clone, and nothing was pushed to topside.

🤖 Generated with Claude Code