Skip to content

A guard that refuses reads as an ordinary pass in CI #5

Description

@lucy-wolfe

The problem

A repo guard can refuse — decline to check something, and say why — instead of passing or failing. That distinction is invisible in CI.

The guard shard runner prints one line per file. A file whose subtests were skipped prints exactly what a fully-passing file prints:

PASS [parked-suite-triage.test.mjs] (194ms)

Run that guard directly and you see the reason:

CANNOT CHECK: the snapshot corpus commit de72d660 is not in this repository

Read CI and it is an ordinary pass.

Why it matters

This project's standing doctrine is that a check which looks like it covered you is worse than one that is absent, because a green nobody can interpret is a green people stop reading. A refusal is honest by design — but only if somebody sees it, and CI is where people look.

Concretely: parked-suite-triage runs 15 of its 17 rules against the live tree and refuses 2 that need a commit from the pre-publication history. That is the correct behaviour. Nothing in CI says so.

The fix

Runner-side, not guard-side. The shard runner's per-file line gains the counts, and echoes the first skip message when there is one:

parked-suite-triage: 15 pass, 2 SKIPPED — CANNOT CHECK: the snapshot corpus commit de72d660 is not in this repository

One change to the reporter makes every guard's refusals visible, now and in future.

The guard-side alternative — each guard printing its refusal more loudly — is rejected deliberately: it needs every guard to adopt a convention, and it puts presentation in the wrong layer.

Acceptance

  • A guard with skipped subtests is distinguishable from a fully-passing one in the CI log.
  • The reason is present, not just the count.
  • A fully-passing guard's output does not get noisier.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions