Re-pin the CI verification contract to the shard-matrix gate - #26
Closed
Parad0x-Labs wants to merge 1 commit into
Closed
Parad0x-Labs wants to merge 1 commit into
Parad0x-Labs wants to merge 1 commit into
Conversation
The suite still pinned the pre-migration single-job ops/verify.py gate: the 'Run authoritative verification' step it demands has not existed since the workflow became the ten-shard matrix + routed macOS job (owner commits 9fd7ee7/006d874), so the gate case failed at the missing step and the collect_only/step_disabled sabotage cases crashed mutating a step that is not there. The ruff pin still asserted 0.15.16, two versions behind the 0.16.7 the tree adopted with dependabot PR #5 and that test_install_surface_contracts.py already pins. The weakening law is restated against the current structure: exact lint and manifest-collection commands in the verify job, the full ten-shard matrix, the exact shard pytest prefix consuming the resolver's own file list, unconditional hidden-file log uploads (PR #11's law) on both test jobs, build gated on all three, and the same no-disabled-step, no-swallowed-output scan. The sabotage cases mutate the shard RUN step now -- a shard weakened to --collect-only executes nothing. 4 CI failures (main run 35570948370, every PR run) -> 6 passed; install-surface and delivery contracts unchanged and green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What failed
tests/test_ci_verification_contract.py— 4 of 6 cases red in the main CI census (run35570948370) and every PR run since; deterministic locally:
test_push_and_pr_ci_use_the_exact_authoritative_gate— demands a verify-job step named"Run authoritative verification" running
ops/verify.pytest_contract_mutations_are_rejected[collect_only],[step_disabled]— the sabotagemutations crash mutating that same nonexistent step
test_verification_dependencies_are_exactly_pinned— assertsruff==0.15.16; the treeadopted
ruff==0.16.7with dependabot PR Bump ruff from 0.15.16 to 0.16.7 #5 (andtests/test_install_surface_contracts.pyalready pins 0.16.7 — the two contract files had drifted apart)
Root cause
The suite pins the pre-migration single-job gate. The workflow it describes — one verify
job running
ops/verify.py --workers 4 …over the whole suite — was replaced by the owner'sshard architecture (commits
9fd7ee7,006d874,ca9eb4d, plus merged PR #11'shidden-file artifact uploads): a verify job that lints with the pinned ruff and produces the
canonical pytest collection, a ten-shard Linux matrix where every collected file runs in
exactly one shard, a routed macOS job for the macOS-only suites, and build gated on all
three. The suite was never re-pinned; standing failure since those workflow changes.
Repair (test-only; the workflow is untouched — it is what main authorizes)
The weakening law the old suite enforced, restated against the current structure:
paths-ignore; noifon the verify/tests/macos jobs;exactly one
setup-pythonper job pinned to3.12.13python -m ruff check .and exactpython ops/pytest_manifest.py … -- -qcommands, no step-levelifneeds: verify, the fullshard: ["0"…"9"]matrix (a dropped shard is a droppedtenth of the suite), the exact shard pytest prefix, and the run consuming exactly the
resolver's own
shard-<N>-files.txt.verification-logs/unconditionally withinclude-hidden-files: true(PR Keep one collector per directory when pytest re-collects a parent package #11's law — without it the uploads are empty)needs: [verify, tests, macos]; global nocontinue-on-error, no|| true/| tail/| teepytest==9.1.0,ruff==0.16.7(in agreement with the install-surface contract)The sabotage mutations now target the shard RUN step — the modern equivalent of weakening
the gate (
--collect-onlyappended executes nothing; a disabled shard step skips a tenth ofthe suite).
Validation
tests/test_install_surface_contracts.py+tests/test_delivery_contracts.py(the macOS-list coupling) — 18 passed, unchanged.
ruff check .clean (pinned 0.16.7).Base: main
8153a96. Head:mission/ci-contract-repin. One file changed(
tests/test_ci_verification_contract.py, +76/−27).