Close evidence-guard gaps: wire npm packaging tests into dist-validation, correct stale OSTW oracle CI claims - #203
Merged
Merged
Conversation
#183 moved the npm tarball determinism/executable-bit regressions (#123) from compatibility/tests to scripts/tests and declared them part of distribution validation, but no CI step ever executed them. Add the unittest discover step to the dist-validation matrix (Python and Node are already set up there, and the suite exercises npm pack end-to-end when available). Verified locally: 7 tests pass in ~0.6s.
PR #177 deliberately removed the ostw-reference CI job (upstream .NET oracle replay is evidence infrastructure owned by del-rs, not a Wright merge gate), but two documents still claimed the run_oracle.py drift check guards CI. State the reality: the drift check is a manual maintainer command, oracle reproducibility workflows belong to del-rs (del-rs#49, tracked in Wright by #182), and Wright CI consumes the recorded evidence via the #119 differential without re-deriving it.
The #123 regression suite had never executed on Windows (it was unwired until the previous commit). CI on windows-latest showed the staged exec-less bin files carry tar mode 0o666 there instead of POSIX 0o644, so the literal 'mode 0o644' assertion failed while the production verify_tarball rejection worked correctly. Assert the stable observable contract instead: the rejection names the offending bin file and reports the observed mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Entropy-audit follow-up (Audit candidate 2): two declared evidence guards did not match reality.
dist-validation. chore(compat): remove duplicated language-oracle infrastructure #183 moved the [Distribution correctness] Make npm package-mode validation portable on Windows #123 npm tarball determinism/executable-bit regressions fromcompatibility/teststoscripts/testsand stated they would "run through distribution validation", but no CI step ever executed them — the suite was dead weight since 2026-08-19. This adds the missingunittest discoverstep to the dist-validation matrix (3 OSes), which already sets up Python 3.12 and Node.js 22 and runs the adjacentverify-dist.py/test-npm.pysteps.ostw-referenceCI job (upstream .NET oracle replay is evidence infrastructure owned bydel-rs, not a Wright merge gate), butdocs/ostw/compatibility-baseline.mdandcompatibility/ostw/README.mdstill claimed therun_oracle.pydrift check "guards CI". The docs now state the reality: the drift check is a manual maintainer command, future oracle reproducibility workflows belong todel-rs(del-rs#49, tracked in Wright by Track removal of residual OSTW owner-style evidence after deltin-rs migration #182), and Wright CI consumes the recorded evidence via the [M13] Compile the accepted OSTW corpus slice to Workshop with reference differential validation #119 compile differential without re-deriving it.test_verify_tarball_still_rejects_non_executable_binfailed on a literalmode 0o644assertion: Windows-staged exec-less files carry tar mode0o666, and the productionverify_tarballrejection is correct on both platforms. The assertion now checks the stable observable contract (rejection names the offending bin file and reports the observed mode) instead of Unix-specific mode digits.Why not re-wire
run_oracle.pyinto CI insteadPR #177 made an explicit ownership decision: Wright PRs must not acquire and execute the pinned upstream .NET OSTW runtime, and oracle reproducibility workflows should be owned by
del-rsas maintainer/evidence infrastructure. Re-adding the gate would reverse that decision; the remaining problem was only the stale doc claims, which this PR fixes.Evidence / validation
All results below come from CI run on this PR (run 32558927508 + the re-run for the last commit) and local branch runs:
python3 -m unittest discover -s scripts/tests—Ran 7 tests, OK(local, both before and after the assertion fix)mode 0o644literal — the defect the unwired suite had been hiding; ubuntu/macos passed. After the fix, all three OSes pass.python3 -m unittest discover -s compatibility/tests—Ran 2 tests, OK(untouched)python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"— parsesgit diff --check— cleangrep -rn "guards CI|CI drift guard" --include="*.md"— no remaining stale claimsTesting-policy notes
0o644→ platform-agnostic mode assertion) does not change the protected contract — verify_tarball must reject non-executable bin entries — which remains asserted; the removed literal encoded a Unix-specific incidental, evidenced by the Windows CI run observing0o666with a correct rejection.Known gaps that remain visible (not hidden)
compatibility/ostw/{corpus,reference,results}.json,probes/) has no automated re-derivation guard in Wright CI by design (ci: remove legacy OSTW reference baseline gate #177);compatibility/tests/test_evidence.pycoverscompatibility/fixtures/self-consistency only. Durable ownership of that evidence is tracked by del-rs#49 / Track removal of residual OSTW owner-style evidence after deltin-rs migration #182.scripts/inventory-ostw-corpus.pyremains a zero-reference manual tool in the same OSTW evidence area; out of scope here.