feat(REQ-30): no vacuous oracles — negative controls + standing CI gate (#258) - #262
Merged
Conversation
…te (#258) varve (#258) observed that the airgapped e2e test is named for sign-then- verify but never calls verify_signature — "the check ran, the check passed, and the check could not see the thing." An audit found this shape in several places and, more importantly, that nothing prevented new instances. This closes the concrete gaps and adds a gate so they fail CI. Negative controls (each mutation-verified: break the guard -> test fails): - verifier.rs: three negatives through the PUBLIC verify_signature entry — fail-closed no-time-source (verifier.rs:227, previously ZERO coverage), hash-mismatch, tampered-signature. The private verify_crypto was already covered (REQ-23); the public wrapper's gates were positive-only. - provisioning/ca.rs: test_certificate_chain_validation now drives the REAL OfflineVerifier (webpki) with a positive case AND a forged-issuer-DN negative control, instead of comparing distinguished-name strings. DISCOVERED DEFECT (the vacuous test concealed it): webpki verify_for_usage with KeyUsage::required(codeSigning) enforces RFC-5280 EKU nesting, so a PrivateCA intermediate without the CodeSigning EKU could not sign a device cert this module's own OfflineVerifier would accept — the exact flow verification.rs documents. Fixed by adding CodeSigning EKU to intermediate CAs (create_signed_cert). Behavioral change: device chains issued through a PrivateCA intermediate now validate offline. CHANGELOG-worthy for v0.12.0. Honest-name / non-swallowing fixes: - airgapped_e2e.rs: the CI-running test_bundle_fetch_and_parse no longer launders a network failure to green; the OIDC structure-only test is renamed to test_keyless_signature_structure_is_wellformed. - fulcio.rs: extracted certificate_from_response so the empty-chain guard is unit-testable; the tautology test now drives it and asserts the specific FulcioError. - verify-core keys.rs: the assertion-free test_secret_key_no_clone is now an #[ignore]d TODO (a runtime test cannot observe a missing trait impl; needs a compile_fail case) rather than a false green. Standing gate (verification/vacuous-oracle-check/check.py, wired into rust.yml as vacuous-oracle-gate, no continue-on-error): - Fails CI when a workflow test selector matches zero targets (cargo kani --harness, -p <pkg>, --test <target>) — the class that let a kani pkg:wsc entry and a coverage -p miss go green earlier this cycle. - Fails CI when a lib-bearing workspace crate is omitted from cargo llvm-cov and not in COVERAGE_EXEMPT. - Coverage fix (rust.yml): added -p wsc-verify-core -p wsc-attestation to llvm-cov (the omission the gate now flags). wsc-component is exempted (cdylib, all code #[cfg(target_arch=wasm32)], nothing host-coverable). Tests: wsc lib 609 pass/3 ignored, airgapped_e2e 5/2, wsc-verify-core 165/1. Gate: clean exit 0; coverage-omission exit 1; bogus-selector exit 1. Refs: #258 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aR3Md1h46K9wAUWMQiESH
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the gaps varve reported in #258 and adds a standing CI gate so the class can't recur.
The problem (varve, #258)
The airgapped e2e test is named sign-then-verify but never calls
verify_signature— "the check ran, the check passed, and the check could not see the thing." An audit found this shape in several places, and nothing prevented new instances.Negative controls (each mutation-verified — break the guard → the test fails)
verifier.rs— three negatives through the publicverify_signature: fail-closed no-time-source (verifier.rs:227, previously zero coverage), hash-mismatch, tampered-signature.verify_cryptowas already covered (REQ-23); the public wrapper was positive-only.provisioning/ca.rs—test_certificate_chain_validationnow drives the realOfflineVerifier(webpki) with a positive case and a forged-issuer-DN negative, instead of comparing DN strings.webpki
verify_for_usagewithKeyUsage::required(codeSigning)enforces RFC-5280 EKU nesting, so aPrivateCAintermediate without the CodeSigning EKU could not sign a device cert this module's ownOfflineVerifierwould accept — the exact flowverification.rsdocuments. Fixed by adding CodeSigning EKU to intermediate CAs. Behavioral change: device chains issued through aPrivateCAintermediate now validate offline. (CHANGELOG for v0.12.0.)Honest-name / non-swallowing fixes
airgapped_e2e.rs: the CI-runningtest_bundle_fetch_and_parseno longer launders a network failure to green; the OIDC structure-only test is renamed honestly.fulcio.rs: extractedcertificate_from_responseso the empty-chain guard is unit-testable; the tautology test now drives it.keys.rs: the assertion-freetest_secret_key_no_clone→#[ignore]d TODO (a runtime test can't observe a missing trait impl) rather than a false green.Standing gate —
verification/vacuous-oracle-check/check.py(wired asvacuous-oracle-gate, nocontinue-on-error)kani --harness,-p <pkg>,--test <target>) — the class that let a kanipkg:wscentry and a coverage-pmiss go green earlier this cycle.cargo llvm-covand not inCOVERAGE_EXEMPT.-p wsc-verify-core -p wsc-attestationto llvm-cov (the omission the gate now flags).Evidence
wsclib 609✓/3 ignored,airgapped_e2e5✓/2,wsc-verify-core165✓/1.Part of the v0.12.0 varve-interop line (plan: #261). Refs #258.
🤖 Generated with Claude Code