Skip to content

feat(REQ-30): no vacuous oracles — negative controls + standing CI gate (#258) - #262

Merged
avrabe merged 1 commit into
mainfrom
feat/req-30-vacuous-oracles
Aug 21, 2026
Merged

feat(REQ-30): no vacuous oracles — negative controls + standing CI gate (#258)#262
avrabe merged 1 commit into
mainfrom
feat/req-30-vacuous-oracles

Conversation

@avrabe

@avrabe avrabe commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Closes the gaps varve reported in #258 and adds a standing CI gate so the class can't recur.

The problem (varve, #258)

The airgapped e2e test is named sign-then-verify but never calls verify_signature"the check ran, the check passed, and the check could not see the thing." An audit found this shape in several places, and nothing prevented new instances.

Negative controls (each mutation-verified — break the guard → the test fails)

  • verifier.rs — three negatives through the public verify_signature: fail-closed no-time-source (verifier.rs:227, previously zero coverage), hash-mismatch, tampered-signature. verify_crypto was already covered (REQ-23); the public wrapper was positive-only.
  • provisioning/ca.rstest_certificate_chain_validation now drives the real OfflineVerifier (webpki) with a positive case and a forged-issuer-DN negative, instead of comparing DN strings.

⚠️ Discovered defect (the vacuous test concealed it)

webpki verify_for_usage with KeyUsage::required(codeSigning) enforces RFC-5280 EKU nesting, so a PrivateCA intermediate without the CodeSigning EKU could not sign a device cert this module's own OfflineVerifier would accept — the exact flow verification.rs documents. Fixed by adding CodeSigning EKU to intermediate CAs. Behavioral change: device chains issued through a PrivateCA intermediate now validate offline. (CHANGELOG for v0.12.0.)

Honest-name / non-swallowing fixes

  • airgapped_e2e.rs: the CI-running test_bundle_fetch_and_parse no longer launders a network failure to green; the OIDC structure-only test is renamed honestly.
  • fulcio.rs: extracted certificate_from_response so the empty-chain guard is unit-testable; the tautology test now drives it.
  • verify-core keys.rs: the assertion-free test_secret_key_no_clone#[ignore]d TODO (a runtime test can't observe a missing trait impl) rather than a false green.

Standing gate — verification/vacuous-oracle-check/check.py (wired as vacuous-oracle-gate, no continue-on-error)

  • Fails CI when a workflow test selector matches zero targets (kani --harness, -p <pkg>, --test <target>) — the class that let a kani pkg:wsc entry and a coverage -p miss go green earlier this cycle.
  • Fails CI when a lib-bearing workspace crate is omitted from cargo llvm-cov and not in COVERAGE_EXEMPT.
  • Coverage fix: added -p wsc-verify-core -p wsc-attestation to llvm-cov (the omission the gate now flags).

Evidence

  • Tests: wsc lib 609✓/3 ignored, airgapped_e2e 5✓/2, wsc-verify-core 165✓/1.
  • Gate: clean → exit 0; coverage-omission → exit 1; bogus-selector → exit 1.
  • Mutation table (pass→fail→pass) for all four negatives verified locally.

Part of the v0.12.0 varve-interop line (plan: #261). Refs #258.

🤖 Generated with Claude Code

…te (#258)

varve (#258) observed that the airgapped e2e test is named for sign-then-
verify but never calls verify_signature — "the check ran, the check passed,
and the check could not see the thing." An audit found this shape in several
places and, more importantly, that nothing prevented new instances. This
closes the concrete gaps and adds a gate so they fail CI.

Negative controls (each mutation-verified: break the guard -> test fails):
- verifier.rs: three negatives through the PUBLIC verify_signature entry —
  fail-closed no-time-source (verifier.rs:227, previously ZERO coverage),
  hash-mismatch, tampered-signature. The private verify_crypto was already
  covered (REQ-23); the public wrapper's gates were positive-only.
- provisioning/ca.rs: test_certificate_chain_validation now drives the REAL
  OfflineVerifier (webpki) with a positive case AND a forged-issuer-DN
  negative control, instead of comparing distinguished-name strings.

  DISCOVERED DEFECT (the vacuous test concealed it): webpki verify_for_usage
  with KeyUsage::required(codeSigning) enforces RFC-5280 EKU nesting, so a
  PrivateCA intermediate without the CodeSigning EKU could not sign a device
  cert this module's own OfflineVerifier would accept — the exact flow
  verification.rs documents. Fixed by adding CodeSigning EKU to intermediate
  CAs (create_signed_cert). Behavioral change: device chains issued through a
  PrivateCA intermediate now validate offline. CHANGELOG-worthy for v0.12.0.

Honest-name / non-swallowing fixes:
- airgapped_e2e.rs: the CI-running test_bundle_fetch_and_parse no longer
  launders a network failure to green; the OIDC structure-only test is
  renamed to test_keyless_signature_structure_is_wellformed.
- fulcio.rs: extracted certificate_from_response so the empty-chain guard is
  unit-testable; the tautology test now drives it and asserts the specific
  FulcioError.
- verify-core keys.rs: the assertion-free test_secret_key_no_clone is now an
  #[ignore]d TODO (a runtime test cannot observe a missing trait impl; needs
  a compile_fail case) rather than a false green.

Standing gate (verification/vacuous-oracle-check/check.py, wired into
rust.yml as vacuous-oracle-gate, no continue-on-error):
- Fails CI when a workflow test selector matches zero targets (cargo kani
  --harness, -p <pkg>, --test <target>) — the class that let a kani pkg:wsc
  entry and a coverage -p miss go green earlier this cycle.
- Fails CI when a lib-bearing workspace crate is omitted from cargo llvm-cov
  and not in COVERAGE_EXEMPT.
- Coverage fix (rust.yml): added -p wsc-verify-core -p wsc-attestation to
  llvm-cov (the omission the gate now flags). wsc-component is exempted
  (cdylib, all code #[cfg(target_arch=wasm32)], nothing host-coverable).

Tests: wsc lib 609 pass/3 ignored, airgapped_e2e 5/2, wsc-verify-core 165/1.
Gate: clean exit 0; coverage-omission exit 1; bogus-selector exit 1.

Refs: #258
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aR3Md1h46K9wAUWMQiESH
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.41096% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/lib/src/airgapped/verifier.rs 88.88% 3 Missing ⚠️
src/lib/src/signature/keyless/fulcio.rs 83.33% 3 Missing ⚠️
src/verify-core/src/signature/keys.rs 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@avrabe
avrabe merged commit 0a30ee8 into main Aug 21, 2026
22 checks passed
@avrabe
avrabe deleted the feat/req-30-vacuous-oracles branch August 21, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant