Skip to content

Bind audit reports to exact inputs - #9

Merged
tmusser merged 15 commits into
mainfrom
agent/bind-audit-reports-to-inputs
Aug 18, 2026
Merged

Bind audit reports to exact inputs#9
tmusser merged 15 commits into
mainfrom
agent/bind-audit-reports-to-inputs

Conversation

@tmusser

@tmusser tmusser commented Aug 18, 2026

Copy link
Copy Markdown
Owner

What changed

  • add deterministic SHA-256 input bindings for public audit_spec() reports
  • bind first-party Chart.audit() reports to their chart contract, data, claim, and installed package version
  • emit bound report schema 0.3 with component hashes plus a bundle hash
  • add matches_spec(...), matches_chart(...), and generic matches_inputs(...) verification helpers
  • canonicalize spec mappings so JSON key order does not change the fingerprint
  • preserve row order, index, column order, dtype metadata, categorical ordering, missing values, infinities, timestamps, and timedeltas in explicit data fingerprints
  • keep wall-clock timestamps out of the content identity so identical inputs reproduce the same binding
  • document the provenance boundary and require agents to re-audit/re-verify after post-audit mutations
  • update CLI and isolated-wheel report-shape assertions for the bound schema

Why

A READY report was previously detached from the exact inputs that produced it. An agent could audit one spec, then edit the spec, data, or claim and still retain a mechanically green report with no intrinsic way to detect that drift.

This PR makes the audit artifact content-addressed. A matching report means the inputs being shared are the same inputs that produced the verdict.

It does not make READY a scientific or human-approval verdict. The hashes prove content identity only.

Behavior

  • unchanged spec/data/claim -> same binding and matches_spec(...) == True
  • spec mutation -> old report no longer matches
  • data mutation -> old report no longer matches
  • claim mutation -> old report no longer matches
  • first-party chart contract mutation -> old report no longer matches
  • equivalent spec mapping key order -> same fingerprint
  • inline spec data remains covered by the spec hash when no explicit data= argument is supplied

Validation

Final GitHub Actions run #189 on head 9c59bf6 passed completely:

  • Python 3.10 test suite + CLI/trap checks: PASS
  • Python 3.11 test suite + CLI/trap checks: PASS
  • Python 3.12 test suite + CLI/trap checks: PASS
  • Python 3.13 test suite + CLI/trap checks: PASS
  • build + twine validation: PASS
  • isolated wheel install: PASS
  • installed CLI report-schema/input-binding smoke check: PASS

CI also caught two useful integration issues during development:

  1. Zero-argument super() in the slotted dataclass report subclass failed on Python 3.10-3.12; the implementation now uses explicit AuditReport base dispatch.
  2. The isolated-wheel smoke test still expected report schema 0.2; it now expects 0.3 and verifies subject/data/claim/bundle hashes plus installed-version equality.

Focused mutation coverage is in tests/test_input_binding.py; CLI report-shape coverage is in tests/test_cli.py and the package workflow.

@tmusser
tmusser marked this pull request as ready for review August 18, 2026 09:12
@tmusser
tmusser merged commit e8b2e20 into main Aug 18, 2026
5 checks passed
@tmusser
tmusser deleted the agent/bind-audit-reports-to-inputs branch August 18, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant