Skip to content

Reject conflicting master data and preserve prior cleaning evidence - #5

Merged
dev-belly merged 1 commit into
mainfrom
codex/audit-master-integrity-20261013
Oct 13, 2026
Merged

dev-belly merged 1 commit into
mainfrom
codex/audit-master-integrity-20261013

Conversation

@dev-belly

Copy link
Copy Markdown
Owner

Problem and behavior

The cleaner previously kept the first supplier/employee record for each ID. Two rows with different bank accounts or roles therefore silently selected different master data when the extract order changed, affecting shared-account features and employee context in audit queries.

Repeated IDs now merge only when every source field agrees after existing text normalisation. Missing cells compare equal to missing cells, while populated versus missing attributes conflict. Validation runs before lossy date parsing and reports the source file, row positions, ID and field names without exposing bank-account values. Cleaned supplier IDs are used for orphan-reference checks.

All three cleaned tables and the quality report are serialised to temporary files before publication. Previously complete outputs are restored after handled publication failures; an initial run removes partially published files. If rollback itself fails, the error identifies recovery paths and retains previous bytes. This remains a single-writer CLI: multi-file reader snapshots and process-crash recovery are outside this change.

Validation

  • 36 new regression cases, including reversed conflicting inputs, missing-value semantics, unknown source fields, date parsing, later serialisation failures and replacement/rollback failures.
  • Local Windows full suite: 501 collected, 500 passed, one existing Makefile check skipped because make is unavailable. All 93 cleaning tests pass.
  • Repository lint passes. The complete pipeline ran in a separate QA copy, preserving checked-in demonstration artifacts. Headline results remain 30,128 vouchers, 248 High/Critical vouchers and a 300-row review plan; all 10 tracked report contents match after OS newline normalisation.
  • Chinese interview case: docs/MASTER_DATA_CASE.zh-CN.md, with business decisions, reproducible tests, follow-up answers and explicit scope boundaries.

@dev-belly
dev-belly merged commit 2e5b3ab into main Oct 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant