fix(anon): salt strength counted characters, and length rescued a repeating pattern (BACKLOG #1166) - #522
Open
wshallwshall wants to merge 1 commit into
Open
fix(anon): salt strength counted characters, and length rescued a repeating pattern (BACKLOG #1166)#522wshallwshall wants to merge 1 commit into
wshallwshall wants to merge 1 commit into
Conversation
…eating pattern (BACKLOG #1166) Keyer refused a salt shorter than MIN_SALT_LEN CHARACTERS, which is not the same as entropy. Measured before building: sixteen letter-a characters were ACCEPTED while a short salt was refused in the same run, so the gate was live and measuring the wrong property. A guessable salt makes de-identification surrogates guessable, which is a re-identification risk. THE FIRST CUT HAD THREE DEFECTS. An adversarial reviewer found all three and each is fixed here; they are worth naming because each is a recognisable shape. 1. LENGTH RESCUED A DEGENERATE PATTERN. The total-bits estimate is length-scaled, so a two-symbol cycle at 16 characters scored 16.00 bits and was refused while the same pattern at 32 characters scored 32.00 and passed. That is the length floor defeating the entropy floor. Fixed by checking the RATE separately -- MIN_SALT_ENTROPY_BITS_PER_CHAR = 2.0 -- which refuses a two- or three-symbol cycle at ANY length while leaving a random decimal salt (about 3.32 bits per character) untouched. 2. THE REFUSAL NEVER REACHED THE OPERATOR. The salt was validated inside the per-message loop and the tee CLI wraps each message in a total except Exception. That is correct -- it must never let an anonymizer failure surface a body -- but it swallowed a CONFIGURATION error and reported it as N failed messages, steering the operator to their rule map. Fixed by validating once BEFORE the loop: the salt is a property of the run. 3. THE FLOOR CONTRADICTED THE ADR WHILE CITING IT. The message invoked ADR 0030 section 4 while accepting 32 estimated bits, where the ADR requires 128. Both are now stated: the ADR governs GENERATION; this estimator grades a supplied string and is a loose lower bound, so a floor at 128 would refuse conformant secrets. Clearing the screen does NOT establish ADR compliance. AND A FOURTH I CAUGHT, WHICH THE REPO ALREADY GUARDED. tee/anon/keying.py is a verbatim vendored copy and the CLI imports the VENDORED one, so fixing only the engine module left the CLI accepting what the engine refused. I nearly added a drift guard; test_anon_parity.py already has one and fires on exactly that, verified by desyncing and watching it fail. The lesson is about method: I ran one test FILE, and a file-scoped run cannot see a guard in another file. Four mutations, each red, zero vacuous. 76 passed across the anonymizer core, the vendor parity check and the tee CLI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the second defect #1166's own prose named: "the minimum-salt check counts CHARACTERS rather than measuring entropy".
Measured before building: sixteen letter-
acharacters were ACCEPTED while a short salt was refused in the same run -- the gate was live and measuring the wrong property. A guessable salt makes de-identification surrogates guessable, which is a re-identification risk.The first cut had three defects. An adversary found all three.
1. Length rescued a degenerate pattern. The total-bits estimate is length-scaled, so a two-symbol cycle at 16 chars scored 16.00 and was refused, while the same pattern at 32 chars scored 32.00 and passed. That is the length floor defeating the entropy floor -- not a blind spot a docstring can disclaim.
Fixed by checking the rate separately (
MIN_SALT_ENTROPY_BITS_PER_CHAR = 2.0):"ab"x16 (32 chars)"ab"x32 (64 chars)"abc"x21 + a (64 chars)token_urlsafe(24), hex, 16 random digits2. The refusal never reached the operator. The salt was validated inside the per-message loop, and the tee CLI wraps each message in a total
except Exception. That is correct -- it must never let an anonymizer failure surface a body -- but it swallowed a configuration error and reported it as N failed messages, steering an operator with a weak salt to extend their rule map. Wrong file entirely.Fixed by validating once before the loop. The salt is a property of the run, not of any message.
3. The floor contradicted the ADR while citing it as authority. The error text invoked ADR 0030 §4 while accepting 32 estimated bits, where that ADR requires 128. Both facts are now stated: the ADR governs generation; this estimator grades a supplied string and is a loose lower bound -- a genuine 128-bit secret rendered as 16 decimal digits estimates ~40 bits here, so a floor at 128 would refuse conformant secrets. Clearing this screen does not establish ADR compliance, and the comment now says so outright.
A fourth I caught -- which the repository already guarded
tee/anon/keying.pyis a verbatim vendored copy and the CLI imports the vendored one. Fixing only the engine module left the CLI accepting a salt the engine refused.I nearly added a drift guard.
tests/test_anon_parity.py::test_shared_logic_files_are_byte_identicalalready exists and fires on exactly that -- verified by desyncing the file and watching it fail, then restoring. My test was redundant and has been removed.The real lesson is about method: I ran one test file, and a file-scoped run cannot see a guard living in another file.
Proof
Four mutations, each RED, zero vacuous: per-character floor disabled, vendored copy drifted, total-bits floor disabled, and the two floors shown to fire independently (neither substitutes for the other).
76 passed across the anonymizer core, the vendor parity check and the tee CLI.
Residual
The estimator stays order-blind and dictionary-blind: sixteen distinct sequential letters measure the arithmetic maximum and pass. Nothing short of generating the salt ourselves fixes that -- which is what ADR 0030 already requires.
Generated with Claude Code