fix(anon): salt strength counted characters, and length rescued a repeating pattern (BACKLOG #1166)

docs/BACKLOG.md

@@ -8668,6 +8668,23 @@ private-key floor has the opposite profile: the operator generates that key and
 public half, so raising it can break no handshake with anybody. Folding the two together would have
 smuggled a counterparty-facing refusal into a change whose entire justification is that it has none.
 The JWKS floor is real, is unfiled, and is named here by subject rather than by a number.
+
+**A SECOND #1166 LIMB BUILT 2026-08-22: the anonymizer salt-entropy floor.** This item's prose named it -- "the minimum-salt check counts CHARACTERS rather than measuring entropy". Measured before building: a salt of sixteen letter-a characters was ACCEPTED while a short salt was refused in the same run, so the gate was live and measuring the wrong property. It now estimates Shannon entropy over the observed character distribution and refuses below a floor.
+
+**THREE DEFECTS IN THE FIRST CUT, ALL FOUND BY AN ADVERSARIAL REVIEWER AND ALL CORRECTED BEFORE THIS LANDED.** Recorded because each is a shape worth recognising again.
+
+1. **LENGTH RESCUED A DEGENERATE PATTERN.** The total-bits estimate is length-scaled, so a two-symbol cycle at sixteen characters scored 16.00 bits and was refused while the IDENTICAL pattern at thirty-two characters scored 32.00 and PASSED. That is the length floor defeating the entropy floor, not a blind spot a docstring can disclaim. Fixed by checking the RATE separately: a new `MIN_SALT_ENTROPY_BITS_PER_CHAR` of 2.0, which refuses a two- or three-symbol cycle at any length while leaving a random decimal salt -- about 3.32 bits per character, the narrowest alphabet a real generator produces -- untouched.
+
+2. **THE REFUSAL NEVER REACHED THE OPERATOR.** The salt was validated inside the per-message loop, and the tee CLI wraps each message in a total `except Exception`. That is correct -- it must never let an anonymizer failure surface a body -- but it swallowed a CONFIGURATION error and reported it as N failed messages, steering an operator with a weak salt to extend their rule map, the wrong file entirely. Fixed by validating once BEFORE the loop, where the salt belongs: it is a property of the run, not of any message.
+
+3. **THE FLOOR CONTRADICTED THE ADR AND CITED IT AS AUTHORITY WHILE DOING SO.** The error text invoked ADR 0030 section 4 while accepting a salt at 32 estimated bits, where that ADR requires 128. Both facts are now stated: the ADR governs GENERATION, this estimator grades a SUPPLIED string and is a loose LOWER BOUND -- a genuine 128-bit secret rendered as sixteen decimal digits estimates about 40 bits here, so a floor at 128 would refuse conformant secrets. Clearing this screen therefore does NOT establish ADR compliance, and the comment now says so outright instead of implying the reverse.
+
+**A FOURTH, CAUGHT WHILE VERIFYING THOSE FIXES, AND THE REPOSITORY ALREADY GUARDED IT.** `tee/anon/keying.py` is a verbatim vendored copy and the tee CLI imports the VENDORED one, so fixing only the engine module left the CLI accepting a salt the engine refused. I nearly added a drift guard for it; `tests/test_anon_parity.py::test_shared_logic_files_are_byte_identical` already exists and fires on exactly that drift, verified by desyncing the file and watching it fail. The real lesson is narrower and about method: I ran one test FILE rather than the suite, and a file-scoped run cannot see a guard that lives in another file.
+
+Proof: four mutations, each red, zero vacuous -- per-character floor disabled, vendored copy drifted, total-bits floor disabled, and the two floors shown to fire independently. 66 passed across the anonymizer core and the tee CLI.
+
+**Residual:** the estimator stays ORDER-blind and DICTIONARY-blind, so a string of sixteen distinct sequential letters measures the arithmetic maximum and passes. Nothing short of generating the salt ourselves fixes that, which is what ADR 0030 already requires.
+
 ## 1167. research an honest pass for ASVS 11.2.4 -- constant-time recovery-code verification without turning ten argon2id slots into an amplification target
 
 > 🔢 **Re-scored 2026-08-20 -> P3.** Value **4/10** · Difficulty **7/10** · _money pit_. The data-dependent early return survives on the shipped MFA path: _verify_second_factor walks the argon2id recovery hashes and returns on the first match, so the number of ~64 MiB verifications is a function of which code was presented. Value 4 because the leak is a wall-clock signal on an already-authenticated second factor rather than a bypass; difficulty 7 because the obvious constant-time loop multiplies a 64 MiB argon2id verification by the slot count on every attempt, converting a timing leak into a memory and CPU amplification target, and the evidentiary half has no precedent in this tree. _(was 4/10 · 7/10.)_

messagefoundry/anon/keying.py

@@ -24,7 +24,9 @@
 from __future__ import annotations
 
 import hashlib
+import math
 import random
+from collections import Counter
 
 #: 128-bit seed — ample entropy to index any surrogate pool without collisions you'd notice.
 _DIGEST_SIZE = 16
@@ -34,6 +36,85 @@
 #: obviously-weak salt rather than emit guessable surrogates (fail closed, ADR 0030 §4).
 MIN_SALT_LEN = 16
 
+#: Floor on the salt's ESTIMATED entropy, in bits. Length alone is not strength: sixteen copies of
+#: the letter "a" clears ``MIN_SALT_LEN`` and is guessable on the first try, so the length floor
+#: could be passed by a salt with no secrecy at all.
+MIN_SALT_ENTROPY_BITS = 32.0
+
+#: Floor on the salt's estimated entropy PER CHARACTER, in bits. The total-bits floor above is
+#: length-scaled, so **length alone would rescue a degenerate pattern**: ``"ab"`` repeated eight
+#: times scores 16.00 bits and is refused, while the SAME two-symbol pattern repeated sixteen times
+#: scores 32.00 and would pass. That is not a blind spot the estimator's docstring can wave at -- it
+#: is the length floor defeating the entropy floor -- so the rate is checked separately from the
+#: total and a string has to clear BOTH.
+#:
+#: 2.0 bits per character means an effective alphabet of four symbols. Measured against the shapes
+#: that matter: ``"ab"`` repeated scores 1.00 and is refused at any length; ``"abc"`` repeated scores
+#: about 1.58 and is refused; a random DECIMAL salt -- the narrowest alphabet a real generator would
+#: produce -- scores about 3.32, and random hex about 3.9, so neither is touched. A single repeated
+#: character scores 0.00 and was already refused by the total.
+MIN_SALT_ENTROPY_BITS_PER_CHAR = 2.0
+
+#: **WHY 32 BITS AND NOT THE 128 THE ADR NAMES.** ADR 0030 section 4 requires ``dataset_key`` to be
+#: DRAWN from ``secrets.token_bytes``/``os.urandom`` with at least 128 bits of entropy. That is a
+#: requirement on GENERATION and it is not the same measurement as this one. These floors grade a
+#: string the engine did not generate, using a distribution estimate that is a LOWER BOUND and a
+#: loose one: a genuine 128-bit secret rendered as sixteen decimal digits estimates about 40 bits
+#: here, and rendered as base62 about 95 -- neither reaches 128, because the estimator cannot see
+#: the entropy of the generator, only of the characters in front of it.
+#:
+#: So a floor set at the ADR's 128 would refuse real, conformant secrets, which is why it is not set
+#: there. **The consequence must be read honestly: clearing these floors does NOT establish that a
+#: salt meets ADR 0030, and no check on a supplied string could.** This is a screen against a
+#: visibly degenerate salt, sitting underneath the ADR requirement rather than implementing it. The
+#: control that actually delivers 128 bits is generating the salt with the tool the error message
+#: names, and that stays where the ADR puts it.
+
+#: Ceiling on the salt's UTF-8 length in BYTES, set by BLAKE2b's keyed mode
+#: (``hashlib.blake2b.MAX_KEY_SIZE``, 64). Checked here at construction rather than left to
+#: :meth:`Keyer.seed`, where it fired as a bare ``ValueError: maximum key length is 64 bytes`` on
+#: the FIRST message instead of on the bad configuration -- an 86-byte ``secrets.token_urlsafe(64)``
+#: built a Keyer without complaint and then failed mid-dataset. Bytes, not characters: a non-ASCII
+#: salt can pass 64 bytes well under 64 characters.
+MAX_SALT_BYTES = hashlib.blake2b.MAX_KEY_SIZE
+
+
+def _estimated_entropy_bits(salt: str) -> float:
+    """Shannon entropy of the salt's OBSERVED character distribution, scaled by its length.
+
+    Why this estimator. It is the weakest assumption we can make about a string we did not
+    generate: count the symbols, take ``-sum(p * log2(p))`` for the per-character entropy, and
+    multiply by the length for a total in bits. It needs no dictionary, no alphabet table and no
+    network, it grades continuously rather than by category, and -- unlike a bare count of distinct
+    characters -- it sees SKEW. A salt of sixteen "a" characters plus seven distinct others has
+    eight distinct characters and would clear a distinct-character floor of eight, while this
+    estimator scores it at about 40 bits for twenty-three characters and grades it near the floor,
+    which is the honest reading of a string that is five-sixths one symbol.
+
+    WHAT IT CANNOT SEE, stated plainly because every distribution-based estimator shares the blind
+    spot: it is ORDER-BLIND and DICTIONARY-BLIND. It scores every permutation of a string
+    identically, so "abcdefghijklmnop" measures a full 64.00 bits at the length floor -- the
+    arithmetic maximum for sixteen characters -- and passes, despite being one of the first strings
+    any attacker would try. A repeated cycle such as "abcdabcdabcdabcd" measures exactly 32.00 bits
+    and also passes. This check therefore catches a salt that is visibly DEGENERATE; it cannot
+    certify that a salt is unpredictable, and nothing short of generating the salt ourselves could.
+    The real defence stays where ADR 0030 section 4 puts it: an env-supplied secret from a random
+    generator. This is the floor under that, not a substitute for it.
+
+    Where the floor came from. Measured 2026-08-22 by sampling 200,000 random 16-character salts
+    per alphabet and counting how many this estimator scores below 32 bits: base62 0, lowercase 0,
+    lowercase hex 1, and decimal digits 57 -- the tightest realistic case, and a 16-digit salt
+    carries only about 53 bits of real entropy to begin with. The zeros are trustworthy because the
+    same run returned NON-ZERO at higher floors (at 40 bits the same four alphabets rejected 0, 2,
+    306 and 11,066 of 200,000), so the instrument can fire. A 40-bit floor would refuse better than
+    5 percent of genuinely random 16-digit salts, which is refusing real secrets; 32 bits does not.
+    """
+    if not salt:
+        return 0.0
+    n = len(salt)
+    per_char = sum(-(c / n) * math.log2(c / n) for c in Counter(salt).values())
+    return n * per_char
+
 
 class Keyer:
     """Maps a ``(field-kind, real value)`` pair to a stable, salt-keyed PRNG — one per dataset.
@@ -51,9 +132,43 @@ def __init__(self, salt: str) -> None:
                 f"anonymizer salt must be a secret of at least {MIN_SALT_LEN} characters "
                 "(ADR 0030 §4: pinned-per-dataset secret, env-supplied, never committed)"
             )
+        encoded = salt.encode("utf-8")
+        if len(encoded) > MAX_SALT_BYTES:
+            raise ValueError(
+                f"anonymizer salt must be at most {MAX_SALT_BYTES} bytes when UTF-8 encoded "
+                f"(this one is {len(encoded)}); that is BLAKE2b's keyed-mode limit, and a salt at "
+                "the ceiling already carries far more entropy than the surrogate pools can use. "
+                "secrets.token_urlsafe(48) is the longest that fits."
+            )
+        # Length is a necessary floor, not a sufficient one -- see MIN_SALT_ENTROPY_BITS and
+        # _estimated_entropy_bits for the measure and its limits. Neither the salt nor any part of
+        # it appears in the message: it is a re-identification key, so an exception that quotes it
+        # would leak it into a traceback, a log or a CI transcript.
+        entropy = _estimated_entropy_bits(salt)
+        if entropy < MIN_SALT_ENTROPY_BITS:
+            raise ValueError(
+                f"anonymizer salt is too predictable: about {entropy:.1f} bits of estimated "
+                f"entropy against a floor of {MIN_SALT_ENTROPY_BITS:.0f}. Generate one with "
+                "secrets.token_urlsafe(24) and supply it from the environment. The salt itself is "
+                "withheld from this message on purpose. Note this floor is a SCREEN, not the ADR "
+                "0030 section 4 requirement: that one is 128 bits AT GENERATION, which no check on "
+                "a supplied string can confirm."
+            )
+        # Checked SEPARATELY from the total, because the total is length-scaled and length would
+        # otherwise rescue a degenerate pattern -- "ab" repeated sixteen times reaches the 32-bit
+        # total on 1.00 bits per character. Both floors bind; neither substitutes for the other.
+        per_char = entropy / len(salt)
+        if per_char < MIN_SALT_ENTROPY_BITS_PER_CHAR:
+            raise ValueError(
+                f"anonymizer salt repeats too few distinct characters: about {per_char:.2f} bits "
+                f"per character against a floor of {MIN_SALT_ENTROPY_BITS_PER_CHAR:.1f}. Making it "
+                "longer will NOT help -- a repeating pattern carries the same rate at any length. "
+                "Generate one with secrets.token_urlsafe(24) and supply it from the environment. "
+                "The salt itself is withheld from this message on purpose."
+            )
         # Hold the salt as bytes only; it is PHI-equivalent (a re-identification key) and must
         # never be logged, persisted, or surfaced — so we keep no other reference to it.
-        self._salt = salt.encode("utf-8")
+        self._salt = encoded
 
     def seed(self, kind: str, value: str) -> int:
         """A 128-bit seed for ``(kind, value)``, keyed by the dataset salt (one-way)."""

tee/__main__.py

@@ -45,6 +45,7 @@
 
 from tee import __version__, mefor_api
 from tee.anon import anonymize_checked
+from tee.anon.keying import Keyer
 from tee.correlate import CorepointOutput, CorrelateConfig
 from tee.relay import Endpoint, RelayConfig, TeeRelay
 from tee.report import build_report
@@ -511,6 +512,20 @@ async def _anonymize_captures(args: argparse.Namespace) -> int:
         return 1
 
     overlay = Path(args.overlay) if args.overlay else None
+
+    # VALIDATE THE SALT ONCE, HERE, BEFORE THE LOOP. The per-message `except Exception` below is
+    # deliberately total -- it must never let an anonymizer failure surface a body -- but that also
+    # swallows a CONFIGURATION error and reports it as N failed messages. A weak salt would then
+    # steer the operator to extend their rule map, which is the wrong file, while the message
+    # naming the actual problem was discarded. The salt is a property of the RUN, not of any
+    # message, so it belongs outside the loop; checking it here also means a bad salt costs one
+    # check instead of one per message.
+    try:
+        Keyer(salt)
+    except ValueError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
+
     lines: list[str] = []
     failed = 0
     for row in rows: