ci: reject control bytes and bidi codepoints in tracked source (authoring toolchain injects live ESC)

[dean0x]

Background

The authoring toolchain used to write this repo decodes \u+4-hex escape sequences
in tool arguments into real bytes before they reach disk. A doc comment written
as the six characters � lands on disk as a single literal 0x1B ESC byte.

This has now happened three times during the #176 / PR #253 work. The most
recent occurrence required commit b94ef80 to strip a literal 0x1B from a doc
comment — on the branch whose entire subject was hardening terminal-escape
rendering (CWE-150). The failure mode is silent: the byte renders as nothing in
most diff viewers, git diff shows a line that looks identical to its neighbours,
and review passes over it.

Problem

There is no gate — local or CI — that rejects control bytes or bidirectional
control codepoints in tracked source. A stray ESC in a shipped .rs, .md, or
.json file would:

• be published to crates.io / npm / PyPI inside the artifact,
• defeat the CWE-150 sanitization boundary the project just built (the sanitizer
  guards runtime rendering of untrusted input, not the compiler's own source),
• and, in the bidi case (U+202A–U+202E, U+2066–U+2069), enable
  Trojan-Source-style visual reordering that makes reviewed code differ from
  compiled code.

Detection trap — read before writing the check

BSD grep on macOS has no -P. grep -P $'\x1b' -r . exits 2 and prints
nothing. A scan written that way reports a clean tree whether or not the tree
is clean, and it fails open on exactly the developer machines where the bytes are
being introduced.

Any scan added here MUST be positive-controlled: plant a known ESC byte in a
scratch file, assert the scan finds it, and only then trust a clean result on the
real tree. Prefer a portable implementation (rg, python3, or git grep -P
with an explicit availability check that hard-fails when unavailable) over grep -P.

Proposed change

1. A scanner over tracked files that rejects:
   • C0 control bytes other than \t (0x09), \n (0x0A), and \r (0x0D) — in
     particular ESC (0x1B);
   • the Unicode bidi overrides/isolates U+202A–U+202E and U+2066–U+2069.
2. Wire it into CI as a hard gate, and into a pre-commit hook for local fast-fail.
3. Allow-list any file that legitimately needs these bytes (test fixtures that
   assert sanitizer behaviour — e.g. the CWE-150 fixtures from Harden terminal-escape rendering across all MdsError variants (CWE-150) #176) via an
   explicit, enumerated exception list, not a glob.
4. Self-test the scanner: a CI step that plants a control byte in a temp file and
   asserts the scanner exits non-zero, so the gate can never silently fail open
   the way grep -P does on BSD.

Acceptance Criteria

• Scanner rejects a planted 0x1B in a tracked .rs / .md / .json file
• Scanner rejects a planted U+202E in a tracked source file
• Scanner passes on the current tree at main
• Scanner's own detection is positive-controlled in CI (plant → expect failure)
• Implementation does not depend on grep -P without an availability hard-fail
• Exception list for sanitizer test fixtures is explicit and enumerated
• Documented in CONTRIBUTING.md with the BSD-grep trap called out

Reference

• Harden terminal-escape rendering across all MdsError variants (CWE-150) #176 / PR fix: harden terminal-escape rendering across all MdsError variants (CWE-150) #253 (b681129) — terminal-escape rendering hardening (CWE-150)
• Commit b94ef80 — the third strip of a toolchain-injected ESC byte
• Related but distinct: Reject control characters in filenames at the input boundary #265 rejects control characters in filenames at the
  product's input boundary; this issue is about control bytes in the repo's
  own tracked source, which is a build/CI concern, not a compiler concern.