Wrong assumption
Every bundled secret token pattern anchors its start on a leading \b (internal/adapter/scanner/patterns.go): \bghp_…, \bghs_/gho_/ghu_/ghr_…, \bgithub_pat_…, \bAKIA…, \bxox[baprs]-…, \bAIza…, \bsk_live_/sk_test_…, \bsk-ant-/sk-proj-/sk-svcacct-…, \beyJ….….…. RE2 \b is an ASCII word boundary. The implicit assumption is that a real leaked token is always immediately preceded by a non-word byte (=, :, ", /, space, newline), so the leading \b always holds.
That assumption breaks under percent-encoding. When a URL/JSON body is URL-encoded, every non-word delimiter becomes %XX whose final byte is a hex digit (a word char): =→%3D, "→%22, :→%3A, /→%2F, ?→%3F. The token itself is made of unreserved URL characters and stays literal and in the clear. So the byte immediately before the literal token prefix is always [0-9A-Fa-f], \b fails, and the scanner's own regex — which would otherwise match the token verbatim — never fires. This is a validate-before-canonicalize gap: the redactor scans raw bytes and never percent-decodes.
Exact trigger (input → observed vs correct)
A session transcript line containing a percent-encoded URL that carries a real credential, e.g. an OAuth/redirect/magic-link:
...&redirect=%2Fauth%3Ftoken%3Dghp_<36+ alnum PAT>...
- Observed:
\bghp_[A-Za-z0-9]{36,} does not match (%3D leaves D, a word char, right before ghp_), so scanner.ScanText returns no finding, Redact changes nothing, and the raw live PAT is written into the committed (and typically pushed) transcript record.
- Correct: the live token is detected and redacted; the store's documented guarantee — "a stored record can never contain a live secret" — holds.
Empirically verified against the actual compiled regexes (all return no match, while the plaintext token=ghp_… matches fine):
\bghp_[A-Za-z0-9]{36,} vs %3Dghp_<36 alnum> → no match
%22ghp_… / %2Fghp_… → no match
redirect%3Dghp_… (nested) → no match
\bAKIA[0-9A-Z]{16} vs %3DAKIA<16> → no match
\beyJ…\.…\.… vs link%3Ftoken%3DeyJ….….… → no match (URL-encoded magic-link JWT)
Reachability (the two-stage fail-closed guard does NOT catch it)
internal/core/history/history.go:139-162 — Capture redacts raw session transcripts (attacker-influenceable content) at the write boundary:
text := string(raw) (line 139) — scanned raw; there is no percent-decode pre-pass anywhere in the scan pipeline (grep of scanner/history/lifeboat/launch is clean).
- Stage one (140-141):
ScanText + Redact — misses the encoded-delimiter token.
- Stage one-and-a-half (150-155): a literal backstop, but it collapses only the caller's
$HOME; secrets have no literal backstop.
- Stage two / verify (159): re-scans the redacted text with the same detector, so the same leading-
\b blind spot misses the token identically; blockingResidual finds nothing and the write proceeds.
The code comment at history.go:143-148 already acknowledges this exact failure mode — "a span that detector's trailing-boundary heuristic dropped would slip through both stages" — and only guards home paths against it. The same reachability applies to lifeboat pack and launch dry-run/ship, which redact planned bytes through the same scanner.
Why this is distinct from what's already fixed
internal/adapter/scanner/scanner.go:400-405 explicitly disclaims "a real secret preceded by content that is not itself a match found here" as an accepted \b-boundary limitation. The adjacency machinery (ledger iss-185 adjacencyProbe, iss-188 stolenJunctions) only recovers a token abutting a previously matched token. This report shows the accepted limitation is not merely inert "filler-text noise": percent-encoding is a systematic, deterministic, realistic instance of it that places a %XX hex byte before the literal token in any URL-encoded URL/JSON, turning an "accepted" gap into a reachable live-secret leak into a committed record. Not present in the issue tracker or the abcd ledger (the scanner ledger entries cover serialized cross-leak, adjacency, config-read, and degraded-scanner — none is about encoded delimiters).
Sibling sweep
Affected = every bundled token pattern whose prefix begins with a word char and carries a leading \b (patterns.go): github_pat (ghp_, line 64), github_server_token (ghs_, 69), github_oauth (gho_, 74), github_user_token (ghu_, 79), github_refresh (ghr_, 84), github_pat_finegrained (93), anthropic_key (sk-ant-, 109), openai_project_key (114), openai_service_account (119), aws_access_key (AKIA, 125), slack_token (xox, 137), google_api_key (AIza, 147), stripe_live_key (152), stripe_test_key (157), jwt_shaped (eyJ, 162). All 15 miss when a %XX delimiter precedes the token.
Not affected: pem_private_key (103) and rp_session_key (48) have no leading \b. The network patterns (net_ipv4/net_ipv6/net_mac) also carry a leading \b, but their separators (./:) are themselves encoded, changing the token shape — a separate, lesser concern worth a follow-up check, not part of this confirmed finding. The identity matchers (matchers.findings) are also cited in the scanner.go:400-405 disclaimer and should be checked for the same encoded-boundary interaction.
CWE anchor
CWE-180 (Incorrect Behavior Order: Validate Before Canonicalize) — the redactor scans before percent-decoding; CWE-176/CWE-20 (improper handling of an encoding / ASCII-only boundary); leaf impact CWE-312 / CWE-532 (sensitive information written into a stored file).
Adversarial validation (both lenses independently confirmed)
- Reachability lens: "Reachability is real —
history.go scans raw transcripts and stage-two re-scans with the same detector, so the missed token slips both stages; percent-encoding rewrites every delimiter into a %XX hex word-char while the token stays literal, so the leading \b can never hold — a systematic class (encoded OAuth/redirect/magic-link URLs), not a contrived string."
- Correctness lens: "
%3Dghp_REALTOKEN is a complete, valid, percent-decodable live credential the scanner's own regex would match but for the leading \b landing on the hex byte D; the false-positive objection only rules out the naive fix of dropping \b, not the defect's existence; against history's absolute guarantee that a stored record can never contain a live secret, writing a raw live JWT/PAT into the committed record is a genuine correctness defect."
Fix direction (one sentence)
Scan a percent-decoded copy of the text (or make the token start-boundary percent-encoding-aware) so an encoded delimiter cannot mask a literal token — without dropping the leading \b, which legitimately suppresses mid-blob false positives.
Wrong assumption
Every bundled secret token pattern anchors its start on a leading
\b(internal/adapter/scanner/patterns.go):\bghp_…,\bghs_/gho_/ghu_/ghr_…,\bgithub_pat_…,\bAKIA…,\bxox[baprs]-…,\bAIza…,\bsk_live_/sk_test_…,\bsk-ant-/sk-proj-/sk-svcacct-…,\beyJ….….…. RE2\bis an ASCII word boundary. The implicit assumption is that a real leaked token is always immediately preceded by a non-word byte (=,:,",/, space, newline), so the leading\balways holds.That assumption breaks under percent-encoding. When a URL/JSON body is URL-encoded, every non-word delimiter becomes
%XXwhose final byte is a hex digit (a word char):=→%3D,"→%22,:→%3A,/→%2F,?→%3F. The token itself is made of unreserved URL characters and stays literal and in the clear. So the byte immediately before the literal token prefix is always[0-9A-Fa-f],\bfails, and the scanner's own regex — which would otherwise match the token verbatim — never fires. This is a validate-before-canonicalize gap: the redactor scans raw bytes and never percent-decodes.Exact trigger (input → observed vs correct)
A session transcript line containing a percent-encoded URL that carries a real credential, e.g. an OAuth/redirect/magic-link:
\bghp_[A-Za-z0-9]{36,}does not match (%3DleavesD, a word char, right beforeghp_), soscanner.ScanTextreturns no finding,Redactchanges nothing, and the raw live PAT is written into the committed (and typically pushed) transcript record.Empirically verified against the actual compiled regexes (all return no match, while the plaintext
token=ghp_…matches fine):Reachability (the two-stage fail-closed guard does NOT catch it)
internal/core/history/history.go:139-162—Captureredacts raw session transcripts (attacker-influenceable content) at the write boundary:text := string(raw)(line 139) — scanned raw; there is no percent-decode pre-pass anywhere in the scan pipeline (grep of scanner/history/lifeboat/launch is clean).ScanText+Redact— misses the encoded-delimiter token.$HOME; secrets have no literal backstop.\bblind spot misses the token identically;blockingResidualfinds nothing and the write proceeds.The code comment at
history.go:143-148already acknowledges this exact failure mode — "a span that detector's trailing-boundary heuristic dropped would slip through both stages" — and only guards home paths against it. The same reachability applies tolifeboatpack andlaunchdry-run/ship, which redact planned bytes through the same scanner.Why this is distinct from what's already fixed
internal/adapter/scanner/scanner.go:400-405explicitly disclaims "a real secret preceded by content that is not itself a match found here" as an accepted\b-boundary limitation. The adjacency machinery (ledger iss-185adjacencyProbe, iss-188stolenJunctions) only recovers a token abutting a previously matched token. This report shows the accepted limitation is not merely inert "filler-text noise": percent-encoding is a systematic, deterministic, realistic instance of it that places a%XXhex byte before the literal token in any URL-encoded URL/JSON, turning an "accepted" gap into a reachable live-secret leak into a committed record. Not present in the issue tracker or the abcd ledger (the scanner ledger entries cover serialized cross-leak, adjacency, config-read, and degraded-scanner — none is about encoded delimiters).Sibling sweep
Affected = every bundled token pattern whose prefix begins with a word char and carries a leading
\b(patterns.go):github_pat(ghp_, line 64),github_server_token(ghs_, 69),github_oauth(gho_, 74),github_user_token(ghu_, 79),github_refresh(ghr_, 84),github_pat_finegrained(93),anthropic_key(sk-ant-, 109),openai_project_key(114),openai_service_account(119),aws_access_key(AKIA, 125),slack_token(xox, 137),google_api_key(AIza, 147),stripe_live_key(152),stripe_test_key(157),jwt_shaped(eyJ, 162). All 15 miss when a%XXdelimiter precedes the token.Not affected:
pem_private_key(103) andrp_session_key(48) have no leading\b. The network patterns (net_ipv4/net_ipv6/net_mac) also carry a leading\b, but their separators (./:) are themselves encoded, changing the token shape — a separate, lesser concern worth a follow-up check, not part of this confirmed finding. The identity matchers (matchers.findings) are also cited in thescanner.go:400-405disclaimer and should be checked for the same encoded-boundary interaction.CWE anchor
CWE-180 (Incorrect Behavior Order: Validate Before Canonicalize) — the redactor scans before percent-decoding; CWE-176/CWE-20 (improper handling of an encoding / ASCII-only boundary); leaf impact CWE-312 / CWE-532 (sensitive information written into a stored file).
Adversarial validation (both lenses independently confirmed)
history.goscans raw transcripts and stage-two re-scans with the same detector, so the missed token slips both stages; percent-encoding rewrites every delimiter into a%XXhex word-char while the token stays literal, so the leading\bcan never hold — a systematic class (encoded OAuth/redirect/magic-link URLs), not a contrived string."%3Dghp_REALTOKENis a complete, valid, percent-decodable live credential the scanner's own regex would match but for the leading\blanding on the hex byteD; the false-positive objection only rules out the naive fix of dropping\b, not the defect's existence; against history's absolute guarantee that a stored record can never contain a live secret, writing a raw live JWT/PAT into the committed record is a genuine correctness defect."Fix direction (one sentence)
Scan a percent-decoded copy of the text (or make the token start-boundary percent-encoding-aware) so an encoded delimiter cannot mask a literal token — without dropping the leading
\b, which legitimately suppresses mid-blob false positives.