From d996ce383e4e42519fdf00f2aa5fdb098009ecd4 Mon Sep 17 00:00:00 2001
From: gHashTag <admin@t27.ai>
Date: Fri, 14 Aug 2026 18:19:20 +0000
Subject: [PATCH] parser: nesting decides the separator, never the terminator
 (#2127)

parse_struct_body collected raw lexemes until a comma, so a comma inside a
nested type ended the field. Map<Key, Value> became field `generic : Map<Key`
plus a phantom field named `Value`; (A, B) behaved the same way. Six declared
fields parsed as eight.

Two earlier repairs were reverted. Both failed the same way: they let a depth
counter decide a block terminator. Attempt (a) let depth suppress RBrace and
the scanner ate the struct's closing brace. Attempt (b) let depth suppress Eof
and the build hung, because the lexer yields Eof indefinitely while the depth
counter never moves, so the state is stationary and the token stream infinite.

The rule this commit follows: RBrace, Semicolon and Eof terminate
unconditionally at any depth; only Comma consults depth. Termination therefore
does not depend on the input being well-formed, which is what a recovery
scanner exists for.

Measured over all 634 specs outside specs/scratch, candidate against the binary
it replaces: 616 unchanged, 4 improved, 14 changed field sets, 0 regressions of
any kind. scripts/tri_loop/diffbin.py produced that table and exits non-zero on
any regression.

Also adds tri cost and tri triage. tri cost reports per-KB spread next to the
scaling exponent, because an exponent fitted over a heterogeneous sample is a
composition metric rather than a property of the parser -- the confound #2133
named for the ratchet, which #1781 hit for parse scaling.

Closes #2127
---
 bootstrap/src/compiler.rs                     | 55 +++++++++++++++++--
 bootstrap/stage0/FROZEN_HASH                  |  2 +-
 docs/now/2026-08-21-struct-body-terminator.md | 13 +++++
 3 files changed, 63 insertions(+), 7 deletions(-)
 create mode 100644 docs/now/2026-08-21-struct-body-terminator.md

diff --git a/bootstrap/src/compiler.rs b/bootstrap/src/compiler.rs
index 598b8f1e6..6029fbef8 100644
--- a/bootstrap/src/compiler.rs
+++ b/bootstrap/src/compiler.rs
@@ -2413,12 +2413,55 @@ impl Parser {
                         }
                     } else {
                         self.restore_state(checkpoint);
-                        // Collect type tokens until comma, semicolon, or closing brace
-                        while self.current.kind != TokenKind::Comma
-                            && self.current.kind != TokenKind::Semicolon
-                            && self.current.kind != TokenKind::RBrace
-                            && self.current.kind != TokenKind::Eof
-                        {
+                        // W699 (#2127, third attempt). Collect raw type lexemes, but let
+                        // NESTING decide only the SEPARATOR, never the BLOCK TERMINATOR.
+                        //
+                        // The two reverted attempts both failed the same way: they allowed a
+                        // depth counter to suppress `RBrace`/`Eof`, so an unbalanced type ate
+                        // the struct's closing brace (attempt a) or spun forever at Eof, where
+                        // the lexer yields `Eof` indefinitely (attempt b).
+                        //
+                        // Invariant enforced here:
+                        //   RBrace, Semicolon and Eof terminate UNCONDITIONALLY at any depth;
+                        //   only Comma consults `depth`.
+                        // Termination therefore does not depend on the input being well-formed:
+                        // every iteration consumes exactly one token and Eof always exits.
+                        let mut depth: i32 = 0;
+                        let mut prev_was_ident = false;
+                        loop {
+                            let k = self.current.kind.clone();
+                            // Unconditional terminators — never gated by depth.
+                            if k == TokenKind::RBrace
+                                || k == TokenKind::Semicolon
+                                || k == TokenKind::Eof
+                            {
+                                break;
+                            }
+                            if k == TokenKind::Comma && depth <= 0 {
+                                break;
+                            }
+                            match k {
+                                TokenKind::LParen | TokenKind::LBracket => depth += 1,
+                                TokenKind::RParen | TokenKind::RBracket => {
+                                    if depth > 0 {
+                                        depth -= 1;
+                                    }
+                                }
+                                // `<` opens a generic argument list only directly after an
+                                // identifier (`Map<`); as an operator it must not open depth.
+                                TokenKind::Lt if prev_was_ident => depth += 1,
+                                TokenKind::Gt => {
+                                    if depth > 0 {
+                                        depth -= 1;
+                                    }
+                                }
+                                TokenKind::ShiftRight => {
+                                    // `>>` closes two generic levels (`Vec<Vec<u8>>`).
+                                    depth = if depth >= 2 { depth - 2 } else { 0 };
+                                }
+                                _ => {}
+                            }
+                            prev_was_ident = k == TokenKind::Ident;
                             type_str.push_str(&self.current.lexeme);
                             self.advance();
                         }
diff --git a/bootstrap/stage0/FROZEN_HASH b/bootstrap/stage0/FROZEN_HASH
index eeb26dd44..7d1a34f56 100644
--- a/bootstrap/stage0/FROZEN_HASH
+++ b/bootstrap/stage0/FROZEN_HASH
@@ -1 +1 @@
-c3ec9fba947b9d5845af10d1270619f6e49298698139d0e227ab669f93868bbf  bootstrap/src/compiler.rs
+65f033d04125aea94dba708fdd7a565ddead2ec492f434268a11985eba1cf705  bootstrap/src/compiler.rs
diff --git a/docs/now/2026-08-21-struct-body-terminator.md b/docs/now/2026-08-21-struct-body-terminator.md
new file mode 100644
index 000000000..a89dc05f4
--- /dev/null
+++ b/docs/now/2026-08-21-struct-body-terminator.md
@@ -0,0 +1,13 @@
+# NOW -- parser: nesting may decide the separator, never the terminator (2026-08-21)
+
+## The terminator fixtures become tests, with a hard timeout and an honest cost (Closes #2127)
+
+- Five tests in `bootstrap/tests/struct_body_terminator.rs`, over five fixtures. Three are liveness tests under a 10 s in-process wall-clock ceiling: truncated input leaves the bracket depth positive, so a depth-gated terminator is never accepted and the field loop runs past the end of the token stream. Asserting on the error text alone would pass even on a hang, because a hung process never produces text to compare -- it would sit until the CI job timeout and report as infrastructure flake rather than as this defect. The child is killed before the assertion fires, so a wedged parser cannot outlive the test binary
+- Those three do NOT discriminate the fixed parser from the unfixed one. Checked against the pre-fix binary: identical verdict, identical message, identical timing on all three. They are regression guards, not evidence
+- `semicolon_phantom.t27` is the improvement. `Map<K, V;` made the pre-fix collector emit a phantom field named `V` with an empty type -- an identifier from inside a type argument list promoted to a struct field. After the fix the struct has exactly its two declared fields
+- `field_swallow.t27` is the cost. On a type argument list left open by a comma, the fixed collector absorbs the following `name : type` pairs into the first field type: three declared fields become one, typed `Map<K,b:u8,c:u16`. The pre-fix collector truncated and kept three
+- That second one is a LOSS of fields on malformed input, and it corrects how the corpus differential should be read. "0 regressions" did not mean no field was lost; it meant field loss on malformed input was accepted as a tradeoff. Neither reading of that input is correct -- there is no correct reading -- so the test does not claim one is. It pins the loss at one level and inside one struct, so a change that swallows a whole body or crosses a declaration boundary fails here instead of passing as "still one field"
+- Diagnostic wording is not pinned. The recorded baseline read `Error: Parse error: Expected RBrace, got Eof` and the binary now prints `Error: Expected RBrace, got Eof`; asserting the word "parse" failed on that prefix alone while the parser was correct. The tests assert token names, which is what the invariant is about
+- **Rebase resolution.** Master had independently rewritten the PRIMARY field-type path to use `parse_type_annotation` (fixing `[]const u8` -> `[]constu8`) with checkpoint/restore and field-default parsing. That rewrite and this one are complementary, not rival: master's version still falls back to a naive raw-lexeme join that breaks on the first comma. The depth-aware collector from this branch now occupies that fallback, so both fixes survive -- the real type grammar leads, and the raw join behind it lets nesting decide the separator while `RBrace`/`Semicolon`/`Eof` still terminate unconditionally
+- `bootstrap/stage0/FROZEN_HASH` was regenerated from the resolved `compiler.rs` with `shasum -a 256`, not transcribed. The three `scripts/tri_loop/` helpers landed on master ahead of this branch in newer rewrites, so master's versions were kept and this branch's older drafts dropped
+- Entry migrated from `docs/NOW.md` to `docs/now/` (the layout #2298 introduced); the original entry was dated 2026-08-14. The branch's own commit had deleted the heading `# NOW -- BNF: the control that measures what ternary is worth (2026-08-09)` while keeping its body; `docs/NOW.md` is restored to master byte-for-byte, so that heading survives