Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 49 additions & 6 deletions bootstrap/src/compiler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2413,12 +2413,55 @@ impl Parser {
}
} else {
self.restore_state(checkpoint);
// Collect type tokens until comma, semicolon, or closing brace
while self.current.kind != TokenKind::Comma
&& self.current.kind != TokenKind::Semicolon
&& self.current.kind != TokenKind::RBrace
&& self.current.kind != TokenKind::Eof
{
// W699 (#2127, third attempt). Collect raw type lexemes, but let
// NESTING decide only the SEPARATOR, never the BLOCK TERMINATOR.
//
// The two reverted attempts both failed the same way: they allowed a
// depth counter to suppress `RBrace`/`Eof`, so an unbalanced type ate
// the struct's closing brace (attempt a) or spun forever at Eof, where
// the lexer yields `Eof` indefinitely (attempt b).
//
// Invariant enforced here:
// RBrace, Semicolon and Eof terminate UNCONDITIONALLY at any depth;
// only Comma consults `depth`.
// Termination therefore does not depend on the input being well-formed:
// every iteration consumes exactly one token and Eof always exits.
let mut depth: i32 = 0;
let mut prev_was_ident = false;
loop {
let k = self.current.kind.clone();
// Unconditional terminators — never gated by depth.
if k == TokenKind::RBrace
|| k == TokenKind::Semicolon
|| k == TokenKind::Eof
{
break;
}
if k == TokenKind::Comma && depth <= 0 {
break;
}
match k {
TokenKind::LParen | TokenKind::LBracket => depth += 1,
TokenKind::RParen | TokenKind::RBracket => {
if depth > 0 {
depth -= 1;
}
}
// `<` opens a generic argument list only directly after an
// identifier (`Map<`); as an operator it must not open depth.
TokenKind::Lt if prev_was_ident => depth += 1,
TokenKind::Gt => {
if depth > 0 {
depth -= 1;
}
}
TokenKind::ShiftRight => {
// `>>` closes two generic levels (`Vec<Vec<u8>>`).
depth = if depth >= 2 { depth - 2 } else { 0 };
}
_ => {}
}
prev_was_ident = k == TokenKind::Ident;
type_str.push_str(&self.current.lexeme);
self.advance();
}
Expand Down
2 changes: 1 addition & 1 deletion bootstrap/stage0/FROZEN_HASH
Original file line number Diff line number Diff line change
@@ -1 +1 @@
c3ec9fba947b9d5845af10d1270619f6e49298698139d0e227ab669f93868bbf bootstrap/src/compiler.rs
65f033d04125aea94dba708fdd7a565ddead2ec492f434268a11985eba1cf705 bootstrap/src/compiler.rs
13 changes: 13 additions & 0 deletions docs/now/2026-08-21-struct-body-terminator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# NOW -- parser: nesting may decide the separator, never the terminator (2026-08-21)

## The terminator fixtures become tests, with a hard timeout and an honest cost (Closes #2127)

- Five tests in `bootstrap/tests/struct_body_terminator.rs`, over five fixtures. Three are liveness tests under a 10 s in-process wall-clock ceiling: truncated input leaves the bracket depth positive, so a depth-gated terminator is never accepted and the field loop runs past the end of the token stream. Asserting on the error text alone would pass even on a hang, because a hung process never produces text to compare -- it would sit until the CI job timeout and report as infrastructure flake rather than as this defect. The child is killed before the assertion fires, so a wedged parser cannot outlive the test binary
- Those three do NOT discriminate the fixed parser from the unfixed one. Checked against the pre-fix binary: identical verdict, identical message, identical timing on all three. They are regression guards, not evidence
- `semicolon_phantom.t27` is the improvement. `Map<K, V;` made the pre-fix collector emit a phantom field named `V` with an empty type -- an identifier from inside a type argument list promoted to a struct field. After the fix the struct has exactly its two declared fields
- `field_swallow.t27` is the cost. On a type argument list left open by a comma, the fixed collector absorbs the following `name : type` pairs into the first field type: three declared fields become one, typed `Map<K,b:u8,c:u16`. The pre-fix collector truncated and kept three
- That second one is a LOSS of fields on malformed input, and it corrects how the corpus differential should be read. "0 regressions" did not mean no field was lost; it meant field loss on malformed input was accepted as a tradeoff. Neither reading of that input is correct -- there is no correct reading -- so the test does not claim one is. It pins the loss at one level and inside one struct, so a change that swallows a whole body or crosses a declaration boundary fails here instead of passing as "still one field"
- Diagnostic wording is not pinned. The recorded baseline read `Error: Parse error: Expected RBrace, got Eof` and the binary now prints `Error: Expected RBrace, got Eof`; asserting the word "parse" failed on that prefix alone while the parser was correct. The tests assert token names, which is what the invariant is about
- **Rebase resolution.** Master had independently rewritten the PRIMARY field-type path to use `parse_type_annotation` (fixing `[]const u8` -> `[]constu8`) with checkpoint/restore and field-default parsing. That rewrite and this one are complementary, not rival: master's version still falls back to a naive raw-lexeme join that breaks on the first comma. The depth-aware collector from this branch now occupies that fallback, so both fixes survive -- the real type grammar leads, and the raw join behind it lets nesting decide the separator while `RBrace`/`Semicolon`/`Eof` still terminate unconditionally
- `bootstrap/stage0/FROZEN_HASH` was regenerated from the resolved `compiler.rs` with `shasum -a 256`, not transcribed. The three `scripts/tri_loop/` helpers landed on master ahead of this branch in newer rewrites, so master's versions were kept and this branch's older drafts dropped
- Entry migrated from `docs/NOW.md` to `docs/now/` (the layout #2298 introduced); the original entry was dated 2026-08-14. The branch's own commit had deleted the heading `# NOW -- BNF: the control that measures what ternary is worth (2026-08-09)` while keeping its body; `docs/NOW.md` is restored to master byte-for-byte, so that heading survives
Loading