Form-field fast path silently drops every row past page 1 of a multi-page eval-gen PDF

[gavinbee]

The fast path looks widgets up by exact name, but a multi-page PDF does not have the names the template expects on pages 2+. Those rows come back empty, and because main.py commits to the fast path whenever the PDF has form fields, there is no vision fallback to catch it. Officials are silently missing from the parse.

Cause

deck-eval-gen builds a multi-session form by merging one filled copy of the blank template per page. PyMuPDF disambiguates the repeated field names on merge by appending an xref number, so page 2 carries:

Name of OfficialRow1 [226]
Name of OfficialRow2 [242]
...

form_extract.extract_page resolves per-row widgets by exact name:

widget_name = widget_template.replace("{i}", str(row_index))
raw = widgets.get(widget_name)
if raw is None:
    continue  # widget not present on this page

Name of OfficialRow1 never matches Name of OfficialRow1 [226], so every row on the page is skipped as absent.

main.py then offers no way out:

if pdf_io.has_form_fields(pdf_path):
    return _run_form_field(args, pdf_path)

A PDF with form fields never reaches the vision path, so the empty pages are not recovered — they are just gone.

Reproduction

Generate an 11-official session (2 pages at rows_per_page=9) with deck-eval-gen, then run the real extract_page against swim_ontario_v1:

template: swim_ontario_v1, rows_per_page=9
pages: 2

--- page 1 ---
  widget names as they appear : ["Name of OfficialRow1", "Name of OfficialRow2"] ...
  values actually in the form : ["Official Number1", "Official Number2", "Official Number3"]
  officials the parser recovered: [...] (total 9)

--- page 2 ---
  widget names as they appear : ["Name of OfficialRow1 [226]", "Name of OfficialRow2 [242]"] ...
  values actually in the form : ["Official Number10", "Official Number11"]
  officials the parser recovered: [] (total 0)

The values are present and correct in the PDF. The parser recovers none of them.

Not a regression: verified against deck-eval-gen at its initial commit, which produces byte-identical field names. Any multi-page form ever generated is affected. Single-page forms — the common case, 9 or fewer officials per session — are fine, which is presumably why this has gone unnoticed.

Suggested fix

Normalise the suffix when building the widget map, in pdf_io.read_widgets or at lookup in form_extract:

_XREF_SUFFIX_RE = re.compile(r"\s*\[\d+\]$")
name = _XREF_SUFFIX_RE.sub("", w.field_name)

deck-eval-gen does exactly this in its read_field_values test helper. Doing it here fixes it for every producer, including forms filled by other tools, rather than making it deck-eval-gen's job to emit unsuffixed names.

Worth a regression test with a genuinely multi-page fixture — a single-page fixture cannot catch this.

Related

• Found while working swimblocks/deck-eval-gen#17. Documented on that side in AGENTS.md so nobody re-derives it.
• Also worth considering separately: _run_form_field returning a parse with zero rows is not obviously a success. A sanity check — form fields present but nothing extracted — would have surfaced this years earlier.