perf: avoid copying chunked matches before sorting

[cgfixit]

Proposed changes

ChunkedKeywordPattern.finditer already owns a private list of all regex matches.
It previously passed that list to sorted, allocating a second list of references.
Sort the existing list in place and yield it, retaining the same (start, end) key
and stable pattern priority for ties.

Add one parametrized regression covering empty input, no matches, overlapping spans,
empty matches, and duplicate spans from different patterns. No match deduplication
or public behavior is changed.

Invariant/governance impact: lazy model loading, signatures, output/state formats,
thresholds, dependency declarations, and CI gates are unchanged.

Types of changes

• Performance: temporary allocation reduction
• Regression coverage
• Public behavior or dependency change

Benefits / why

Windows Python 3.12.0rc3, tracemalloc around full iterator consumption, identical
compiled regex patterns/text for before and after:

Workload	Matches	Before peak	After peak	Reduction
2 chunks, 48,000 characters	8,000	2,113,802 B	2,049,674 B	64,128 B (3.0%)
8 chunks, 192,000 characters	32,000	8,467,866 B	8,211,691 B	256,175 B (3.0%)

Synthetic input repeats term0 ... termN 4,000 times, one compiled word-boundary
pattern per chunk. Timing uses nine alternating before/after rounds of five calls:
2 chunks 10.37→11.53 ms, 8 chunks 106.28→87.81 ms. Timing is mixed/noisy; this PR claims
reduced allocation, not a throughput speedup. The before method comes from an archive
of the exact base commit. 300 deterministic generated cases also preserve identical
match spans, text, and pattern priority.

A further check on all 323,590 characters of src/sample_input.txt with 12 custom
keywords split across two chunks produced identical 215 matches: peak allocation
76,798→74,895 B (2.5% lower), median 167.12→158.27 ms over three single-call rounds.
Keywords: ransomware, CVE, BERT, RAG, OSINT, malware, model, safety, alignment,
embedding, agent, privacy. The slower 363-keyword corpus timing run was interrupted;
no result is claimed for that configuration.

Risks to monitor

This is a small memory reduction. Matching still materializes all matches and sorts
them; it does not provide bounded-memory streaming. Normal single-pattern matching
does not use this adapter. The optional integration suite remains broken at its
existing AutoTokenizer mock boundary; errors were reproduced on clean origin/main.

Completed checklist

• Local main fast-forwarded; feature branch based on latest origin/main
• Ruff lint and formatting pass
• Strict mypy passes on Python 3.12 and 3.13 (10 source files)
• Unit coverage: 121 passing cases on each Python version
• Constrained isolated environments; pip check passes on both versions
• Production CI CLI smoke body passes on Python 3.12 with fake model boundaries
• Full local test suite compared to clean origin/main: before 119 passed / 16
  errors; after 122 passed / same 16 errors (121 unit + 1 integration success)
• Whitespace, scope, and generated-artifact staging reviewed
• Full suite green: pre-existing integration fixture errors, no tests skipped or weakened
• Real-model inference: not run; no model/tokenization/embedding behavior changes
• Local Gitleaks executable: unavailable; GitHub's configured scan will run on this PR

Further comments

Python 3.12 is the host's existing 3.12.0rc3; Python 3.13 is stable 3.13.12.
Only stemmer.py and its unit tests are committed; benchmark logs, baseline archive,
model-free smoke outputs, and virtual environments remain ignored local artifacts.

Branch naming / merge order / base commit

Branch: codex/optimize-chunk-sort. Target: main.
Base: 8b2087191122770449e704da19f0f2d47de16ae9 (merged PR #33).
Standalone draft PR; no unmerged dependencies.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-06T15:36:00.301480Z	49d764c	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.