Drop the output re-parse and the full pre-run snapshot - #11
Open
IPerception wants to merge 4 commits into
Open
Conversation
render() rebuilt the browsable document with parse(result.output) -- a second full parse, producing a second full set of segment objects, for a document processText already held. It now returns that document and render() takes it. Measured on 10/25/50 MB synthetic 837P, like for like: run time falls 26-37% (50 MB: 2565 ms -> 1848 ms). Retained heap is essentially flat, about 2%. PERFORMANCE.md predicted "roughly a third of peak heap" and that is wrong. The prediction treated the re-parse as a copy held alongside the mutated document, but the mutated document became unreachable the moment processText returned, so only one of the two was ever retained. What the re-parse actually cost was the time to do it. The doc is corrected in a later commit, with the measurements. The benchmark stopped counting a re-parse that no longer happens. Note it had been discarding its re-parse immediately while the app keeps it in state.doc forever, so its heap column understated the app; the baseline above was re-measured with that corrected. render.mjs section [6] covers what changed -- the document you browse, which parity.mjs does not see because the output string was never at risk. Verified load-bearing by restoring the re-parse, which fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfabdBfKu77QsNYVZkc3nn
Preparation for dropping the full pre-run snapshot from processText. Once the change list is built from marks instead of from a clone-and-compare over every segment, a change a rule makes without marking it is unrecoverable -- the before-value is gone the moment the rule overwrites it. Nothing checked that. _marks.mjs audits both directions and is called from the suites that already have each rule wired up, rather than becoming a tenth suite that would have to re-lift the engine to reach them: parity.mjs [13] for the two engine rules, deid.mjs [18] for DeidentifyRule. The two directions are not equally serious. "unmarked" must always be empty. "overclaimed" is expected and harmless -- StringReplaceRule counts a match even when the replacement equals it, so ANYTOWN -> ANYTOWN marks every hit and changes nothing. That case is now pinned explicitly, because it is the reason processText keeps sameSegment as a filter over marked segments instead of trusting the marks outright. All three rules pass as they stand; this commit changes no behaviour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfabdBfKu77QsNYVZkc3nn
processText cloned every segment in the document before the rules ran, so it could diff against that copy afterwards and work out what moved. On a 5M-segment file that is a second full working set built to find a few thousand changes. Each rule now copies a segment on its first write to it and carries the copy in the mark it already reported, so only changed segments are ever copied. All three rules had a single mutation site to hook -- put() in DeidentifyRule, and one assignment each in the other two -- so the capture is a few lines apiece. processText keeps the FIRST copy it sees per segment. Two rules can touch one segment, and the second rule's copy already contains the first rule's edit; the change list has to show what arrived in the file, not what the second rule happened to find. parity.mjs [14] pins this, and fails with a fabricated before value if the rule is flipped to last-writer-wins. sameSegment stays, as a filter over marked segments rather than a sweep over all of them. The marks say what a rule touched, not what it managed to change, and StringReplaceRule counts a match even when the replacement equals it. Dropping the filter would surface those as changes with identical before and after. What it no longer catches -- a rule editing without marking -- is now covered directly by the mark audits added in the previous commit. Measured on synthetic 837P, against the same tree with only this change absent: 100 MB 5502 ms -> 4057 ms (-26%) 125 MB 10812 ms -> 6968 ms (-36%) 50 MB 1848 ms -> 1770 ms (-4%) Retained heap is unchanged and the out-of-memory ceiling has not moved: the clone was transient, so it never showed up in a post-GC measurement. What it cost was the work of building it and the GC pressure of carrying it, which is why the gain grows with the file -- at 125 MB the extra copy was crowding the heap limit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfabdBfKu77QsNYVZkc3nn
…bers Two things in this file were wrong rather than merely stale. The heap column understated the app. The benchmark discarded its copy of the browsable document immediately; the app keeps it in state.doc for as long as the file is open. The 125 MB row moving from 1.99 GB to 2.90 GB is that correction, not a regression. And the column is heap after a forced GC, not peak heap, though it was labelled peak. That is what made the predictions for both items wrong: they were sold as memory savings on the theory that five copies were held at once, when the two being removed each lived and died inside a single run. Both delivered time instead -- together roughly halving run time at 100-125 MB -- and moved the heap column by about 2%. The reasoning error is written down next to the result, since the same mistake is available to anyone reading the copy table. Also records that the ceiling did not move: 150 MB was out of memory before and still is. Only the streaming parse moves that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfabdBfKu77QsNYVZkc3nn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
web/PERFORMANCE.mdbacklog items 2 and 3. Four commits, each reviewable on its own.What changed
Item 2 — stop re-parsing the output.
render()rebuilt the browsable documentwith
parse(result.output), a second full parse producing a second full set ofsegment objects for a document
processText()already held. It now returns thatdocument and
render()takes it.Item 3 — snapshot only what changes.
processText()cloned every segment beforethe rules ran so it could diff afterwards. Each rule now copies a segment on its
first write to it and carries the copy in the mark it already reported, so only
changed segments are ever copied. All three rules had a single mutation site to hook.
processTextkeeps the first copy per segment: two rules can touch one segment,and the second rule's copy already contains the first rule's edit, so last-writer-wins
would show a
beforethat never existed in the input file.sameSegmentwas kept, as a filter over marked segments rather than a sweep overall of them.
StringReplaceRulecounts a match even when the replacement equals it,so
ANYTOWN→ANYTOWNmarks every hit and changes nothing; without the filter eachwould surface in the Changes tab as a change with identical before and after.
Results
Measured against the same tree with only the change absent, not against the numbers
in the doc — those were taken on another machine.
Both were predicted as memory savings and delivered time savings instead. Item 2
was expected to remove "roughly a third of peak heap" and moved the heap column ~2%.
The reasoning error was treating the doc's five copies as five things held at once,
when the two removed were each built and discarded inside a single run. The
out-of-memory ceiling has not moved: 150 MB failed before and still does.
Two things in
PERFORMANCE.mdwere wrong rather than stale, and are corrected withthe measurements: the heap column understated the app (the benchmark discarded the
browsable document the app keeps in
state.doc), and it is heap after a forced GCrather than peak, which is precisely why it could not see either copy.
Correctness
Item 3 moves the correctness of the change list — and of the Limited Data Set's only
record of what it rewrote — onto the accuracy of each rule's
marks. Nothing checkedthat before, so
web/tests/_marks.mjswas added and committed first, passingagainst the old behaviour, before anything was removed. It audits both directions for
all three rules, from the suites that already have each rule wired up.
Two tests were verified load-bearing by breaking the code and watching them fail:
render.mjs[6]parity.mjs[14] with a fabricatedbeforeParity with the Python engine is byte-for-byte unchanged throughout, which is the
property that matters most here: none of this may alter a single byte of output.
🤖 Generated with Claude Code