Skip to content

fix: integrate sharded frame trees and parser-v3 cache reuse - #273

Closed
codeasier wants to merge 9 commits into
mainfrom
codex/integrate-frame-tree-269
Closed

codeasier wants to merge 9 commits into
mainfrom
codex/integrate-frame-tree-269

Conversation

@codeasier

Copy link
Copy Markdown
Owner

Relationship to #269

This is a maintainer-owned integration candidate for #269, not an update to its author's fork. It preserves #269's three original commits unchanged and merges current main (3f167ec, including #271 onboarding and the descriptor-scoped fstat regression fix) through a two-parent integration commit. The original PR remains open and untouched.

Behavior carried forward

  • Sharded native/msinsight frame trees retain ordered caller paths and conserve inclusive/self bytes, requested bytes and block counts; legacy text-only evidence remains explicitly incomplete.
  • Latest-request-wins frame-tree cancellation covers Python and SQLite work, queued obsolete requests, analyzer close/cache invalidation and independent worker connection ownership.
  • Compatible parser-v3 import reuse checks source identity, complete member inventory, required schema and indexed endpoints, not arbitrary row content, DB hashes or full frame scans. It does not seed query-validation caches or receipts. Native content validation remains intact.
  • Parser-v2 compatible caches are preserved and require a new output directory. Existing compatible targets are never replaced, including with --force.

Integration fixes

  1. Keep docs: streamline first-use reports and Agent Skills onboarding #271's complete first-use/onboarding and skill recovery changes while retaining feat: add sharded frame-tree analysis and lightweight cache reuse #269's bilingual frame-tree/cache/cancellation guidance.
  2. Repair the stale interop assertion that expected arbitrary edited row values to fail lightweight import. Existing feat: add sharded frame-tree analysis and lightweight cache reuse #269 tests and docs already deliberately permit that reuse. The regression now proves edited bytes are not overwritten, and new owned/external member/schema/endpoint failures prove every remaining file is preserved, with and without force.
  3. Independent static review found that JSON-encodable frame metadata deeper than 128 could make optional evidence abort native text import. Validate normalized JSON depth before any stack rows are inserted; omit invalid optional evidence as a whole. New native/compatible boundary cases check depth 128/129, preserved text, following-event coverage, no partial frame rows and read-only reuse/query.

Validation and remaining gates

  • Independent review of the complete candidate diff found no remaining confirmed correctness blocker after the optional-frame fix.
  • Python AST parsing and git diff --check passed. Project unit tests/builds are intentionally delegated to the existing CI rather than duplicated locally.
  • CI for this exact head is pending. Python 3.10–3.13, lint/type/format, fixture provenance, required Chromium interactions, clean-wheel and installed-wheel dataset acceptance must be evaluated from their actual results; old feat: add sharded frame-tree analysis and lightweight cache reuse #269 failure/skip outcomes are not a pass for this candidate.
  • Prior real-data and GUI evidence recorded by feat: add sharded frame-tree analysis and lightweight cache reuse #269 predates final cancellation/parser-v3 changes. This candidate does not claim fresh upstream GUI, real provider, release or deployment acceptance.
  • No tags, releases, deployments, credentials or workflow gates are changed. Draft remains until exact-head CI and acceptance/review requirements are satisfied.

Snapshot Runtime Provenance

  • Snapshot provenance evaluation completed.
    Snapshot provenance decision: updated

shaanjiangcun and others added 7 commits October 29, 2026 19:50
Preserve PR #269 history alongside #271 onboarding and fstat isolation.
Align interoperability regression with existing lightweight cache semantics,
while verifying structural rejection and byte preservation with force.
Use the reader's normalized JSON depth limit before storing an event stack.
Preserve text-compatible imports when optional frame metadata is too deep,
with native and compatible boundary and byte-preservation regressions.
Retain historical schema identity without claiming current GUI reuse.
Warn about known v2 reparse behavior and require isolated copied inputs,
a verified consumer build and separate acceptance evidence.

Copy link
Copy Markdown
Owner Author

Current candidate: 5fdc8375cfdfbaeb71e1746249d0d8c3853c33fe (normal fast-forward; latest main 666c9b3 and #272 badges preserved).

Independent source review found a known interoperability blocker, not just missing GUI evidence: pinned 101f65b hardcodes parser-v2, and checked current upstream master 5c722719e1d288106f8d18588b0a33d979a4065b still computes parser-v2. Current parser-v3 exports cannot be a matching no-reparse cache for these consumers. The old pinned parser's mismatch path removes manifest/log and starts the producer. Do not open the only copy of a v3 export in that GUI.

This commit corrects bilingual current-target claims and documents an isolated-copy retest plan. targetRevision is retained as a historical schema/cache-identity anchor: it does not select a consumer or prove GUI support. Runtime parser-v3 choice, historical fixtures/receipts and strict hash gates are unchanged. Product direction (existing v2-consumer reuse versus v3 awaiting upstream adaptation) remains unresolved; no consumer target is invented and this PR must not merge on a green unit suite alone.

Ordinary exact-head CI is being rerun. The installed-wheel dataset/200-seed acceptance job only runs on workflow_dispatch or schedule, so its PR-event skip is not a pass. Once the head is final, run the existing Test workflow on codex/integrate-frame-tree-269 (no inputs) and verify the resulting head_sha; retain dataset/differential artifacts. No workflow gate is changed, no real GUI run is claimed, and no tag/release/deployment is performed.

Compare the unchanged event base fields exactly while independently verifying
ordered frame fields through the installed standalone CLI. Require complete
source coverage, frame identities, and nonempty evidence; add refusal tests.

Merge current main to preserve PR #274 bilingual README updates alongside
parser-v3 cache-preservation and latest-request cancellation contracts.

Copy link
Copy Markdown
Owner Author

Follow-up at 6cf8a152adca5f2d450a0c6522b37dd2a3b380f5

The preceding head passed the full Python 3.13 test suite (3,142 tests), coverage floors, wheel build and source audit, then exposed a stale clean-wheel comparison: dataset event now includes seven ordered-frame evidence fields absent from the standalone default event row. This head preserves exact parity for all nine base fields and independently queries the installed standalone event_frames command to check ordered filename/line/name values, duplicates, counts, event/position ownership and completeness. Dataset source coverage and full raw-array identity/namespace are checked separately; arbitrary raw extra fields are not claimed independently compared by that public standalone surface. Empty boundary stacks require additional nonempty frame evidence, not a vacuous pass. Added positive/refusal regressions and both-format orchestration coverage; no workflow/gate weakening.

Also merged current main f876665 as a second parent, preserving PR #274's complete bilingual README updates alongside this PR's cache/frame/cancellation additions. Independent static review found no blocker in these changes; source/test static checks passed, but they do not replace actual installed-wheel execution.

Fresh exact-head validation is running: Test, Platform compatibility. The parser-v3/current-msinsight incompatibility and pending product decision described above remain unchanged. Manual/scheduled dataset acceptance and applicable real-GUI acceptance remain outstanding. This PR remains Draft and must not be merged on ordinary CI alone.

Copy link
Copy Markdown
Owner Author

Final ordinary-CI receipt — 6cf8a152adca5f2d450a0c6522b37dd2a3b380f5

Both exact-head workflows are now terminal success:

  • Test / 41224192339: Python 3.13 full suite 3,234 passed, 14 skipped, 7 deselected, 4 subtests; 93% total coverage and all critical floors passed. Python 3.10–3.12, lint/format/type checking, provenance and required Chromium interactions also passed.
  • The same run built/audited the wheel and passed the actual isolated clean-wheel CLI/skills acceptance. Native and compatibility exports each covered 8,180 real events in two shards, with peak/base-event parity, complete ordered-frame field/identity checks and nonempty frame evidence. This is offline CLI acceptance, not a live-agent or GUI claim.
  • Platform compatibility / 41224192340: all four Windows partitions and macOS passed (macOS 3,218 passed). The tested merge tree matches the candidate tree dff8133f06e5853d7ba6c8f11466e560d13c8687.

Final recheck: actual main remains f876665, included as a parent; the candidate is clean/mergeable, independently reviewed, but remains Draft / not eligible to merge.

Outstanding gates:

  1. Resolve the product decision: current parser-v3 exports are incompatible with the checked parser-v2 msinsight cache consumers. The historical targetRevision is not a verified v3 consumer target.
  2. On the final chosen head, manually run Test, select branch codex/integrate-frame-tree-269, leave inputs unchanged (this workflow defines none), and verify its run head_sha exactly matches the chosen head. The Installed-wheel dataset acceptance job must actually execute/pass its required installed-wheel dataset checks and 200-seed paired differential acceptance, with both evidence artifacts present. Its PR-event skip here is not acceptance.
  3. After a matching fixed consumer/build is established, run the documented GUI acceptance using a separately copied trusted pickle and adjacent candidate export in a new isolated directory, never the only copy or a parser-v2 cache. Record build/wheel/source identities, cache UP_TO_DATE and no-producer-invocation evidence, cross-shard curves/blocks, repeated navigation/cancellation, and before/after inventories. The checked v2 consumers cannot establish v3 reuse; their reparse path may modify the copied cache. Historical GUI receipts do not cover this head.

Original #269 and its foreign-fork branch remain unchanged. No tag, release, deployment or merge was performed.

Copy link
Copy Markdown
Owner Author

Closing this parallel integration candidate to consolidate follow-up work in #269. This is not a rejection of the feature and does not mean #269 has absorbed every change here. The branch codex/integrate-frame-tree-269 is retained at 6cf8a152; no branch or commit is being deleted.

Comparison with #269 at 39d0793 confirms that its core optional-frame depth preflight, stale event-comparator error and interop content-edit expectation have been repaired. It also fixes the allocation-ID membership O(N²) scan still present in this candidate. The following remaining patch themes should be considered in #269 rather than lost with this PR:

  1. Bilingual compatibility documentation: historical 101f65b versus current v3, targetRevision meaning, old-consumer reparse/cache-mutation warning, and isolated-copy GUI acceptance plan. See the corrected protocol and safe retest.
  2. Stronger clean-wheel evidence: independent installed standalone event_frames comparison, strict base-field types and complete references, source identity/coverage, and nonempty frame evidence. See comparator and CLI proof queries.
  3. Compatible-cache preservation regressions for owned/external caches, missing member/schema/endpoint damage, both force modes and unchanged focus: tests.
  4. Depth-boundary regressions: depth128 accepted/depth129 omitted, a following valid event still covered, and unchanged bytes through reuse/read-only queries: tests. The alternative shared-helper layout in feat: add sharded frame-tree analysis and lightweight cache reuse #269 need not be mechanically replaced.

Items 2–4 identify remaining verification coverage, not a claim that the original fixed product defects still reproduce. #269 still needs current-main conflict resolution, review and CI for its final commit, plus the applicable installed-wheel/differential and coordinated-consumer acceptance. This PR's historical passing runs do not transfer to #269.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants