Skip to content

[investigation/do-not-merge] surface legacy instrument error.stack - #1212

Draft
vringar wants to merge 2 commits into
masterfrom
investigate/legacy-error-stack-trace
Draft

vringar wants to merge 2 commits into
masterfrom
investigate/legacy-error-stack-trace

Conversation

@vringar

@vringar vringar commented Jun 29, 2026 •

Copy link
Copy Markdown
Contributor

Investigation / do-not-merge

This PR is stacked on #1211 (test/assert-rewrite-helpers). It now carries
two investigation commits (both end in pytest.fail to print their evidence
in the CI log; both EXPECTED TO FAIL by design — do not merge):

  1. surface ... error.stack — reproduces fix(extension): strip moz-extension frames from legacy instrument error stacks #1207's flow and dumps the literal
    page-observable error.stack of a thrown instrumented call.
  2. demonstrate that popping the top stack frame cannot hide the wrapper —
    answers the follow-up question below.

Commit 1 — what the leak actually is

Faithfully reproduces #1207's (harden/legacy-error-stacks) flow: instruments
window.atob, page code calls atob("not valid base64 !@#$%") so it throws an
InvalidCharacterError, the page catches it and stashes e.stack into
document.title; a CaptureStackCommand reads the title back across the process
boundary.

#1207 asserts "moz-extension://" not in stack — a no-op: the legacy
instrument runs in the page world, so there is never a moz-extension://
frame to leak (the assertion passes vacuously). This commit instead surfaces
the captured stack via pytest.fail. The real, page-observable leak #1207's
assertion misses is a page-attributed wrapper frame
(getInstrumentJS/instrumentFunction/<) sitting as the top frame:

getInstrumentJS/instrumentFunction/<@<page-url>:<line>:<col>   <- wrapper (frame 0)
<page-call-site>@<page-url>:<line>:<col>                       <- page's atob() call site

Commit 2 — can the instrument "pop the top frame" to hide that wrapper?

The follow-up hypothesis: have the page-world instrument patch error.stack to
drop the top frame, hiding the wrapper. Verdict (empirically): no — this is
another page-world cross-realm ceiling (the #56/#57 family), not a cheap win.

Do not ship an instrument patch. Measured on Firefox 152 (also reproduced
direct-selenium with the REAL inlined getInstrumentJS):

How Firefox exposes error.stack (dictates the patch). Error.prototype.stack
is a native accessor (get stack, [native code]); instances have no own
stack data property
. So a patch must redefine that accessor. But
DOMException.prototype.stack is a SEPARATE native accessor — and the
#1207/#1212 throw cases (atob, getImageData) raise a DOMException
(InvalidCharacterError / IndexSizeError), whose stack does not come from
Error.prototype. So an Error.prototype.stack patch is inert for exactly
the cases #1207 cared about (dom_hidesWrapper: false, both variants); you would
have to also patch every DOMException/Error subtype's stack accessor.

Two patch variants, where the patch does apply (a real Error,
JSON.parse("{bad") → SyntaxError, routed through the wrapper):

measure crude (drop top frame) targeted (drop only instrument frames)
hides wrapper (real Error)? yes yes
over-pops a legit page Error? YES — eats the genuine top frame no
patch page-observable? YES — getter non-native (get, not get stack; JS-body toString) YES — same
cross-realm bypassable? YES YES

Over-pop (crude), verbatim. A legit page Error with no wrapper on its
stack — crude silently strips its real top frame legitPlainErrorTop:

main-realm (crude patch):  ["measure :130:21", "<anon> ...", "<anon> ..."]   # legitPlainErrorTop EATEN
targeted patch:            ["legitPlainErrorTop :53:18", ...]                # preserved

Crude corrupts the top frame of every page Error — a correctness disaster and
itself a trivial detector (function f(){return new Error().stack} f() no longer
shows f). Targeted avoids over-pop but is still non-native-detectable.

Cross-realm bypass (both variants), verbatim. The same Error, read main-realm
(patched, wrapper hidden) vs. through a pristine, never-patched Error.prototype
stack getter pulled from a same-origin <iframe> realm:

main-realm (patched):   ["pageThrowsReal :50:16", "grab :57:13", "measure ...", ...]
iframe-realm (native):  ["getInstrumentJS/instrumentFunction/< :33:27",        # wrapper REVEALED
                         "pageThrowsReal :50:16", "grab :57:13", "measure ...", ...]

The iframe realm's getter stays native and untouched by the main-realm patch and
re-derives the unfiltered stack, wrapper and all — the same architectural wall as
the #57 toString bypass. A main-realm-only patch cannot close it.

Why net-negative: inert for the actual DOMException throws unless you also
patch every error subtype's stack; detectable (non-native accessor — the same
toString-wall tell as #57); crude over-pops legitimate page frames; targeted still
non-native-detectable and still cross-realm-bypassed. The wrapper-frame leak is a
property of running in the page world; a page-world stack-getter patch is defeated
by the same cross-realm ceiling as the toString leak.

The second commit's test asserts this ceiling, then pytest.fail-dumps the full
measured JSON so the literal stacks appear in the CI log.

Purpose

Let reviewers read the literal page-observable error.stack (commit 1) and the
empirical evidence that "pop the top frame" is a ceiling, not a fix (commit 2),
against ground truth — before deciding whether any error-stack hardening is worth
shipping for the legacy instrument.

@codecov

codecov Bot commented Jun 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 8.86076% with 72 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.26%. Comparing base (975b09c) to head (eba7f78).

Files with missing lines Patch % Lines
test/test_js_instrument_stack_pop.py 6.97% 40 Missing ⚠️
test/test_js_instrument_error_stack.py 11.11% 32 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1212      +/-   ##
==========================================
- Coverage   62.31%   61.26%   -1.06%     
==========================================
  Files          40       42       +2     
  Lines        3930     4009      +79     
==========================================
+ Hits         2449     2456       +7     
- Misses       1481     1553      +72     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Base automatically changed from test/assert-rewrite-helpers to master June 29, 2026 23:46
@vringar
vringar force-pushed the investigate/legacy-error-stack-trace branch from 37ddc30 to 6c5ad4d Compare July 20, 2026 22:54
@vringar
vringar force-pushed the investigate/legacy-error-stack-trace branch from 6c5ad4d to 70d3d73 Compare August 25, 2026 09:18
@vringar
vringar force-pushed the investigate/legacy-error-stack-trace branch from 70d3d73 to cd44a22 Compare September 6, 2026 14:55
…gacy instrument wrapper (investigation, do-not-merge)

Empirical companion to the error.stack surfacing test. A self-contained page
models the page-world instrument wrapper frame, installs an
Error.prototype.stack getter patch in two variants (crude = drop top frame;
targeted = drop only instrument frames), and measures the cost.

Findings (Firefox 152): the wrapper-throwing web-API cases (atob/getImageData)
raise DOMExceptions whose stack comes from a SEPARATE accessor, so the patch is
inert for them; where it applies (real Errors) crude over-pops legitimate page
frames, both variants leave a page-detectable non-native stack getter, and both
are defeated cross-realm by a pristine iframe-realm getter. Popping the top
frame is a page-world cross-realm ceiling, not a viable fix.
@vringar
vringar force-pushed the investigate/legacy-error-stack-trace branch from cd44a22 to eba7f78 Compare September 6, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant