Skip to content

test: Phase 1 exit check as an expected-fail e2e spec - #135

Open
m-vawter wants to merge 1 commit into
mainfrom
test/phase-1-exit-check
Open

m-vawter wants to merge 1 commit into
mainfrom
test/phase-1-exit-check

Conversation

@m-vawter

Copy link
Copy Markdown
Collaborator

NEXT.md's Phase 1 exit says "an app that reaches for parent.document or the host's localStorage fails". Nothing checks that today, so this adds the check now, ahead of the code that will meet it.

  • tests/e2e/sandbox-isolation.spec.ts:
    • plants a fake provider key where Settings stores it;
    • installs a probe app that tries the host's localStorage and window.parent.document on mount, and reports what it got to /__probe;
    • reads that report at the network layer, which also sees requests from iframes, so the check doesn't depend on how 1.2 mounts apps.
  • Two tests:
    • control: the probe runs and reports;
    • isolation: it can't read either. This one is test.fail() until 1.2, so CI stays green while the gap is open.
  • When 1.2 lands, Playwright reports the isolation test as passing unexpectedly. That's the cue to delete the test.fail() line in the 1.2 PR. If 1.2 changes how app code can reach the network, the control is the part to adjust.
  • support/desktop.ts gains installApp and openApp hook helpers. tests/e2e/README.md gets a short section on expected-fail specs.

@Sullux, this is meant as a ready target for 1.2; happy to reshape it to match the bridge you have in mind.

Closes #134.

Verification

Each run merged the branch into current main (d2f3679) in a clean container:

  • tsc -p tests/e2e and eslint tests/e2e are clean.
  • The full suite passes, 10/10: the control passes and the isolation test counts as an expected failure.
  • With test.fail() removed, the isolation test fails for the right reason:
    Expected substring: not "e2e-planted-key-isolation-probe", Received string: "{"provider":"openai",…,"apiKey":"e2e-planted-key-isolation-probe",…}".
    So today an app can read the stored key.
  • markdownlint on the README: 0 errors.

🤖 Generated with Claude Code

NEXT.md's Phase 1 exit says an app that reaches for parent.document or
the host's localStorage fails. sandbox-isolation.spec.ts installs a
probe app that tries both on mount and reports what it got to /__probe,
read at the network layer so the check doesn't depend on how 1.2 mounts
apps. The isolation test is test.fail() until the sandboxed runner
lands; a control test proves the probe runs. support/desktop.ts gains
installApp and openApp hook helpers, and the e2e README describes
expected-fail specs.

Closes #134.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: Phase 1 exit check as an expected-fail e2e spec (1.2 follow-up)

1 participant