Skip to content

feat(playwright-notion): extract comments and download attachments - #20

Open
xtieume wants to merge 5 commits into
mainfrom
feat/notion-comments-assets
Open

xtieume wants to merge 5 commits into
mainfrom
feat/notion-comments-assets

Conversation

@xtieume

@xtieume xtieume commented Sep 22, 2026

Copy link
Copy Markdown
Owner

Notion's native export drops every comment, and the converter fallback only linked attachments by their signed URL, which expires within days. Both gaps hit requirement pages hardest: that is where scope changes and final formulas get decided.

download.mjs now:

  • keeps discussion/comment/notion_user from the record map and writes .comments.md, numbered feat: add playwright-notion skill #1..#n with anchors, each thread linking back to its Notion discussion; commented blocks reference it inline
  • downloads attachments to assets// and rewrites links to the local copies, including files held in database row properties (NOTION_MAX_MB, NOTION_MEDIA, NOTION_NO_ASSETS)
  • deep-links headings and toggles to the exact block in Notion
  • resolves person and page mentions to names instead of @user / [[page]]
  • prefers loadCachedPageChunkV2 and takes spaceId from the response
  • reports block, file and comment counts per page, and names every file skipped for size or media type

The driver is skipped under NOTION_SELFTEST and playwright is imported lazily, so test-download.mjs can exercise the renderers without deps.

Notion's native export drops every comment, and the converter fallback
only linked attachments by their signed URL, which expires within days.
Both gaps hit requirement pages hardest: that is where scope changes and
final formulas get decided.

download.mjs now:
- keeps discussion/comment/notion_user from the record map and writes
  <page>.comments.md, numbered #1..#n with anchors, each thread linking
  back to its Notion discussion; commented blocks reference it inline
- downloads attachments to assets/<page>/ and rewrites links to the
  local copies, including files held in database row properties
  (NOTION_MAX_MB, NOTION_MEDIA, NOTION_NO_ASSETS)
- deep-links headings and toggles to the exact block in Notion
- resolves person and page mentions to names instead of @user / [[page]]
- prefers loadCachedPageChunkV2 and takes spaceId from the response
- reports block, file and comment counts per page, and names every file
  skipped for size or media type

The driver is skipped under NOTION_SELFTEST and playwright is imported
lazily, so test-download.mjs can exercise the renderers without deps.
loadPageChunk returns one level of the page and then reports an empty
cursor, so every heading came out with its children missing: a real page
extracted as 16 blocks and a single comment. Walk down from the root and
pull the missing records with syncRecordValues - blocks, then their
discussions, comments and authors.

The walk is bounded to the page: a chunk also carries unrelated blocks
from the sidebar, and following those crawls the whole workspace.

Also make Notion's relative in-page links absolute, so they still resolve
once the markdown lives outside the app, and depend on playwright-core
instead of playwright - the scripts only attach over CDP, so the browser
download was 500MB of nothing.

Verified on a real page: 16 blocks/1 comment before, 230 blocks, 9
comments and 27 downloaded files after.
Attaching to the user's everyday browser meant closing it, and CDP on a
branded build's default profile dir is refused outright. agent-browser.sh
launches the Chrome for Testing that ships with Playwright against a
profile under ~/.cache instead: the window appears once for the login,
and every run after that is headless, with the everyday browser never
touched.

Nothing is copied, so App-Bound Encryption is not in the way - the same
binary writes and reads its own cookies. start-browser.sh stays for the
case where a page is only reachable from the personal profile.
…tion

The skill was never really about Notion: it is about pulling a
requirement, its comments and its attachments out of a source that only
a logged-in browser can reach. Slack and GitHub have the same problem,
so they now share the skill and its output contract.

doc.mjs owns that contract - comment numbering and anchors, the
reference line the body carries, the attachment download - so a new
source only writes the part that is actually specific to it.

slack.mjs reads the web client's token and calls conversations.history /
replies in the page; the DOM is useless there because Slack
virtual-scrolls. Replies become the comments document.

github.mjs needs no browser at all: gh already holds the credentials. It
takes the description, conversation comments, review verdicts and the
review comments left on lines of the diff, which are a separate API and
usually where the code-level decisions are.

Scripts are now named for their source, and all of them take a bare URL
as well as a file of URLs.
Closing the last window leaves the browser process alive with an empty
target list, and connectOverCDP then fails with "Browser context
management is not supported", which reads like a version problem rather
than an empty browser. connectCdp opens a target first.

The github script only ever wrapped `gh pr view --json` and the review
comments API, which the agent can call directly, so it earned nothing by
being a script. The skill says which calls to use instead.

Also fix agent-browser.sh --headed: an empty array expansion aborts
under `set -u` on the bash 3.2 that ships with macOS, so asking for a
window was the one path that could not launch.

Slack verified end to end on a real thread: message plus 5 replies, user
names and mentions resolved, permalinks carrying thread_ts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant