Skip to content

[compass] Fix: a scroll's reply came before the scroll was drawn, so the next screenshot was stale (B-055) - #87

Merged
skulitom merged 4 commits into
mainfrom
compass/anode-fix-2026-10-22-scroll-settle
Oct 22, 2026
Merged

skulitom merged 4 commits into
mainfrom
compass/anode-fix-2026-10-22-scroll-settle

Conversation

@skulitom

@skulitom skulitom commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Report

Compass inbox entry B-055 (C:\DEV\Compass\ANODE-BUGS.md, filed 2026-10-22 by compass-play-upload). In seat Chrome on Anode 0.11.3, seat_scroll -5 (or seat_key pagedown) was followed by a seat_screenshot that still showed the page unscrolled; a later screenshot showed it scrolled. A coordinate click aimed from the stale screenshot landed on a tooltip in the app list instead of the "Policy status" link (13:14, and again at 13:15 and 13:17 on AdMob). The same "scrolls apply late" was noted on 10-12 and 10-19.

Root cause

input.scroll in the seat host called InputInjector.Scroll (one SendInput wheel event) and answered ok at once. SendInput only queues the event. Chrome, Edge, Firefox, WPF and UWP lists animate a wheel or PageDown scroll over several frames, and a seat can draw fewer frames than the user's own screen, so the reply, and a screenshot taken straight after it, came before the scroll was drawn. Nothing in the reply said the screen was still moving.

Fix

New Core/Capture/ScreenSettle.cs, used by the seat host around input.scroll and around input.key chords containing PageUp, PageDown, Home or End (VK 0x21 to 0x24, any modifiers). Every other key is unchanged.

  • Two GDI frames 40 ms apart before the input. If they differ (a video, a game, an animation), the input runs and the reply says changing with no wait, so games lose about 40 ms, not seconds.
  • After the input: a frame every 40 ms until 200 ms of quiet after a change (settled), 500 ms with no change (unchanged), or the cap of 2.5 s, or half the request's time left if shorter (changing). Cancellation or a failed capture gives unknown.
  • The result has screen, screenWaitMs and a summary, such as "Scrolled 5 notches down; the screen settled 320 ms later." The input runs at most once: a capture failure never skips or repeats it, and an input error still reaches the caller.
  • A request cancelled before its input (its deadline, or a stopping seat, during the ~40 ms baseline) sends nothing and fails as timed out, as InputInjector.Press already does for a key (found by the Codex review of the first push, fixed in 9bcc07c).
  • Frames are compared by a SHA-256 of their size and pixels, hashed row by row from one reused bitmap, so a scroll holds what a screenshot does (the first push kept two full-size copies as well, about 95 MB at 4K and 768 MB at 8192x8192; flagged by the GitHub Codex review, fixed in a51c8f5). Nothing activates a window, moves the pointer or shows the viewer.

Docs: the seat_scroll and seat_key descriptions, PROTOCOL.md "Input", USAGE.md's scroll and key rows, anode help scroll / help key, a TROUBLESHOOTING entry ("A click right after a scroll lands on the wrong element"), one line in the agent skill, and a CHANGELOG bullet under Unreleased.

How it was tested

  • New quick check "scroll screen settle" (Cli/InputChecks.cs), with stand-in frames, clock and waits (no capture, no input): an animated scroll settles at 320 ms; a still page is unchanged at 500 ms; an already moving screen is changing with no wait after the input; endless motion stops at the cap and at half of short timeoutMs values; capture failures (null or throwing, before or after the input) give unknown with the input run once; cancellation mid-poll gives unknown, and cancellation while grabbing the baseline, between the baseline frames or before starting sends nothing; a resolution change counts as a change; reused frame buffers; a late first frame; a key hold that used most of the request; the key decision (pagedown, ctrl+end, pgup, home, shift+pageup, VK_NEXT, ctrl+vk24 settle; enter, a, space, down, ctrl+s don't); an input failure surfaces once.
  • Red/green: Codex implemented the first version (job 2026-10-22-anode-fixer-B-055). Review found that a quiet period or start window could end while frames are compared, giving a negative wait (WaitOne(-20) throws after the scroll was sent; -1 would wait until the request's deadline). A stand-in clock that moves while frames are compared made the new check fail on that version ("screen wait outside the poll interval: -20") and pass with the fix (settled at 350 ms, unchanged at 520 ms, as traced by hand).
  • The cancellation case was red too: on 4843ca3 the new "cancelled while grabbing the baseline" case failed (the wheel event was sent), and it passes on 9bcc07c.
  • The frame fingerprint is checked on in-memory bitmaps (no screen capture): the same picture hashes the same; a changed last pixel, and the same bytes in another shape, don't. Red with the row loop cut one row short ("a changed last pixel kept the frame's fingerprint").
  • FOCUS command scripts\build.ps1 -QuickTest -OutputDirectory artifacts\pkg-build on the final head (a51c8f5, main merged in): Release build, 0 warnings, 120/120 quick checks, check-docs.ps1 42 documents against 41 tools consistent.
  • Reviews: read-only Codex (bin/codex-review.sh) on 4843ca3 found the cancellation problem; on 9bcc07c, no actionable regressions. The GitHub Codex review raised the same cancellation point and the memory point, both fixed above. A last read-only pass on a51c8f5 is noted in the comments.

Still needs a manual check in a real seat

The real capture path was not run (this routine may not start a seat or capture the user's desktop). In a seat: open a long page in seat_browser Chrome, click blank page, seat_scroll -5, then seat_screenshot at once. Expected: the reply says settled (or changing on a page with animation) and the screenshot shows the scrolled page. Also worth noting: screenWaitMs for a few scrolls, and one seat_key pagedown.

🤖 Generated with Claude Code

skulitom and others added 2 commits October 22, 2026 23:33
…creenshot was stale (B-055)

input.scroll answered as soon as SendInput queued the wheel event, while Chrome and
other apps animate a scroll over several frames, so a screenshot taken straight after
it showed the old position and a coordinate click from it could hit another element.

ScreenSettle (Core/Capture) now watches the seat's screen around input.scroll and
input.key chords with PageUp, PageDown, Home or End: two frames before the input (a
screen already moving skips the wait), then a poll every 40 ms until 200 ms of quiet
after a change (settled), 500 ms with no change (unchanged), or a 2.5 s cap (changing),
capped at half the request's time left. The result reports screen, screenWaitMs and a
summary; a capture failure never stops or repeats the input. New quick check
"scroll screen settle" with stand-in frames and clock, including a quiet period that
ends while frames are compared (a negative wait). Tools, PROTOCOL, USAGE,
TROUBLESHOOTING, CLI help, the skill and CHANGELOG describe it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T22:38:01.871495Z 4843ca3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4843ca3885

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +71 to +72
// Keep input outside capture's catch: a real input error must still reach the caller, without a retry.
input();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip the scroll when cancellation wins the baseline poll

When seat_stop reaches the host during the new 40 ms pre-input poll, Wait observes cancellation and records an unknown result, but execution still falls through to input(). The input.scroll delegate has no cancellation check, so it can call SendInput after shutdown has begun; cancellation that occurs before the action should return or throw before sending the wheel event.

AGENTS.md reference: AGENTS.md:L63-L69

Useful? React with 👍 / 👎.

Comment thread src/Anode/Core/Capture/ScreenSettle.cs Outdated
Comment on lines +142 to +145
_bitmap = new Bitmap(bounds.Width, bounds.Height, PixelFormat.Format32bppArgb);
_graphics = Graphics.FromImage(_bitmap);
_next = new byte[checked(bounds.Width * bounds.Height * 4)];
_previous = new byte[_next.Length];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound the memory used to compare screen frames

Every scroll or scrolling-key action allocates a full-resolution bitmap plus two equally sized managed buffers. At the supported 8192×8192 display size, each is 256 MiB, so one action requires at least 768 MiB; even 4K requires about 95 MiB and repeats those large allocations on every action. High-resolution seats can therefore suffer severe GC stalls or fail capture from memory pressure, defeating the settle feature; compare bounded/downsampled frames or reusable hashes instead.

Useful? React with 👍 / 👎.

skulitom and others added 2 commits October 22, 2026 23:43
…5 review)

The two baseline frames before a scroll take about 40 ms. A request cancelled in
that time (its deadline, or a stopping seat) still sent its wheel event and
answered success, though its caller had given up. It now throws before the
input, as InputInjector.Press does for a key, so it sends nothing and answers
timed out. Quick check cases: cancelled while grabbing the baseline, between the
baseline frames, and before starting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each watched scroll kept a full-size bitmap plus two full-size frame buffers:
about 95 MB at 4K and 768 MB at the largest seat (8192x8192). Frames are now
hashed row by row from the one reused bitmap (the size and pixels, SHA-256), so
a scroll holds what a screenshot does. Quick check: the same picture hashes the
same, a changed last pixel and another shape with the same bytes don't (red
with the last row skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@skulitom
skulitom merged commit a1434af into main Oct 22, 2026
2 checks passed
@skulitom
skulitom deleted the compass/anode-fix-2026-10-22-scroll-settle branch October 22, 2026 22:53
skulitom added a commit that referenced this pull request Oct 23, 2026
…ase (#88)

* [compass] Anode 0.11.4 release record: #81, #82 and #87 join the release (C-072)

CHANGELOG: the Unreleased entries (#81 B-048, #82 B-053, #87 B-055 and
the Windows-MCP guide from #83/#85) move into 0.11.4, dated 2026-10-23.
RELEASE_NOTES: the three fixes and the guide, and the changelog anchor.
RELEASE-READINESS: scope adds #81, #82, #87 and the docs PRs (#83, #85,
#86) and notes #66 to #68; 120 quick checks and 42 documents against
41 tools; coverage limits for the three fixes and a live scroll look
for #87 before the tag; the B-053 known issue is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* [compass] 0.11.4 record: #79 in scope, #81's check described as it runs, the outside user's fix named (#57)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant