Bug: Black regions return without a resize: Windows recreates the RDPGFX surface at the same size, and 005 (#118) does not fire

[pletch]

Component: guacd / RDP (RDPGFX), browser client H.264 path
Related: #118 (closed) — same Windows behaviour, different trigger
Host: Windows 11 25H2 (OS Build 26200.9445), RDPGFX with H.264 (AVC444)
Tested on: a fork of rustguac with H.264 passthrough and additional diagnostics; the analysis of 005 below is against main

Summary

The black "ghost" regions from #118 also appear mid-session with no resize at all. Windows occasionally deletes and recreates its RDPGFX surface at the same size. The new surface starts empty, so its first picture decodes black with a full-screen region rect. Windows then repaints only the regions it thinks have changed, so the rest of the desktop stays black until something repaints it. A resize fixes it.

patches/005-rdp-resize-dirty-flush.patch doesn't cover this, for two reasons:

1. It never runs. 005 hooks guacd's desktop-resize callback. A same-size recreation doesn't go through it: there's no ResetGraphics and no Server resized display to …, only DeleteSurface / CreateSurface.
2. Even if it ran, it would re-send black. 005 works by flushing guacd's own framebuffer, which after a resize still holds the right pixels. Here, Windows' black picture has already been decoded into that framebuffer (full-screen rect), so a flush would send black again. The last correct copy of the desktop is the one the browser is showing. With H.264 passthrough, in the case where guacd skips the decode, its framebuffer never holds pixels for H.264 regions at all, so there is nothing correct for 005 to re-send, after a resize or otherwise.

Evidence

The episodes below were caught with per-operation RDPGFX tracing in guacd, plus browser-side probes of every painted keyframe. Times are local and from a single session each.

Episode A

15:03:44.939  guacd  RDPGFX DeleteSurface: surface=0
15:03:44.985  client keyframe, view 0, numRegionRects=0   -> decoded 100% "decoder green" (zeroed YUV)
15:03:45.075  guacd  RDPGFX DeleteSurface: surface=0
15:03:45.075  guacd  RDPGFX CreateSurface: surface=0, 2992x2000      <- surface was already 2992x2000
15:03:45.119  client keyframe, view 0, 1 rect covering 100%          -> decoded 100% black
15:03:49.023  client display 77% black; decoder healthy, later deltas land correctly

Episode B

12:35:40.072  guacd  RDPGFX DeleteSurface: surface=0
12:35:40.194  guacd  RDPGFX DeleteSurface: surface=0
12:35:40.194  guacd  RDPGFX CreateSurface: surface=0, 2992x2000      <- same size, no resize
~12:35:40.2   client keyframe painted
12:35:42.813  client display 46% black; black persisted >120s, cleared by the host redrawing the affected regions

On the same day, every other CreateSurface either came with a resize (connect-time fit, entering or leaving fullscreen) or was the first surface of a new connection. Windows repaints fully after those, and none was followed by black. The two same-size recreations were the only ones followed by black, and both were.

Why the obvious fix won't work

As #118 documented, asking Windows to repaint doesn't help on RDPGFX. A patch that sent SuppressOutput off/on and RefreshRect after every resize fired and the black stayed, because Windows doesn't re-send its surface cache for either. 005's own comment says the same thing for RefreshRect.

What worked (H.264 passthrough path)

Because the browser still shows the correct desktop when the black picture arrives, the client can keep it:

• Decode but don't paint a keyframe that decodes ≥98% black while the framebuffer has kept the same size for 5s. It still has to be decoded, because later pictures reference it. Windows' partial repaint then lands on the old picture, which is what Windows assumes the client is showing. At connect and after resizes it's painted as before, since Windows repaints fully then. A localStorage override turns the behaviour off.
• Don't paint a picture whose region list is empty. Episode A's green keyframe had numRegionRects = 0. In MS-RDPEGFX the region rects are the areas that changed, and FreeRDP's avc420_decompress copies and invalidates only those, so zero rects means "change nothing". A client that paints every decoded picture whole, or that reads a count of zero as "whole picture", shows a full-screen green flash instead.

Likely impact on main

• H.264 passthrough: affected. The h264 handler doesn't read region rects and paints every decoded picture whole, so both the zero-rect green picture and the black surface picture reach the screen.
• guacd decoding (no passthrough): probably affected, by the same mechanism: the black picture is decoded into guacd's framebuffer and sent. Not verified. Without H.264, Windows might send the new surface differently, and I have no trace of that configuration. A fix there would have to live in guacd: withhold a full-surface black update after a same-size CreateSurface, while its framebuffer still holds the previous pixels.

What triggers the recreation

Unknown. It happens on the Windows side, with no client action. Candidates are a secure-desktop switch (UAC, lock screen, Ctrl+Alt+Del), an application entering exclusive fullscreen, or a display driver reset. In both episodes above, the session was idle or in ordinary use.

Happy to share the diagnostic patches or test a candidate fix.