Repository navigation
app: draw decoded video on the gpu in the session - #12
Open
ElCruncharino wants to merge 5 commits into
Open
ElCruncharino wants to merge 5 commits into
ElCruncharino wants to merge 5 commits into
Conversation
This was referenced Oct 23, 2026
ElCruncharino
marked this pull request as draft
October 23, 2026 19:57
ElCruncharino
force-pushed
the
gpu-video-core
branch
from
October 23, 2026 20:05
7e621eb to
9df6b2a
Compare
ElCruncharino
changed the title
ElCruncharino
marked this pull request as ready for review
October 23, 2026 20:07
added 5 commits
October 25, 2026 18:35
A GL layer behind the desktop view draws each video region from its NV12 planes, so the colour conversion happens on the GPU. Backend.Listener gets videoFrame for the backends that hand it planes. Damage over a video region hands it back to the CPU, and what the video had left of that region is repainted from its last frame so the mirror does not flash the old picture. Screenshots and the session thumbnail paint video over the backend's picture, which is stale under video.
A backend can now hand a decoded picture to the window as NV12 instead of composing it into the framebuffer. The mailbox queues it as an event in order with the damage around it, and replaces an unread picture for the same rectangle. It is off unless the window asks, so the Linux client is unchanged. The phone asks, and delivers each picture to the session as Listener.videoFrame.
The library converts every AVC420 picture to RGBA and composites it. The patch lets a handler that draws video itself take the decoder's planes instead, for AVC420 only. Nothing uses it yet.
On the phone the graphics pipeline's AVC420 pictures are cut out as NV12 and passed to the window, instead of being converted to RGBA on the CPU and composited. Only for surfaces mapped 1:1 to the output, so a scaled surface still goes through the compositor. AVC444 is unchanged. Needs patches/ironrdp/0002.
A picture alone ends no frame, so nothing redrew the view to cut its tiles out of the new region.
ElCruncharino
force-pushed
the
gpu-video-core
branch
from
October 26, 2026 15:06
9df6b2a to
f08d62c
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Video that a backend decodes to NV12 planes is drawn by a GL layer behind the desktop view, so the phone's GPU does the colour conversion instead of the CPU.
The layer handles one region at a time. Damage over a video region hands that region back to the CPU, and the desktop view repaints it.
The backend video callback is optional, so backends without video are unaffected. IronRDP's EGFX AVC420 is the first backend to use it. FreeRDP, RustDesk and SPICE still draw on the CPU.
Rust tests pass. Not built for Android yet and not run on a device.