Repository navigation
rustdesk: decode video on the gpu and drop the cpu path - #15
Open
ElCruncharino wants to merge 3 commits into
Open
ElCruncharino wants to merge 3 commits into
ElCruncharino wants to merge 3 commits into
Conversation
added 3 commits
October 23, 2026 15:49
backends that get h264 from their protocol can hand the frames to mediacodec and get nv12 planes back. the decoder lives in common so every backend uses the same one. frames carry their colour matrix and range, and the jni callback is optional, so backends without video don't notice.
a gl layer behind the desktop view draws each video region from its nv12 planes, so the colour conversion happens on the gpu. damage over a video region hands it back to the cpu. what the video had left of that region is repainted from its last frame, so the mirror doesn't flash the old picture. screenshots and the session thumbnail paint video over the backend's picture, which is stale under video.
every rustdesk frame is a whole desktop, so each one goes to the gpu as a single region. the rgb conversion, the changed-row diff and the framebuffer are gone. region reads and thumbnails report no picture now, since the video layer covers them.
ElCruncharino
marked this pull request as draft
October 23, 2026 19:57
ElCruncharino
force-pushed
the
gpu-video-rustdesk
branch
from
October 23, 2026 20:05
cc57ed7 to
e0c9140
Compare
ElCruncharino
changed the title
ElCruncharino
marked this pull request as ready for review
October 23, 2026 20:07
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every RustDesk frame is a whole desktop, so each one goes to the GPU as a
single region. The RGB conversion, the changed-row diff, and the
framebuffer that held the converted pixels are gone.
Region reads and thumbnails now report no picture, since the video layer
covers them. The session paints video over whatever the mirror reads.
Builds and installs as the RustDesk plugin. Not tested against a live
RustDesk host since the cleanup, so treat it as unverified.
Stacked on #12.