Conversation
Add a v4l2h264enc candidate to the H.264 encoder table, between the existing hardware encoders and the software fallbacks in auto mode, and accept v4l2 as an explicit -hwaccel method. V4L2 stateful encoders are the standard hardware encode path on ARM SoCs (Qualcomm venus/iris, Raspberry Pi, NXP i.MX, Rockchip via hantro), where neither NVENC nor VA-API is available. The element only registers when the kernel exposes a matching /dev/video* encoder node, so the existing gst-inspect probe gates selection correctly on systems without one. Encoder tuning goes through extra-controls using the standard V4L2 control names (bitrate in bps, CBR mode, I-frame period/GOP matching the existing keyframe interval, no B-frames, repeated sequence headers). Controls a driver does not expose are skipped by GStreamer with a warning rather than failing the pipeline, so the same stage works across encoder implementations. encoderResult gains an optional afterEncoder caps stage, placed between the encoder and the parser. The v4l2 row uses it to pin the H.264 level (level=4, signaling only): without a fixed level in negotiation the bcm2835 driver on Raspberry Pi fails STREAMON with "Failed enabling i/p port, ret -3". Validated on a Raspberry Pi 4 Model B (bcm2835-codec, kernel 6.12, GStreamer 1.26.2): -test mode against doubletake-test-receiver selects V4L2 hardware encoding, streams 1920x1080@30 through the hardware encoder, and the extra-controls bitrate demonstrably applies. Selection logic covered by contract tests; go test ./... passes on linux/arm64.
A V4L2 driver that does not implement VIDIOC_G_PARM leaves GStreamer with
no frame interval to probe. Rather than falling back to the open framerate
range the hardware accepts, gstv4l2object then advertises a fixed set on
the encoder's sink pad: qcom-iris on the Steam Frame reports
{29/1, 912261120/31457309}, both approximately 29.
Nothing else in the pipeline can intersect that, so the framerate caps
filter upstream of the encoder fails negotiation with not-negotiated (-4)
and capture never starts. Every rate other than 29 fails, including the
default 30, which leaves doubletake unable to encode at all on a device
whose only H.264 encoder is v4l2h264enc.
encoderResult gains an optional beforeEncoder stage, mirroring the
existing afterEncoder, and the v4l2 row uses it to place videorate
immediately before the encoder. videorate adapts to whatever rate the
driver insists on and is a passthrough when the requested rate already
negotiates, so drivers that honour it are unaffected.
Verified on a Steam Frame (Snapdragon 8 Gen 3, qcom-iris, kernel 6.16,
GStreamer 1.24.2): the default -fps 30 now negotiates and streams, where
it previously failed before capture started.
The Wayland capture path always asks xdg-desktop-portal for a node, which requires an org.freedesktop.portal.ScreenCast implementation. A session can publish its compositor output as an ordinary PipeWire node and still offer no such backend: SteamOS on the Steam Frame runs gamescope, which exports a Video/Source node named "gamescope", but ships only kde.portal and kwallet.portal. There, X11 capture is not an alternative either, because gamescope's rootless Xwayland hands back an empty root window and SteamVR renders straight to the headset display via DRM, so ximagesrc yields nothing but black frames. -pipewire-node names such a node directly and skips the portal. A numeric value selects a node ID through pipewiresrc's deprecated path property; any other value selects a name through target-object, which survives the renumbering that makes a saved ID unusable between sessions. Because the node identifies its source on its own, this path deliberately requires neither WAYLAND_DISPLAY nor DISPLAY, so it works over a plain ssh session. The portal and direct-node cases share one pipeline builder rather than duplicating it: the start path now tolerates the absent fd and D-Bus connection that only a portal session carries. Verified on a Steam Frame streaming the live headset view to an Apple TV (AppleTV14,1, tvOS 26.6) through the v4l2h264enc hardware encoder.
Some sources publish video outside any display server: capture cards, loopback devices and virtual cameras. -v4l2-device streams one of them straight through the existing encode path. The case that motivated it is the Steam Frame, where neither existing capture path reaches what the wearer sees. gamescope's PipeWire node carries each app's flat window, composited per virtual connector, so the VR shell, Steam UI, dashboard and overlays never appear in it. X11 is no better: gamescope's rootless Xwayland hands back an empty root window and SteamVR renders straight to the headset display via DRM, so ximagesrc yields only black frames. SteamVR itself exports the composited headset view through v4l2cam onto a v4l2loopback node, which this flag can read. Like -pipewire-node, a device node identifies its source on its own, so this path needs neither WAYLAND_DISPLAY nor DISPLAY and works over a plain ssh session. A device that cannot be opened is reported before the session starts, because a missing node would otherwise surface as a stalled or blank stream rather than an error. Naming more than one capture source is now rejected outright instead of silently preferring one. The X11, V4L2 and test paths share a new startGstCaptureProcess helper rather than repeating the process-launch tail a fourth time. The portal path keeps its own copy: its inherited fd and D-Bus lifetime are worth migrating separately. Verified on a Steam Frame streaming the live SteamVR headset view to an Apple TV (AppleTV14,1, tvOS 26.6) through the v4l2h264enc hardware encoder, and the unreadable-device and conflicting-source paths were exercised on the device.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything needed to make doubletake work on a Steam Frame (Snapdragon 8 Gen 3, SteamOS,
qcom-iris). All three commits were developed and verified on the device.This branch sits on top of
v4l2-h264-encoder(upstream PR omarroth#52), which this fork'smaindoes not have yet — so the first commit here is that PR's, included as its base.d443d05Adapt V4L2 capture rate for drivers without VIDIOC_G_PARMqcom-irisdoes not implementVIDIOC_G_PARM, so GStreamer has no frame interval to probe and advertises a fixed set on the encoder's sink pad —{29/1, 912261120/31457309}, both ≈29 — instead of the open range it reports on the src pad. Nothing else in the pipeline can intersect that, so the upstream framerate caps filter fails negotiation withnot-negotiated (-4)and capture never starts. Every rate other than 29 fails, including the default 30.That matters because the Frame ships GStreamer plugins-base and good only: there is no
x264enc,openh264enc,vah264encornvh264encon the device, sov4l2h264encis the only H.264 encoder available.encoderResultgains abeforeEncoderstage mirroring the existingafterEncoder, and the v4l2 row uses it to placevideorateimmediately before the encoder. It is a passthrough when the requested rate already negotiates, so drivers that honour it are unaffected.c74bbd4Add-pipewire-nodefor capture without a screencast portalThe Wayland path always asks xdg-desktop-portal for a node. A session can publish its compositor output as an ordinary PipeWire node and still offer no
ScreenCastbackend: SteamOS runs gamescope, which exports aVideo/Sourcenode namedgamescope, but ships onlykde.portalandkwallet.portal.A numeric value selects a node ID; anything else selects a name via
target-object, which survives the renumbering that makes a saved ID unusable between sessions. Confirmed in practice — the node moved from id 39 to 151 across a Steam client restart.a231babAdd-v4l2-deviceto capture a V4L2 node directlyThe gamescope node carries each app's flat window, composited per virtual connector (
--virtual-connector-strategy PerAppId), so the VR shell, Steam UI, dashboard and overlays never appear in it. X11 is no better: gamescope's rootless Xwayland returns an empty root window and SteamVR renders straight to the headset display via DRM, soximagesrcyields only black frames.SteamVR itself exports the composited headset view —
v4l2camacquires the headset view overlay and writes it to a v4l2loopback node — which this flag reads directly.Also useful for capture cards and other V4L2 sources. A device that cannot be opened fails before the session starts rather than becoming a silently blank stream, and naming more than one capture source is rejected instead of silently preferring one.
The X11, V4L2 and test paths now share a
startGstCaptureProcesshelper rather than repeating the process-launch tail a fourth time. The portal path keeps its own copy deliberately; its inherited fd and D-Bus lifetime are worth migrating separately.Verification
gofmt,go vetandgo test ./...clean on linux/arm64; new contract tests for the source stages, the framerate ordering, the unreadable-device failure and the conflicting-source rejection.v4l2h264encon/dev/video23.Notes for anyone else on a Frame
gst-plugin-pipewireis not installed by default. It can be fetched unprivileged (pacman -Spprints the URL) and extracted under$GST_PLUGIN_PATHin$HOME, which needs nosudo— useful, since thesteamosuser has no password set — and survives OS updates.extra-controlsonly partially applies onqcom-iris: requesting 2 and 8 Mbps on fixed content produced roughly 19.5 and 26.9 Mbps against ~40.8 uncontrolled. Monotonic, but the target is not held and CBR is not honoured. Worth keeping in mind before trusting the bitrate setting on this driver.