Repository navigation
feat: introduce libeverlive-core and unify the GC573/GC570D control plane - #63
Conversation
- Make card matching data-driven via fallback_match rather than hard-coded IDs - Distinguish unreadable sysfs IDs from legitimate zero values during discovery - Copy registered op vtables and freeze the registry after the first lookup - Validate hdr-ev/volume/led input; reject NaN/Inf and malformed JSON payloads - Detect the running engine via its control socket instead of pgrep - Execute systemctl/service commands directly with argv, without a shell - Launch the service script with fork/exec and report real spawn failures - Cache the request length, key-match "error", reject odd-width YUY2, fix docs - Add SONAME/versioned shared library plus install and uninstall targets - Add everlive_ctl protocol helpers and unit tests; extend registry tests - Translate remaining German build output to English
…erlive-core
Part 1/5 of the modularization plan (see
.opencode/plan/everlive-modularization/01-libeverlive-core.md).
Consolidate the shared control-plane pieces in the core library so the
GC573 streamer, the GC570D daemon, and the unit tests no longer carry
private copies or cross-directory .c includes.
- Move third_party/jsmn.{c,h} from opengc573-userd into libeverlive-core.
- Add everlive_json.{c,h}: typed getters, raw/id helpers, top-level key
detection, canonical JSON escaping, and canonical-integer id formatting.
All functions are stateless and re-entrant.
- Add everlive_ipc.{c,h}: deterministic socket resolution
(override -> EVERLIVE_SOCKET -> EVERLIVE_CONTROL_SOCKET -> XDG_RUNTIME_DIR
-> /run/user/<uid> -> /tmp/everlive-<uid>) plus a prepare variant that
creates the runtime directory for the server side.
- everlive_ctl_proto: reuse the core escaper and detect errors with the
jsmn top-level key check instead of a substring scan. Responses get a
512-token budget; a real status.get reply needs ~135 and the old 128
limit returned JSMN_ERROR_NOMEM.
- everlive_ctl: resolve the live everlive.sock instead of the stale
everlive-experimental.sock, which never matched a running engine.
- everlive_card: add everlive_find_card_by_slug() and reuse it for ops
registration.
- gc573_user_ipc: drop the duplicated parser/escaping/ID code and call
the everlive_json_* API.
- Core Makefile: include the new objects and jsmn in the static/shared
archives, link everlive-ctl against the archive, add dependency
generation, install/uninstall the new headers, and template
everlive.service with PREFIX.
- tests: drop the duplicate -D_GNU_SOURCE and duplicate asan target,
link the core archive instead of recompiling its sources, and update
the escaping expectations to the canonical \n/\t short forms.
Behaviour changes:
- JSON escaping now uses \n, \r, and \t short forms for those control
characters; other control bytes remain \uXXXX.
- everlive_json_format_id rejects non-canonical integers (e.g. "0012")
instead of emitting invalid JSON.
Verified: core builds clean with gcc and clang; all 6 test suites pass
(134 assertions) including ASan/UBSan; GC573 binaries build; everlive-ctl
resolves the unified socket path and honours the env overrides.
Part 2/5 of the modularization plan (see .opencode/plan/everlive-modularization/02-opengc573-userd.md). Mirror the GC570D layout, stop compiling every translation unit with SDL/FFmpeg/PipeWire flags, and replace the private PCI scan and socket path logic with the core helpers. - Move all GC573 sources and headers into tools/opengc573-userd/src/. Binaries stay in the module root so play.sh, install.sh, uninstall.sh and the systemd units keep their paths. - Move test_framegen.c to tests/ (the build rule follows it). - Rewrite the Makefile around incremental objects and three flag groups: hardware/base, capture pipeline (FFmpeg + PipeWire), and the SDL2 renderer. probe, everlive_userd and diag now build without SDL2, FFmpeg or PipeWire installed. - Link every binary against ../libeverlive-core/libeverlive_core.a. - Replace the hand-rolled /sys/bus/pci/devices scan with everlive_detect_installed_cards() and accept only "gc573" descriptors. This fixes a latent bug where a GC570D (1461:0054 with subsystem 0x5700) could be claimed by the GC573 engine. - Resolve the control socket with everlive_ipc_prepare_socket_path(): --ipc-socket, EVERLIVE_SOCKET, the legacy EVERLIVE_CONTROL_SOCKET, then $XDG_RUNTIME_DIR -> /run/user/<uid> -> /tmp/everlive-<uid>. - Generate header dependencies with -MMD -MP and clean src/*.o, src/*.d, the binaries and the framegen selftest. - tests: add the test_framegen target and include it in `make test`; keep the forwarding `test`/`selftest` targets. - Update the module .gitignore and README for the src/ layout and the core-library dependency. Verified: GCC and Clang builds are warning-free; the four binaries build; probe/everlive_userd/diag build with PKG_CONFIG disabled; all 7 test suites pass (164 assertions); `make -q` reports an up-to-date tree.
Part 3/5 of the modularization plan (see
.opencode/plan/everlive-modularization/03-opengc570d-userd.md).
Make the GC570D engine a first-class EverLive citizen: discover the card
through libeverlive-core and serve the same newline-delimited JSON-RPC 2.0
protocol on the same everlive.sock as the GC573 streamer, while keeping the
legacy plain-text gc570d-ctl socket.
Discovery
- Replace the private /sys/bus/pci/devices scan with
everlive_detect_installed_cards() and accept only "gc570d" descriptors
(subsystem 1461:5700). A missing/unreadable subsystem file fails safe
instead of guessing.
New control plane
- src/gc570d-jsonrpc.{c,h}: persistent AF_UNIX clients, SO_PEERCRED UID
check, flock-protected everlive.sock, and line-buffered dispatch.
- Methods: hello, ping, status.get, led.set, led.mode.set,
audio.volume.set, audio.mute.set, audio.channels.set (stereo only),
osd.set, osd.position.set, aspect.set, scale.set, input.set, pip.set,
hotplug, edid.set.
- status.get emits the same schema as the GC573 engine (signal, video,
audio, v4l2, osd, aspect, scale, framegen, led, clients) so the Python
hub consumes it unchanged. FPS is derived from the monotonic frame seq.
- GC573-only methods (mode.set, v4l2.set, hdr.*, framegen.set,
toast.show) return -32601 explicitly so clients can probe capabilities.
- No new thread: the daemon poll includes the listener and all clients and
drains them from the existing loop, keeping LED/I2C and debugfs actions
serialized with the watchdog and re-init paths.
Supporting APIs
- gc570d_led_set_rgb()/gc570d_led_set_effect(): public entry points; the
sysfs effect attribute now delegates to the same implementation.
- gc570d_user_diag_action(): extracted from the plain-text handler and
reused for hotplug/edid.set.
- gc570d_everlive_* getters for the frontend state (port, osd, aspect,
scale, volume, mute, v4l2 paths, headless/running).
- gc570d_user_pipeline_stats(): frame/audio/IRQ snapshot callback.
Build and shutdown
- Compile the daemon from incremental objects, add -I../libeverlive-core
and link libeverlive_core.a; clean src/*.o and src/*.d. Ignore *.d.
- Re-arm the JSON-RPC device pointer around hardware re-init and tear the
listener down before the legacy control socket.
Viewer
- Attach the hub to a running DUO engine (the duo_mode guard is gone) and
prefer IPC for DUO input, HPD, EDID policy, and LED effects, with
gc570d-ctl preserved as the fallback.
Verified: GCC and Clang builds are warning-free; a socket harness exercises
hello, ping, status.get, mutations, unsupported methods, and persistent
connections; all 7 test suites pass (164 assertions); the viewer compiles.
Part 4/5 of the modularization plan (see
.opencode/plan/everlive-modularization/04-tests.md).
One tests/ tree now covers the shared library, the GC570D engine, and the
GC573 framegen module, and everything is reachable through one command.
- Rename tests/test_ipc_json.c to tests/test_json_rpc.c now that the JSON
scanners live in libeverlive-core. Update its banner and keep every
assertion as the regression net for the moved code.
- Rewrite tests/Makefile:
* core tests (card registry, frame conversion, ctl protocol, JSON-RPC)
link libeverlive_core.a instead of recompiling its sources
* GC570D EDID/config tests still compile the engine modules directly
with -DGC570D_USERSPACE because they provide their own kernel-shim
stubs and must not link the archive
* framegen compiles only gc573_user_framegen.c with -pthread -lm
* add -MMD -MP dependency generation for single-source rules (multi-
source link rules omit it because Clang rejects -MMD together with -o)
* group the output by component and fail on the first failing suite
* replace the duplicated/flag-appending asan target with a recipe that
cleans, rebuilds with ASan/UBSan, and runs the suites
- run_tests.sh: support --asan and describe the full suite instead of the
GC570D-only banner.
- Retire the in-engine gc573_framegen_selftest: the GC573 Makefile now
forwards `test`/`selftest` to tests/, adds a `framegen-selftest`
shortcut, and no longer builds or cleans a private binary.
- Ignore all test binaries (including test_json_rpc) and drop the stale
gc573_framegen_selftest entry.
Verified: GCC, Clang, and ASan/UBSan runs pass; 7 suites / 164 assertions;
`./tests/run_tests.sh` and `--asan` both work; the legacy per-engine
selftest targets still forward successfully.
Part 5/5 of the modularization plan (see .opencode/plan/everlive-modularization/05-build-install-scripts.md). One entry point now builds, tests, installs, and cleans every module, and the installers own the shared library instead of leaving it in the source tree. Build - New root Makefile: `all` builds libeverlive-core first, then both engines; `test`, `asan`, `clean`, `install`, and `uninstall` fan out to the module Makefiles with the jobserver and PREFIX/DESTDIR propagated. - build.sh becomes a thin wrapper around the root make and keeps the optional legacy renderer block. Install / uninstall - install.sh builds through the root Makefile, installs the core library, everlive-ctl, everlive-service, public headers, and the PREFIX-templated everlive.service unit, and copies vfio_pci_bind.sh where the service script expects it. - uninstall.sh removes the core artifacts, stale versioned libraries, everlive-ctl/everlive-service, the systemd unit, vfio_pci_bind.sh, and leftover runtime sockets (everlive.sock and .lock). Launchers - play.sh prefers `everlive-ctl detect` with the sysfs loop as fallback, rebuilds the core and the selected engine using real target checks (`make -q <binary>`) instead of the always-dirty phony default, and keeps the gc573 default when no card is found. - everlive-service.sh shares the same detection path and now finds the GC570D bind helper in libexec as installed by the engine Makefile. CI and docs - CI uses the root Makefile for GCC and Clang, runs the unified test suite and sanitizer target, exercises the framegen forwarding target, verifies everlive-service.sh syntax, and cleans at the end. - README documents the from-source build/test commands and the shared core; CHANGELOG gains an Unreleased section for the modularization work. Verified: root build/test/asan pass; staged `make install` and `make uninstall` with DESTDIR round-trip exactly the expected files; the everlive.service ExecStart honours PREFIX; all shell scripts pass `bash -n`; the service script detected and launched the GC570D engine with the JSON-RPC socket bound.
…y, and input hardening - Add install/uninstall targets to tools/opengc573-userd/Makefile and wire them into root Makefile - Install GC573 engine binaries to BINDIR and apply cap_ipc_lock in install.sh - Search local repo, /usr/local/bin, and PATH for engine binaries in everlive-service.sh - Restore default max brightness when switching from LED off to an active effect on GC570D - Guard everlive_yuy2_to_rgb_scaled() against odd src_w - Reject non-finite double values (NaN/Infinity) in everlive_json_get_double() - Derive target UID from SUDO_USER in uninstall.sh to safely clean user runtime socket - Add unit test coverage for odd src_w and non-finite JSON doubles
|
Huge thanks @PolygonPrincess for this fantastic architectural overhaul! Unifying We took this branch for a spin on real GC573 hardware:
I've pushed a commit (
Feel free to undraft whenever you're ready, and we will merge it straight into |
|
@Everlite Thanks for addressing the open items and providing testing on your end for GC573! I think before I undraft this, do we want to include documentation for future contributors on how to implement other capture cards in EverLive by creating a set of standards and procedures to follow? Not sure if that would be considered in-scope for this PR or worthy of its own! |
|
I think having a dedicated contributor guide (e.g. That said, since PR #63 is already quite large (+5.6k lines) and all checks and hardware smoke tests are now complete, I'd suggest keeping this PR focused on the architecture and control-plane unification. We can merge #63 now to land the foundation into If you're happy with that, feel free to mark this ready for review and we'll merge! |
|
@Everlite Sounds good to me! |
Summary
This branch turns
libeverlive-coreinto the shared foundation for both capture engines and gives the GC570D the same control plane as the GC573.It removes the duplicated JSON-RPC scanner, socket-path logic, and PCI discovery code, moves the GC573 sources into a
src/layout, adds a unifiedeverlive.sockendpoint to the DUO daemon, consolidates the unit tests, and adds a top-level build/install flow that shipseverlive-ctland the core library system-wide.The existing
everlive.sockwire format is unchanged (proto: 1, newline-delimited JSON-RPC 2.0); GC573 clients keep working, and the GC570D now answers the same protocol alongside its legacygc570d-ctlsocket.Motivation
jsmn, JSON escaping, request validation, and socket-path resolution existed only inside the GC573 engine; tests reached them by#include-ing a.cfile.gc573_vfio.caccepted1461:0054without checking the subsystem — a GC570D could be claimed by the GC573 engine.everlive-ctllooked foreverlive-experimental.sock, which no engine ever bound.gc570d-ctl.everlive-ctl, or the universaleverlive.serviceunit.tests/Makefilerecompiled core sources and carried merge artifacts (duplicate-D_GNU_SOURCE, duplicateasantarget).What changed
Shared core (
tools/libeverlive-core)everlive_json.{c,h}: stateless,jsmn-backed typed getters, raw/id helpers, top-level key detection, JSON escaping, and ID normalisation.everlive_ipc.{c,h}: deterministic socket resolutionoverride → $EVERLIVE_SOCKET → $EVERLIVE_CONTROL_SOCKET → $XDG_RUNTIME_DIR → /run/user/<uid> → /tmp/everlive-<uid>,plus a
preparevariant that creates the runtime directory for the server side.everlive_ctl_proto.cnow reuses the shared escaper and detects errors with a real top-level key check instead of astrstrheuristic.everlive_ctl.cconnects to the liveeverlive.sock; the staleeverlive-experimental.sockpath is gone.everlive_card: addedeverlive_find_card_by_slug().jsmnmoved into the core and is compiled into both the static and shared archives.0.2.0,SONAME libeverlive_core.so.0; headers install with the library.GC573 (
tools/opengc573-userd)src/; binaries stay at the module root soplay.sh, the installers, and systemd units keep their paths.probe,everlive_userd, anddiagno longer require SDL2/FFmpeg/PipeWire headers.libeverlive_core.a.everlive_detect_installed_cards()and accepts onlygc573descriptors. This fixes the DUO misidentification:1461:0054+ subsystem0x5700is rejected, and an unreadable subsystem file fails safe.--ipc-socket,EVERLIVE_SOCKET, and the runtime fallbacks all agree with the hub andeverlive-ctl.GC570D (
tools/opengc570d-userd)gc570ddescriptors.src/gc570d-jsonrpc.{c,h}: persistent AF_UNIX clients,SO_PEERCREDUID check,flock-protectedeverlive.sock, line-buffered dispatch, no extra thread.hello,ping,status.get,led.set,led.mode.set,audio.volume.set,audio.mute.set,audio.channels.set,osd.set,osd.position.set,aspect.set,scale.set,input.set,pip.set,hotplug,edid.set.status.getemits the same schema as the GC573 engine (signal, video, audio, v4l2, osd, aspect, scale, framegen, led, clients), so the Python hub consumes it unchanged.mode.set,v4l2.set,hdr.*,framegen.set,toast.show) return-32601explicitly so clients can probe capabilities.gc570d_led_set_rgb(),gc570d_led_set_effect()(the sysfseffectattribute delegates to the same code),gc570d_user_diag_action(), frontend getters, andgc570d_user_pipeline_stats().drv->remove()and re-arms it after re-probe.gc570d-ctlsocket is unchanged.Viewer
duo_modeguard is gone.gc570d-ctl.Tests
tests/test_ipc_json.c→tests/test_json_rpc.c; it exercises the core scanners directly.libeverlive_core.a; GC570D tests still compile their engine modules with-DGC570D_USERSPACEand their own kernel-shim stubs.tests/and is part ofmake test.make -C tests asancleans, rebuilds with ASan/UBSan, and runs.tests/run_tests.shsupports--asan.Build, install, launch, CI
Makefile:all,test,check,asan,clean,install,uninstall.build.shis a thin wrapper around the root build.install.shinstalls the core library,everlive-ctl,everlive-service, headers, thePREFIX-templatedeverlive.serviceunit, and the GC573 bind script.uninstall.shremoves the core artifacts, stale versioned libraries, the unit, and leftover runtime sockets.play.shpreferseverlive-ctl detect(sysfs fallback), rebuilds using real target checks, and preserves the GC573 default when no card is found.everlive-service.shshares the same detection path and finds the GC570D bind helper inlibexec.Compatibility and behavior changes
proto: 1, newline-delimited JSON-RPC 2.0 overeverlive.sock). The DUO is a new endpoint, not a format change.1461:0054is no longer claimed; cards with an unreadable subsystem ID are reported as unsupported instead of being guessed.\n,\r,\tshort forms (other bytes< 0x20remain\uXXXX). Semantically identical JSON; textual expectations in tests were updated."0012"are echoed as strings instead of invalid bare numbers.--ipc-socket: the parent directory is now created by the shared helper."off".gc570d-ctlremains available.Verification
make -j"$(nproc)"(GCC)make -j"$(nproc)"withCC=clangmake -C tests clean testmake -C tests asan./tests/run_tests.sh/--asanmake install DESTDIR=… PREFIX=/usreverlive-ctl, unit, headers, GC570D artefactsmake uninstall DESTDIR=… PREFIX=/usrbash -non all scripts,py_compileon the hub/run/user/<uid>/everlive.sockbinds, legacy socket still servesRecommended before merge: a GC573 hardware smoke test of
play.sh,everlive-ctl runtime status, and the background service.Reviewer guide
Commit order and focus:
e9f4aec— core JSON/IPC extraction. Start here; everything else consumes these APIs.1f7b450— GC573src/layout, flag split, core discovery. Large diff mostly renames; the behavioural change is the subsystem-aware probe.f4d674f— GC570D JSON-RPC server, LED/diag support APIs, daemon poll integration, viewer attach. Highest-risk commit; reviewgc570d-jsonrpc.c, re-init/shutdown ordering, and the persistent-client buffering.b17494d— unified tests. Check that GC570D tests still avoid linking the core archive.594ce97— root Makefile, installers, launchers, CI. CheckPREFIX/DESTDIRhandling and service paths.43of the78files in the integration delta are pure renames; the logical surface is roughly 35 files.Open items identified during self-review
Before merge, consider addressing:
install.shinstallseverlive.service, but the installed service script resolves the GC573 engine at/usr/local/bin/gc573_user_streamer, whichinstall.shnever installs. On a GC573 system the unit would fail (andRestart=on-failurewould loop). Install the GC573 engine (andsetcapthe installed copy), or resolve the repo path at install time.led.mode.set {"mode":"off"}, the GC570D zeroesled_cdev.brightness; subsequentbreathing/colorfuleffects compute zero components and stay dark until a static RGBled.set. Restore a non-zero brightness when applying a non-off effect.Lower-priority hardening (not blocking): reject non-finite doubles in
everlive_json_get_double()(the GC570D volume handler casts NaN toint), guardeverlive_yuy2_to_rgb_scaled()against oddsrc_w, and derive the runtime UID fromSUDO_USERinuninstall.sh.Checklist
DESTDIR