Skip to content

Learn and locally evaluate a bounded PiPER camera-to-action skill - #32

Open
Tarzelf wants to merge 10 commits into
feat/piper-rgbd-tracker-v01from
feat/piper-camera-action-v01
Open

Tarzelf wants to merge 10 commits into
feat/piper-rgbd-tracker-v01from
feat/piper-camera-action-v01

Conversation

@Tarzelf

@Tarzelf Tarzelf commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

This adds the complete local synchronized-camera/action learning loop for a simulated PiPER placing one known rigid block at two destinations. All 452 registered attempts are accounted for: frozen unseen-start placements are 5/64 learned, 9/64 matched blind, 31/64 fixed and 64/64 authored teacher. The learned candidate fails qualification. The deliverable is reproducible training, installation and honest evaluation, with no arbitrary-video, arbitrary-arm or hardware claim.

  • Capture 25 Hz RGB-D, 50 Hz measurements/commands and 1 ms native checks; bind immutable perception and teacher-query sidecars to their recordings. Preserve missing observations, causal history and executed-versus-suggested action authority.
  • Fit the CPU two-layer policy with training-only normalization, then one registered 50% mixed-control correction round. Exclude native object/contact truth, teacher phase/current command and future observations from learner inputs. Compare full/blind candidates on development data before freezing confirmation.
  • Expose dvidia-train robot prepare/collect/train/evaluate/inspect/package, the shared Training Studio /robot flow and simulation-only dvidia-arm installation. Stage gates, worker exclusion, immutable outputs and failed-attempt accounting remain explicit.
  • Independently reconstruct native grasp, lift, carry, retention and terminal outcomes. Publication cards now render gate rows deterministically without changing gate values, runtime or model selection. Preserve MIT licensing and upstream asset notices; robot assets are provisioned separately.

The passing full-study consistency audit verifies 2,384,532 files, 440 full-interval recordings and 12 intentional input-contract refusals. Both paired superiority gates and the 58/64 placement gate fail; all 16 negative controls behave as registered. Contact-phase diagnostics return 0/749 available object estimates. The selected learner's inference p95 is 0.447 ms, but full-boundary p95 is 36.432 ms on the shared Apple M5 host; these are offline measurements, not a physical 50 Hz deadline. Its fit/save took 10.95 s; shared verified export took 471.30 s.

Validation: 105 camera/action tests, 31 Studio tests and 21 publication tests passed locally. Fresh offline installation of delivery d89b24c passes 12 CLI and four renderer checks; all 19 registered runtime files and six package resources retain their prior identities. Desktop browser verification and one installed native execution are explicitly attributed to unchanged delivery ab9082e: Studio loads the failed benchmark, respects the external worker lock and installs a research candidate without console errors. The single out-of-study execution fails the task scorer, exits 1 and is not retried or counted in confirmation. No narrow-screen live-Studio claim is made. All 36 exact-head CI checks passed for d89b24c.

The registered runtime remains frozen. Delivery source adds only deterministic publication-card ordering and its regression test after ab9082e. The export-cache/action-reference follow-up is separate in PR #33.

The audited research report and recorded viewer are published. The camera-to-action CLI preview is now a public, non-latest prerelease at d89b24c. Anonymous downloads verified all five existing assets (8,348,845 bytes): wheel, full-study audit, JSON/Markdown summaries and checksums. The release receipt binds the exact tag and hashes. The cache preview remains a separate verified release.

The full 36.28 GB frozen-study archive is not public at its Hugging Face destination. Its reviewed publisher reached the four-hour transfer limit on 9 November 2026; a subsequent anonymous check found only the original .gitattributes placeholder. All 475 local frozen files remain preserved, with no automatic retry. The five GitHub assets exclude the raw archive and original model bundle. This status does not affect the separately published reference-action research candidates from PR #36; those are a distinct development experiment, also unqualified.

@Tarzelf
Tarzelf marked this pull request as ready for review November 9, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant