Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ src/dvidia_training/camera_action_capture.py text eol=lf
src/dvidia_training/robot_profiles.lock.json text eol=lf
src/dvidia_training/camera_action_cache.py text eol=lf
src/dvidia_training/camera_action_reference_export.py text eol=lf
src/dvidia_training/camera_action_reference_model.py text eol=lf
src/dvidia_training/camera_action_study.py text eol=lf
src/dvidia_training/piper_rgbd_tracker.py text eol=lf
src/dvidia_training/action_demonstration.py text eol=lf
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ jobs:
run: |
python -B -m unittest tests.test_camera_action_reference_export -v
python -B -m unittest discover -s benchmarks/camera-action-v02-head-preflight -p test_preflight.py -v
- name: Verify reference-model training and inference on synthetic fixtures
run: python -B -m unittest tests.test_camera_action_reference_model -v
- name: Test footage, visual training and loopback studio
run: python -m unittest tests.test_footage_data tests.test_footage_train tests.test_footage_pipeline tests.test_footage_benchmark tests.test_training_studio tests.test_doctor tests.test_coverage tests.test_episode_tape tests.test_reaction tests.test_reaction_metrics tests.test_reaction_convergence tests.test_reaction_benchmark tests.test_motor_memory tests.test_motor_metrics tests.test_motor_benchmark tests.test_observations tests.test_observation_models tests.test_motion tests.test_motion_benchmark tests.test_transition_model tests.test_transition_pilot tests.test_transition_cli tests.test_embodiment_calibration tests.test_embodiment_cli tests.test_reference_sim -v

Expand Down
21 changes: 21 additions & 0 deletions benchmarks/camera-action-reference-model-integration-v01/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 DVIDIA contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
39 changes: 39 additions & 0 deletions benchmarks/camera-action-reference-model-integration-v01/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# DVIDIA reference-model training and inference integration

Two small PiPER research candidates now train and serialize successfully from the same 96,000 training rows. The absolute-reference fit took 5.230713 s and the prior-reference delta fit took 5.097628 s on the measured shared CPU host. Both saved models reproduced all 1,000 outputs in the recorded-trajectory replay. This validates the learning/inference bridge, not task completion.

- [Absolute candidate card](models/absolute/README.md)
- [Delta candidate card](models/delta/README.md)
- [Derived-data card](data/README.md)
- [Compact integration receipt](evidence/integration-result.json)
- [Training reference identity](evidence/training-input-identity.json)
- [Initial native smoke refusal accounting](evidence/initial-smoke-accounting.json)
- [All three engineering batches](evidence/development-smoke-status.json)
- [Independent eight-attempt native audit](evidence/native-smoke-v3-audit.json)
- [Renderer warnings](evidence/renderer-warnings.json)
- [Strict-state failure diagnosis](evidence/native-smoke-diagnosis.json)

The third separately registered native batch completed eight attempts: **0/4 strict placements for each candidate**. Cabs made bilateral contact in 3/4 and a qualified lift in 1/4; Cdelta contacted in 4/4 and lifted in 0/4. Neither completed carry. Unexpected contacts occurred in 2/4 Cabs episodes and 4/4 Cdelta episodes. All four delta traces crossed the strict scorer’s finite TCP coordinate envelope of ±0.5 m; this was not a NaN or physics-warning failure. Full per-case evidence is retained, together with the sixteen earlier infrastructure refusals. No candidate was selected or qualified; the proposed 164-attempt screen and frozen v0.1 result are unchanged.

The two candidate model files are included under `models/absolute/model.json` and `models/delta/model.json`. They are usable through the pinned Python API; this new model format is not supported by the qualified-skill CLI installer. Raw training recordings and derived reference arrays are not redistributed here. Original artifacts and source commit `78ea4fbb4b2733e7738eb425eeecd7cb67a52195` are unchanged. Exact original result SHA-256: `6d44d98488ef26f949412aadc94c32174bd5c4ce879442191b9784deff5b325f`.


## Inspect a retained candidate

Use a source checkout containing this benchmark and the unchanged runtime bytes identified below. Install the package and its existing NumPy dependency with `python -m pip install -e .`, then run Python from the repository root:

```python
from dvidia_training import camera_action_reference_model as reference_model

model = reference_model.load_model(
"benchmarks/camera-action-reference-model-integration-v01/models/delta/model.json"
)
assert not model["task_qualified"]
assert not model["physical_robot_ready"]
policy = reference_model.ModelPolicy(model)
print(model["variant"], model["training"]["samples"])
```

Loading does not start a simulator, read a camera or command hardware. Calling `policy(context)` requires the recorder's exact causal context and retained float64 previous reference, beginning at the registered reset; an arbitrary image or joint vector does not satisfy that contract. Any next native experiment requires its own registration; these two candidates remain unchanged. Do not infer operational task success from successful loading or a training-trajectory replay.

Source at measurement: [`78ea4fbb4b2733e7738eb425eeecd7cb67a52195`](https://github.com/Dvidia-Inference/dvidia-training/tree/78ea4fbb4b2733e7738eb425eeecd7cb67a52195). See [the model bridge contract](../../docs/camera-action-reference-model-v01.md) and [integration notes](../../docs/camera-action-reference-model-integration-v01.md).
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
license: mit
language:
- en
tags:
- robotics
- simulation
- demonstrations
- research
---

# DVIDIA PiPER reference training data

This card describes the derived training inputs for both retained reference-model candidates. This data directory contains an identity receipt; the benchmark does not redistribute raw videos, caches or reference arrays. Both candidate model files are included separately.

The data comprises 96,000 control rows from 96 existing simulation episodes in 32 training starting-position groups: 64 authored-teacher demonstrations covering both destinations, followed by 32 previously collected mixed-control corrections. These are synchronized simulated camera/action data, not recovered actions from human video. Development and confirmation episodes are excluded from fitting.

Each parent row contains 269 causal features, the issued-teacher or separately recorded teacher-query target, group ownership and provenance. Each reference export adds two C-order float64 `(N, 7)` arrays: exact previous normal servo references and feasible target endpoints. Six dimensions use radians; the seventh is the single-finger jaw coordinate in metres. Reset references come from the recording reset; later references come from the preceding step's actual normal ramp state. The original 20 updates are applied to teacher labels. For mixed control this is counterfactual normal-reference supervision, not a statement that those labels were executed.

The parent cache was built through the original strict exporter. The reference-export construction verified sealed recording metadata and caller-pinned parent data; cache/reference loads validate derived bytes without re-reading raw RGB-D frames. This integration experiment did not revalidate raw recordings. Array, protocol, stage, source, parent dataset and manifest identities are in [training-input-identity.json](../evidence/training-input-identity.json).

Source: [DVIDIA training at the benchmark commit](https://github.com/Dvidia-Inference/dvidia-training/tree/78ea4fbb4b2733e7738eb425eeecd7cb67a52195). Training uses only training-data normalization. Current teacher actions, future observations and native object/contact truth are excluded from features. Missing visual observations remain masked, and historical estimates retain their age semantics.

These bounded synthetic demonstrations do not establish transfer to shoes, arbitrary videos, other arms or physical hardware. Model training losses and a single training-trajectory replay do not establish generalization. The original dataset ownership and failure accounting are preserved.

DVIDIA-generated derived data and accompanying documentation use [MIT](../LICENSE). Upstream simulator and robot asset licenses remain separate. No raw human footage, credentials or upstream model weights are included in this benchmark.
Loading
Loading