agent_sdk+code_sim_learning: physics-margin capture gate on honest sysID posterior widths (sysID stack 4/8) - #99
Merged
Conversation
…pages An explore transcript now gets a video player like test pages do, mapping its round (= 0-based learning cycle) to the make_interaction_videos recording ...__cycle<N>.mp4, which has no __task part and concatenates all explore episodes of the cycle. Also rewords two docstrings that were failing CI's docformatter check.
…rollouts Fresh-env repeats were bit-identical replays: every stochastic step (BiRRT sampling, IK restarts) reads the constant CFG.seed at call time, so trials=N and the 3/3 capture-validation gate measured ONE effective sample. run_20260722_204632 (domino_high_friction_turn seed2) captured a plan verified 10/10 trials + 3/3 validation that missed the single real episode; every trials-mode result in that run was 0/N or N/N, never mixed, and a seed-jittered replay flipped the '10/10' plan to ~50/50 (one trial reproducing the real episode's exact failure state). decorrelated_rollout_seed(rollout_idx) offsets CFG.seed per repeat (index 0 keeps the base seed so the canonical rollout stays reproducible; entered after fresh-env creation so task-cache keys are unaffected), wired into the sim.run trials loop and the capture validation repeats. Trials output now states the seed variation.
…env disposal PyBulletDominoComposedEnv lazily creates a second full PyBullet client (the counterfactual-push-probe world) on first certificate evaluation, and every disposal path disconnected only the main client - so each fresh validation env / solved=True trial that scored the evaluator leaked a complete physics world (~150MB, one live client per create-probe-dispose cycle). Six parallel runs accumulating this froze a 16GB machine (2026-07-23). Fix at three layers: a dispose() override on the domino env that releases the probe world first (a dead main client in crash recovery must not strand it) then the main client; dispose_env delegates to env.dispose() when available instead of raw-disconnecting one client; the two remaining direct-disconnect sites in the approach (learn-phase fit env, _recreate_base_env) now go through dispose_env. main.py's per-episode fresh_env.dispose() picks up the override automatically. Verified: create-probe-dispose cycles now end at 0 connected clients (was 1 leaked per cycle); disposal is safe when no probe was created.
…sID posterior widths
run_20260723_091108 seed1: the captured plan validated 8/8 (sim.run
trials + 3/3 fresh-env decorrelated-seed rollouts) at the fitted
lateral_friction 0.5319 and failed the single real episode
deterministically at true 0.5 - replay shows the design's success band
starts between 0.52 and 0.5319, so the 6% fit bias alone flipped the
outcome at every planner seed. Execution-validation repeats all run AT
the fitted values and cannot see zero margin to parameter error; and
the identifiability report printed posterior_std=0 ("identified",
contraction 0.00), so every uncertainty consumer believed friction was
known exactly.
Honest widths (always on):
- grid_seed._resolved_interval: the flat set widened to fit-space
midpoints toward the nearest REJECTED evaluations. Kills the
bisection-collapse zero: refine midpoints lower the best SSE, the
relative flat tolerance shrinks, the flat set collapses to one point,
and a zero-width flat_interval claimed certainty the sweep's finite
evaluations cannot support.
- identifiability_report prefers resolved_interval and floors all
single-sample widths at code_sim_learning_rollout_min_posterior_width
(fit-space, default 0.1 ~ +-10% for log params) - the free-running
replay objective's model bias (measured 1-40% off truth on clean
data) is invisible to any local landscape statistic. 0.1/0.75 prior
is contraction 0.13, still "identified": the floor reports honest
width, it does not veto deployment.
Physics-margin capture gate (flag agent_plan_validation_physics_margin,
default False; the treatment arm enables it):
- physics_sigma_points turns the applied params + report widths into
two full override dicts (all deployed params at -1 sigma, then +1;
multiplicative for log scale, box-clipped). Only applies_fitted
verdicts are perturbed - anchored/NOT-identified params carry
prior-scale widths the fit was never asked to resolve.
- The approach stores the points after each joint rollout fit, clears
them on every _apply_identified_physical_params so they cannot
outlive their fit, and exposes ctx.physics_margin_provider;
_fresh_validation_env_scope gains a physical_overrides kwarg applied
to the fresh env on top of the identified params.
- evaluate_option_plan runs the margin rollouts after execution
validation passes, on fresh envs at the BASE planner seed (failure
attributable to physics alone), and refuses with "PARAM-SENSITIVE
(plan NOT captured)" + per-point outcomes (new
CaptureDecision.PARAM_SENSITIVE_NO_CAPTURE; best-effort submissions
still capture with an honest note). At the run's numbers the -1
sigma point is 0.5319*exp(-0.1)=0.481 < the 0.52 flip point, so this
capture would have been refused in-session.
…mino runs it Identical to agent_po_predicate_invention_al plus agent_plan_validation_physics_margin: True (and the posterior-width floor stated explicitly). The base arm keeps byte-identical behavior (the gate flag defaults False), so comparing the two isolates what the physics-margin capture gate adds. exp_domino.yaml now runs the margin arm in place of the base arm.
yichao-liang
force-pushed
the
sysid-04-margin-gate
branch
from
July 30, 2026 09:00
387814b to
5a6b6b3
Compare
This was referenced Jul 30, 2026
yichao-liang
added a commit
that referenced
this pull request
Jul 31, 2026
… pipeline figure (#107) * docs/sysid: redraw the pipeline figure around the landed honesty fixes The pipeline diagram was still telling the pre-#99 story: red badges on four failure modes with the remedies listed as "planned". All six fixes have since landed on master (PRs #99-#103), so the badges are now green and each one names the run that motivated it, with the legend stating what it replaced. Content changes to the diagram: - New DATA-lane box for declaration evidence (sim.residuals with open-loop rollout plus per-param box sweep), and a CONSUMERS-lane box for the combined-substrate certificate probe. - Stage bodies updated to what the code does now: Huber-capped SSE with summary terms, dropped fits feeding hull candidates, the 32-point sweep running over the disagreement hull, INCONSISTENT swept rather than disarming the gate, dropped params reverting to registry values. Layout and output changes: - Box y-centers are computed in _layout() from per-lane stack heights instead of being hand-placed, so adding a box no longer risks silent overlap with its neighbour. - draw() takes an output path and an include_legend flag, and the script now also emits sysid_pipeline_compact.png for decks that carry the caption themselves. The landscape figure gets real matplotlib legends in place of the end-of-line annotations, which collided with the curves once the y-range widened, and the "true 0.5" marker moves to the bottom of the axes. * docs/slides: collect the reveal.js decks into one directory The three decks had accumulated in three different places (docs/, docs/envs/, docs/sysid/), which made them hard to find and left build_deck.py writing its output next to unrelated figure sources. They now all live under docs/slides/, with shared figures in docs/slides/assets/. Path consequences of the move: - build_deck.py writes to docs/slides/ instead of docs/. - The sysID deck refers to its two figures as ../sysid/*.png rather than as siblings, so the figures stay next to the scripts that generate them. docs/sysid/README.md records that coupling. The two hand-written decks are also refreshed while being moved: - The agent planning deck is rebuilt around run_20260729_232733 (seed 2) in place of run_20260723_110212, including the new run's own scene renders under assets/agent_run_20260729/, its at-a-glance table, and its captured 2-blue relay plan. "Tool surface" is renamed to "tool API" throughout. - The sysID deck is updated from "branch domino-sysid, fixes planned" to the state on master after PRs #96-#103: green badges instead of red, a new slide on why declaring a physical param is a decision and why teacher-forced residuals cannot inform it, and the block parameter family. The task-generation deck is generated output and moves unchanged. * docs/slides: weekly sync deck for 2026-07-30 and its mosaic assets A deck covering the 10/10 al_margin result on domino_high_friction_turn, the five pipeline stages with the changes that landed this week marked inline, and the measured evidence for the robust fit objective, plus the heavy-dominoes, fan and glue work coming next. Assets are 3x3 episode mosaics rather than the individual episode videos: before-learning and after-learning grids for al_margin, and a 2x2 success/failure grid for heavy dominoes. make_al_margin_mosaics.py regenerates them from a run's per-episode mp4s with ffmpeg. It emits mp4 by default because a full-resolution GIF of a mosaic this size measures 193 MB, against roughly 2 MB for the mp4; --gif produces a downscaled GIF for contexts that cannot play video.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourth of eight PRs landing the domino-sysid branch on master in sequence (after #96, #97, #98). Tree is identical to the corresponding slice of that branch rebased onto current master.
This chunk
The physics-margin capture gate: validated captures must also survive replays at sigma-point physics drawn from the sysID posterior, so knife-edge plans that only work at the point estimate stop being banked. Plus the al_margin treatment arm that runs it, decorrelated motion-planner seeds so repeated validation rollouts are independent samples, and a PyBullet world-leak fix that froze 16GB machines at 6 parallel runs.
🤖 Generated with Claude Code