[Real-world active learning] PR 6: real-robot execution as an injected executor - #95
Merged
Conversation
All real-robot behavior moves out of PyBulletDominoRealEnv into a wrapper, leaving that env pure simulation: no robot, no action buffer, no real/dry mode. What is left is a simulated environment whose task comes from a captured scene, which is what it should have been. RealWorldEnv wraps a PyBulletEnv and drives it on hardware. It ships each option's joint trajectory when the option ends, looks at the bench, converts what it saw into a State, and writes that into the twin. The library's observation type never escapes the wrapper: the agent is handed the twin's observation, so CogMan and the perceiver never learn a robot exists. Writing perception into the twin is not a step layered on top of perceiving, it is how perception reaches the agent at all: env.step advances the twin's own physics client and reads the observation back out of it, so without the write a correction is overwritten on the very next action. Knocked-over dominoes are now read back as knocked over. real_geometry gains domino_env_euler, which decomposes a perceived pose into the env's (yaw, roll) under the env<->real body-axis permutation, so roll -- the feature Toppled is defined on -- carries the topple instead of being hardcoded to zero. Verified against real captures: standing dominoes read roll ~ 0.00 deg with yaw matching the previous standing-only helper exactly. The wrap happens at main.setup_environment, not inside create_new_env: the planner builds its own envs through that factory (the option model's private simulator, the shared skill simulator) and must never be handed a robot. The closed loop is not on by default. babyrobot currently offers only file and mock perception, so real_robot_ship_whole_episode stays True and hardware runs remain open-loop exactly as before; the settings comment records what to flip once live perception lands.
…ault babyrobot PR 3 (session-scoped live ZED perception) has landed on main, so the submodule moves from 3565015 to 7603bc8 and the closed loop becomes the default rather than something waiting on a dependency. - real_robot_perception defaults to "zed": the live session, which RealRobot opens on construction and closes on close(). The factory passes our domino_real_table_z through rather than leaving babyrobot's own default in place -- perception and the base -> world transplant have to agree about where the table is, and they are configured separately. - real_robot_ship_whole_episode defaults to False, so trajectories ship per option and the bench is re-perceived between them. Safe now that RealWorldEnv owns the chunking and RealRobot drops a gripper command that repeats the session's state. replay_plan.py pins itself to whole-episode shipping. It replays an EXACT plan, so re-syncing the twin mid-replay would let the option policies see states the recorded plan was never chosen against; it also keeps that debug tool usable with the cameras down.
RealWorldEnv wrapped the env and claimed to be a BaseEnv. That shape carried a defect: it was a BaseEnv but NOT a PyBulletEnv, and the episode loop branches on exactly that (cogman.py:225 and :272, agent_sdk/rendering.py:35) to decide whether to pass render_obs -- so wrapping silently disabled rendering under real_robot_execute. Nothing in the tree does isinstance(x, BaseEnv), so the inheritance bought nothing while failing the PyBulletEnv check cost something real. The ownership inverts. PyBulletEnv declares an ActionExecutor port and holds an optional collaborator, default None; RealRobotExecutor implements it. The env stays the env, so every isinstance check keeps working, and the sentinel get_name, the instance-dict shadow of that classmethod, __getattr__, thirteen delegating stubs and the two-view split all go away with the wrapper. Planning safety becomes structural. PyBulletEnv.simulate ended in `return self.step(action)`, so a hook in step would ship to the robot every time the option model forward-simulated a candidate. step's body is extracted into _step_once; simulate calls _step_once and step calls _step_once plus the hook. The planner therefore cannot reach an executor by construction, rather than by anyone remembering not to wrap its env. The executor is three pieces instead of one class doing three jobs: OptionBoundaryBuffer (pure), TwinCorrector (perception -> the twin), and RealRobotExecutor, which owns the robot and composes them. Its settings are constructor arguments read from CFG once by the factory, so tests configure by construction instead of mutating global config. Whole-episode shipping is removed. It existed because the gripper split had to span an episode; RealRobot's session-wide gripper dedup removed the reason. The flag, flush_real_execution, and replay_plan's special-casing of both go with it -- replay_plan stays deterministic by turning the option-boundary look off, which is the setting that actually governs mid-replay twin re-sync. An episode cut short leaves a partial option buffered; that is dropped at the next reset with a warning rather than shipped, because half a skill is not worth executing on the arm.
amburger66
changed the title
Comment edits, plus the two formatting failures they surfaced: the tests/envs/test_real_robot_execution.py import left unsorted by an earlier edit (which is what failed isort on the last push), and the domino env's module docstring summary reflowed by docformatter. Two settings comments rewrapped to stay inside the 80-column limit; wording unchanged.
amburger66
marked this pull request as ready for review
July 29, 2026 15:46
emilybunna
self-requested a review
July 29, 2026 17:16
amburger66
enabled auto-merge (squash)
July 30, 2026 12:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR 6 of the real-world active learning plan (see its addendum for the design change below). Depends on #94 and on babyrobot #51, which this repins to.
What changes
PyBulletDominoRealEnvloses all real-robot behavior and goes back to being a plain simulated env. Real execution becomes a collaborator the env holds:PyBulletEnvdeclares anActionExecutorport (defaultNone), andRealRobotExecutorimplements it — buffer each option's actions, ship them when the option ends, look at the bench, write what was seen into the twin.The twin sync is the load-bearing part:
env.stepadvances the twin's own physics client and reads the observation back out of it, so between two looks the twin is the only thing producing the agent's world state. Perceive without syncing and the correction is overwritten on the very next action.The library's observation type never escapes the executor — the agent is handed the twin's
State, soCogMannever learns a robot exists.Why an executor rather than a wrapper
The first version of this PR wrapped the env in a
RealWorldEnvthat claimed to be aBaseEnv. That shape had a real defect: it was aBaseEnvbut not aPyBulletEnv, and the episode loop branches on exactly that (cogman.py:225/:272,agent_sdk/rendering.py:35) to decide whether to passrender_obs— so wrapping silently disabled rendering underreal_robot_execute. Nothing checksisinstance(x, BaseEnv)anywhere, so the inheritance bought nothing.Inverting the ownership fixes that and deletes the scaffolding it required: a sentinel
get_name, an instance-dict shadow of that classmethod,__getattr__, thirteen delegating stubs, and a two-view split of one object.Planner safety became structural.
PyBulletEnv.simulateended inreturn self.step(action), so a hook instepwould ship to the robot every time the option model forward-simulated a candidate.step's body is now_step_once;simulatecalls_step_once,stepcalls_step_onceplus the hook. The planner cannot reach an executor by construction rather than by anyone remembering not to wrap its env.Toppled dominoes
domino_env_eulerdecomposes a perceived pose into the env's(yaw, roll), soroll— the featureToppledis defined on — carries the topple instead of being pinned to zero. Verified against real captures: standing dominoes readroll ≈ 0.00°withyawmatching the previous standing-only helper exactly; a second scene has genuine leaners at 23° and −44°.This exposed a latent bug in #94's fixture:
_IDENTITY_QUATis physically a domino on its face, which went unnoticed because the old code hardcodedroll = 0. Fixed, which is why that test file's diff is large.Closed loop on by default
Repinned to babyrobot
7603bc8(live ZED perception).real_robot_perceptiondefaults to"zed", and the factory passes ourdomino_real_table_zthrough rather than babyrobot's own default (−0.041 vs −0.045) — perception and the base→world transplant must agree where the table is. Pinned by a test.Whole-episode shipping is removed: it existed only because the gripper split had to span an episode, and
RealRobot's session-wide dedup removed that reason. The flag,flush_real_execution, andreplay_plan's special-casing all go with it;replay_planstays deterministic by turning the option-boundary look off instead.Worth a reviewer's eye
RealRobotand file perception.split_actions_by_optionwas deleted. Mutation testing showed the branch using it was dead: the buffer holds exactly one option at every ship, so it could only ever return one chunk, and the test passed with the code removed.PyBulletEnvgains ~12 lines for the port and hook. The plan said "no core change"; this is the honest trade against it.Tests
Two files, 24 tests, plus the domino env's. The executor and conformance tests use stubs and must not skip (asserted); one integration test drives a genuine dry
RealRobotso the realStepRequest/Segmentconstruction is exercised somewhere.Ten mutations checked, all caught — including the three guarantees this design adds:
simulatenever reaching the executor, and the reset/step hooks firing.Gates:
tests/envs+tests/pybullet_helpers= 280 passed / 14 skipped without the submodule, 294 passed / 0 skipped with it (same collection total); mypy clean over 703 files; pylint 86/86; isort, yapf, docformatter all verified by exit code.