[Real-world active learning] PR 7: human-gated task rebuild on the stock loop

[amburger66]

PR 7 of the real-world active learning plan. Depends on #95.

What this adds

Between episodes a human rearranges the dominoes, and that episode's task is rebuilt from what the cameras then see. That is what makes the stock online-learning loop work on the bench: every episode faces a physically different scene instead of replanning against one frozen capture. No new entrypoint.

Why the rebuild happens at task-request time

env_task = env.get_train_tasks()[i]     # (1)
cogman.reset(env_task)                  # (2)
run_episode_and_get_observations(...)   # (3) calls env.reset(...)

The task must be rebuilt at (1), for a different reason on each of the two paths:

• Evaluation solves at (2). _solve_task (main.py:774) calls cogman.reset(env_task) with no override policy, so _reset_policy runs approach.solve(task) — before env.reset is ever called. A human reset performed at (3) would plan against a bench that no longer exists.
• Exploration does not solve at (2): the online loop sets an override policy first (main.py:551), so _reset_policy takes that branch. The task still matters, because it is what env.reset initializes the simulated twin from — a stale one starts every episode from the captured scene rather than the arranged bench.

BaseEnv caches tasks, so PyBulletEnv overrides get_train_tasks/get_test_tasks to ask the executor first through a third ActionExecutor hook, tasks_for. _reset_pending is set by after_reset, not at episode end: there is no end-of-episode hook, and the loop always resets before it steps, so marking it on reset gives exactly one prompt per episode.

Deviation from the plan

The plan puts this on RealWorldEnv.get_train_tasks. PR 6 replaced that wrapper with an injected executor, so it lands as a third port hook instead. The ordering argument, the _reset_pending bookkeeping and the "not in the domain env" rule are unchanged — PyBulletDominoRealEnv still contributes only task_from_observation, with no notion of a human.

real_robot_human_reset (default True)

Off keeps the captured scene, which is what replay_plan needs: rebuilding from a live look would rename and re-place the very objects a recorded plan refers to. It is refused at construction if the robot has no perception, rather than raising after the human has been asked to stand by.

replay_plan

Rewired onto CogMan's episode loop with an override policy — the same shape main.py uses online — instead of the deprecated utils.run_policy. With an override set CogMan never asks the approach to solve, so the plan being replayed is the plan that executes; that is also why the approach and monitor are the cheap ones (oracle + trivial), keeping an LLM out of a debug replay. _parse_plan and the plan .txt format are untouched.

Budget

exp_domino_real.yaml drops to 2 cycles × 1 seed from common.yaml's 10 × 3 — every episode costs a physical reset by a human. Verified by loading the config, not by reading it.

A bug found on the way

reset_env with no joints resolves to None, which fails ResetArmReply validation in dry mode and raises outright on hardware unless the RealRobot was constructed with home_joints — which make_real_robot doesn't do. The executor now captures the twin's home configuration at attach time (the simulated arm is still at home then, verified) and passes it explicitly. Mutation M5 covers it.

Tests

• test_domino_real_online.py — 6 tests, stub robot, never skips: the rebuild happens before the task is handed over, one prompt per episode, a new look replaces the cached task, the test split too, human-reset-off keeps the capture, and an unattached env never prompts.
• test_domino_real_offline_e2e.py — a genuine RealRobot(dry=True) with mock perception and an auto-confirming reset, the real executor, real CogMan, real run_episode_and_get_observations. Asserts one prompt per episode, one look per option boundary (6 looks over 2 episodes = 2 resets + 4 boundaries), and well-formed gripper commands. It reproduces main.py:551-565's call sequence rather than invoking main(), which would add arg parsing and result IO without testing more. Skips without the submodule.

Seven mutations checked, all caught: no rebuild; next reset never owed; no reset at startup; human_reset ignored; reset_env sent no home joints; each split hook removed.

Gates: tests/envs + tests/pybullet_helpers + test_main = 292 passed / 15 skipped without the submodule, 302 passed / 0 skipped with it; mypy clean over 709 files; pylint 88/88; isort/yapf/docformatter verified by exit code; submodule untouched.

Still not bench-validated

Nothing here has run on hardware. The human wait, the arm homing before a person reaches in, and per-option shipping are exercised only against a dry robot and mock perception. Worth a first session with real_robot_dry on, to hear the prompts without motion.

[yichao-liang]

LGTM! Congrats on completing the plan!

[amburger66]

Cross-link, per CONTRIBUTING: the babyrobot side of this migration finishes in BasisResearch/BabyRobotPredicator#55 (plan PR 8), which deletes the old JSON bridge_server.py now that this PR moved the last caller off it.