main+agent_sdk: test-blind early-stopping controls, run robustness, log_viewer rewards (sysID stack 2/8) - #97
Merged
Conversation
When True, a solved (env-accepted) training episode counts toward early stopping regardless of its reward, dropping the K*-optimality requirement that early_stop_min_reward encodes. Legitimacy is still enforced by the env's solved verdict itself.
Add the parked domino_high_friction_turn variant (turn bands, domino_turn_ratio 1.0) and opt both high-friction arms into online_learning_early_stopping_ignore_reward_bar. exp_domino now runs agent_po_predicate_invention_al on domino_high_friction (domino_turns and agent_oracle_hybrid_sim parked).
…raction lines, per-cycle test/explore rates, reward on episode chips
skip_redundant_test assumed a previous evaluation had run, but with skip_initial_test the pre-loop test is absent; early stopping on cycle 0 then ended the run with zero eval results. Gate the skip on last_test_summary so at least one evaluation always runs.
Auth/billing failures (e.g. 'organization has disabled Claude subscription access for Claude Code') surface as ordinary assistant text at 1 turn / $0.00, indistinguishable from a no-capture attempt, so run_20260721_161159 burned 10 online cycles on ~300 one-second queries (5 attempts x 2 queries x 3 solves per episode, all seeds) without the agent ever running. query_fatal_error (session_base) flags a response as fatally broken only when the agent made NO tool call and it carries a known auth/billing banner, an error result (error_max_turns excluded), or a stream error. _track_fatal_response counts consecutive fatal queries process-wide (class attribute, surviving fresh-context manager recreation; reset by any healthy query and by utils.reset_config) and raises AgentSessionFatalError at agent_sdk_max_consecutive_fatal_queries (new setting, default 3, 0 disables). Hooked into _run_streamed_query (local sandbox + plain manager) and the Docker manager's query. AgentSessionFatalError is deliberately NOT an ApproachFailure: every broad except between a query and main re-raises it - the solve restart loop (no banked-capture salvage), the sketch-query loop, the final-submission nudge, the model-free solve, both agent explorers (no random fallback), and cogman's episode loop (keep_failed_demos must not absorb it into a failed trajectory). parse_result_message now also captures is_error and the result text.
…y-one round mapping
A long-lived PyBullet world carries history that state-level resets do not clear (residual velocities the reconstruction diff skips, auxiliary joints no reset touches, contact-solver state that survives restoreState), so a test episode's physics depended on everything the run executed before it: run_20260721_205821 seed0's validated capture stalled mid-cascade in the run's env yet completes 10/10 in fresh worlds, including under +-4mm placement jitter. Adds BaseEnv.make_fresh_test_instance()/dispose() with the PyBullet implementation sharing the already-generated task lists (identical tasks, no regeneration) and skipping GUI mode and real_robot_execute; _run_testing runs each episode's execution, rendering, and evaluation in the fresh instance behind test_fresh_env_per_episode (default True). Same rationale as the fresh-env-per-rollout sysID fix in code_sim_learning.rollout_env.
This was referenced Jul 30, 2026
code_sim_learning: sysID verdicts, shared orchestrator, anchor-ablation refits (sysID stack 3/8)
#98
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second of eight PRs landing the domino-sysid branch on master in sequence (after #96). Tree is identical to the corresponding slice of that branch, rebased onto post-#96 master.
This chunk
Adds the domino_high_friction_turn experiment env plus early-stopping controls that stay blind to test outcomes, terminates runs cleanly after consecutive fatal agent-SDK queries, isolates test episodes in fresh env instances, and teaches log_viewer to surface env rewards and correct round mapping.
🤖 Generated with Claude Code