Repository navigation
fix(openclaw): fail fast on a host/image oc version skew - #40
Merged
isadominguez314 merged 2 commits intoSep 15, 2026
Merged
isadominguez314 merged 2 commits into
isadominguez314 merged 2 commits into
Conversation
On a sandboxed run the container's oc writes the session store and the
host's oc reads it back post-run. oc refuses a store written by a
different version ('written by version 2026.9.1-beta.1, but this
command is running 2026.8.2'), which surfaced live as an empty
trajectory on a record stamped status success -- a full cluster
spin-up spent producing nothing gradable, discovered only at scoring.
Probe both versions at the top of _execute (cached, mirroring the
claude_code version probe) and error out before the agent turn when
they differ. Equality rather than an ordering: prerelease suffixes
make 'newer' a guess, and the two binaries are meant to be pinned
together anyway. An inconclusive probe (wrapper script, docker
unavailable) never blocks the run; unsandboxed runs skip the probe
entirely -- there is no second binary to compare against.
…version-skew # Conflicts: # devops_bench/agents/cli/openclaw/agent.py # tests/unit/agents/test_agents_cli_openclaw.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Guards against the failure mode from the first #32 validation run: host
oc2026.8.2 could not read the session store the image'soc2026.9.1-beta.1 had written (…written by version 2026.9.1-beta.1, but this command is running 2026.8.2), so the post-run trajectory export failed and the run publishedtrajectory: [],errors: 1,status: "success"— a full cluster spin-up spent producing nothing gradable, discovered only at scoring time.What changed
Two cached version probes (mirroring
claude_code/agent.py's_claude_versionpattern and philosophy) plus a guard at the top of_execute:_host_oc_version(oc_bin)— parsesoc --versionfrom the host binary._image_oc_version(image)— parses it from the sandbox image via a short-liveddocker run --rm --entrypoint oc <image> --version(generous timeout for a cold pull; cached per image, so once per matrix run).AgentResult.errored(...)naming both versions and the fix, before the agent turn launches — failing in seconds instead of after the run.Deliberate choices:
-beta.1) make "host is newer, probably fine" a guess, and the two binaries are meant to be pinned together anyway (the pin-unification follow-up tracks collapsing the bastion's three version pins into one).config.targetmay be a wrapper without a--versionsurface, docker may be unavailable to the probing process — refusing to run on a probe that merely failed would be worse than the risk it guards. Same rule the claude_code probe follows.Validation
uv run pytest tests/unit→ 1858 passed (6 new: prerelease parsing, inconclusive-probe behavior, skew message content, match/inconclusive silence, the fail-fast path asserting the agent turn never launches, and the unsandboxed skip); ruff check + format clean.🤖 Generated with Claude Code