fix(bastion): pin openclaw to a version that accepts the harness config - #23
Merged
pradeepvrd merged 1 commit intoSep 9, 2026
Conversation
The harness writes `memory.search.enabled` into every per-run openclaw.json so a second run of a task cannot recall the first one's transcript and score on recall rather than on the cluster. That key does not exist before openclaw 2026.8.1, and older oc rejects the entire config instead of ignoring the unknown key. The bastion pin was left at 2026.6.10, so the code and the VM the module builds disagreed. The failure is quiet, which is the reason for the long comment. oc exits 1 after about fifteen seconds having done nothing, but the record still reports status "success" -- the honest signals are `validated: false` and an empty trajectory. Everything downstream then behaves correctly and looks like data: detection scans zero entries and reports no_data, every objective comes back "not observed", correctness is withheld, and the row scores null with no OutcomeScore. Read quickly, that is a run that happened and scored badly rather than an agent that never started. Terraform only installs oc when the VM is created, so this does not reach an existing bastion. The known-issues row carries the in-place upgrade.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One-line pin bump plus a known-issues row. Not a new feature — this unbreaks every openclaw run on
integration.The break
75e42d3(2026-09-02) made the harness writememory.search.enabledinto every per-runopenclaw.json, so runs cannot recall each other's transcripts. That key does not exist before openclaw 2026.8.1. The bastion pin intf/modules/bastion/startup.shis 2026.6.10, unchanged sinceffddbe3.Older
ocrejects the whole config rather than ignoring an unknown key:The agent then exits 1 having done nothing.
Why it is easy to miss
The run does not report as an error. It reports as a success:
status: "success",validated: false, empty trajectory, 0 tools, ~15scheating_report: no_datanot observed_finalize_outcome_scorereturns before emittingOutcomeScoreThe record scores null and drops out of the leaderboard. Nothing in the summary says the agent never ran. I lost two matrix runs to this before finding the
ocstderr.The fix
Bump the pin to
2026.8.2and say in a comment that the two move together:Operator action required — Terraform alone will not fix a running bastion
startup.shonly installsocat VM creation, and it short-circuits on/var/lib/bench-bastion-ready. Existing bastions need:Verified on
bench-bastion: the config that producedmemory: Invalid inputnow reportsConfig valid, and a subsequent run reached the model call.The known-issues row documents both the signature and the in-place fix, since anyone hitting this will search for the symptom rather than the cause.