refactor(config): make the agent config harness-neutral - #144
Merged
Merged
Conversation
Codex reads only AGENTS.md and has no import mechanism, so the full text moves there; CLAUDE.md becomes a one-line @AGENTS.md import, which Claude Code expands to the same content. Carries over the layer-dependency diagram that the draft had rewritten as prose, and replaces the draft's Tool Bridge section — which described a bridge that was never built. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude Code auto-loads sieval/*/CLAUDE.md and .claude/rules/*.md; opencode and Codex do neither, so they need an explicit index. Generating it from the rules' own paths: frontmatter keeps the copy from going stale, and check_agent_rule_map fails CI when it does. Also asserts the 32 KiB budget Codex silently truncates past. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Codex has no project-level commands and reads skills only from .agents/skills/, so that becomes the source of truth, with symlinks for Claude Code (.claude/skills/) and opencode (.opencode/commands/ — its skills are model-invoked only, so the command link is what restores the slash trigger and argument passing). Retires 5 unused commands: the three archived ones plus the weekly and biweekly summaries. .claude/commands/ keeps its .gitignore entry: sieval-pr-review.md.bak remains on disk there and still needs hiding, so the directory is not actually empty. .gitignore needed two more re-inclusions than the .agents/skills/ block alone: .claude/skills/* had no negations, which would have left the Claude Code symlinks themselves untracked (verified in a scratch repo - git status stayed silent on them even after `git add -A`), and opencode's own sieval-pr-review.md command-symlink pointed at the gitignored personal skill and had no rule of its own, so it would have shipped as a dangling symlink. Both are fixed the same way: negate the two team-skill names, leave the personal one covered by the trailing `*`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Three harnesses need the same four checks after an edit. Inlining them per tool would duplicate behavior, and drift there is silent — a one-character difference in a path pattern means a file class stops being checked with no error anywhere. Each harness now dispatches to scripts/post_edit_checks.py. Also adds .opencode/node_modules/ to the root .gitignore: the existing .opencode/.gitignore excludes itself (its own last line is `.gitignore`), so it is never committed and gives a fresh clone no protection. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The `.claude/skills/` negations must stay slash-free because they name symlinks; a trailing-slash negation matches only real directories and would silently untrack both. The neighbouring `.agents/skills/` block does carry slashes, so the two look inconsistent and invite a cleanup that breaks tracking with no error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…an gaps Follow-up to the harness-neutral config work, from testing all three harnesses against their real binaries rather than their docs. Remove `.codex/config.toml`. Asked Codex 0.144.4 what it actually loaded via its own `hooks/list` RPC: the file registered zero hooks, with empty `warnings` and `errors` — silent on every channel. Three independent causes, each sufficient: `command` sat on the matcher table, but `ConfiguredHookMatcherGroup` only carries `matcher` and `hooks`, so the handler needs a nested `[[hooks.PostToolUse.hooks]]`; `matcher = "shell"` matches no tool name (shell is `Bash`, edits are `apply_patch`), and a wrong matcher still registers, so it would fail silently a second time; and Codex delivers the payload as JSON on stdin while `post_edit_checks.py` reads `argv`, so even a corrected file would run the script with no path. A registered hook also comes back `trustStatus: "untrusted"` — project trust does not cover it. Writing the adapter is the right fix, but no Codex credential exists here, so its stdin payload shape cannot be observed and the result could not be verified before shipping. A config that silently does nothing is worse than none: it reads as wired. `AGENTS.md` now records why the wiring is absent and how to run the checks by hand under Codex. Make the `.opencode/commands/` ignore pattern-based, matching the two blocks above it. It listed one personal skill by filename, so the *next* one shipped: its symlink points at a gitignored target, and a public clone would get a dangling link. Verified a fresh personal skill is now ignored across all three surfaces while both tracked symlinks stay at mode 120000 — the negations stay slash-free for the reason the neighbouring comment gives. Anchor the two guideline entries in `sanitize.sh`. The allowlist filters git grep output lines, so an unanchored `AGENTS.md` exempted every line anywhere that merely named the file: `BAD = "/home/u/x" # see AGENTS.md` passed the hardcoded-path scan. Now caught, with the guideline files themselves still exempt. `CLAUDE.md` had the same flaw and is fixed with it. Also document what the harnesses really do with layer rules, measured on a synthetic tree: Codex never reads a `CLAUDE.md` below the root, not even with the working directory inside that folder (renaming it `AGENTS.md` loads it immediately), and opencode only walks upward. The 16 layer and scoped rules are therefore automatic under Claude Code alone; elsewhere the generated map is a pointer, not an injection. Validation: 418 tests in tests/unit/scripts, full preflight green (25 checks), sanitize clean, ruff check + format clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Trim the comments and docstrings added across this branch, keeping the reason each one records — every block here documents a trap that cost a debugging session — while cutting restatement and hedging. Two were also stale after the Codex hook wiring came out: the `post_edit_checks.py` docstring still named three harnesses, and the opencode plugin comment pointed at a Codex that no longer shares it. The plugin comment now says why `file.edited` is the right event — opencode emits it from inside the write/edit/patch tools, so it tracks agent edits rather than any change on disk. Validation: 418 tests in tests/unit/scripts, full preflight green (26 checks, no FAIL/WARN), sanitize clean, ruff check + format clean. Both behavioural fixes re-verified after the trim: a fresh personal skill is still ignored across all three surfaces with both tracked symlinks at mode 120000, and the self-exempting `# see AGENTS.md` probe line is still caught. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The frontmatter parser returned [] for anything that was not a block list, so `paths: ["a/**"]` — valid YAML, and the form someone reaching for brevity would write — silently dropped that rule from the map and rendered it as `_(always)_`. `check_agent_rule_map` could not catch it: the check re-parses the same frontmatter the same wrong way, so both sides agree and it passes. A generator whose whole premise is that an omission cannot happen silently had one failure mode where it does. Raise instead, naming the file. An absent `paths:` key still means "applies everywhere" — that is a real choice, and it keeps working; only a key that is present and unreadable is an error. Verified by reverse mutation: with the guard deleted, all three new parametrised cases fail with DID NOT RAISE. Left alone: the layer glob stays one level (`sieval/*/CLAUDE.md`). Missing a nested layer file is a conservative failure, the six layer directories are a fixed convention, and `**` would sweep in vendored `sieval/community/<vendor>/CLAUDE.md` files. Validation: 422 tests in tests/unit/scripts, full preflight green (26 checks), ruff check + format clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type
Summary
Makes the repo's agent configuration harness-neutral, so opencode gets the
same instructions, skills, and post-edit checks that Claude Code already
had — without duplicating any of it.
AGENTS.mdbecomes the single source of truth for projectinstructions (it was gitignored; now tracked). Root
CLAUDE.mdshrinks to a one-line
@AGENTS.mdimport. Per-layersieval/*/CLAUDE.mdfiles keep their names — that is Claude Code'son-demand load mechanism, and renaming them would break it.
AGENTS.md:scripts/sync_agent_rules.pyderives 16 rows from the 6layer files plus the
paths:front matter of the 10.claude/rules/*.mdfiles. Nothing is hand-copied, so a changed globcannot silently drift —
check_agent_rule_map(new, 15th preflightcheck) fails if the generator was not re-run.
.agents/skills/<name>/SKILL.md, surfaced toClaude Code and opencode via symlinks (all four tracked at mode
120000).
.claude/settings.jsonmove toscripts/post_edit_checks.py, sharedby two callers: Claude Code's hook and
.opencode/plugins/post-edit.ts(listens on
file.edited, zero dependencies). Both pass the editedpath as
argv[1].No Codex wiring (changed during review)
This branch originally shipped a
.codex/config.toml. Review tested itagainst a real Codex 0.144.4 and it registered zero hooks, with empty
warningsanderrors— silent on every channel. Asked via Codex's ownhooks/listRPC:Three independent causes, each sufficient on its own:
commandsat on the matcher table, butConfiguredHookMatcherGroupcarries only
matcherandhooks— the handler needs a nested[[hooks.PostToolUse.hooks]]withtype = "command".matcher = "shell"matches no tool name (shell isBash, edits areapply_patch). A wrong matcher still registers, so this would havefailed silently a second time even after fixing (1).
post_edit_checks.pyreads
argv. A corrected file would still run the script with no path.A registered hook also comes back
trustStatus: "untrusted"— projecttrust does not cover it.
Writing the stdin adapter is the right fix, but no Codex credential
exists in this environment (
codex exec→ 401), so its payload shapecannot be observed and the result could not be verified before shipping.
A config that silently does nothing is worse than none: it reads as
wired. The file is removed;
AGENTS.mdrecords why, and Codex users runpython scripts/post_edit_checks.py <path>by hand.Two gaps found while testing (fixed here)
.opencode/commands/was ignored by literal filename, so the nextpersonal skill would ship: its symlink points at a gitignored target,
and a public clone would get a dangling link. Now pattern-based like
the two blocks above it.
sanitize.sh's guideline allowlist was unanchored. It filters git grepoutput lines, so
AGENTS.mdexempted every line anywhere that merelynamed the file —
BAD = "/home/u/x" # see AGENTS.mdpassed thehardcoded-path scan. Anchored to the
path:lineno:prefix;CLAUDE.mdhad the same flaw and is fixed with it.
Test Plan
Automated
ruff check && ruff format --check)ty check)tests/unit/scriptscheck_agent_rule_mapscripts/sanitize.shclean — no sensitive patternsManual
select_checks()fuzz-tested against a literal transliteration of the four old grep
regexes across 20,000 random paths — 0 mismatches. This includes the
deliberate asymmetry where
.pyidoes not trigger the meta check(the old pattern ended in
\.py$), which is pinned by a test.(
OSErrorcaught), no arguments, unmatched path, and a check thatactually runs and fails.
sieval/tasks/in a live session and captured both sync scriptsrunning. Feeding the same payload to the new single-hook command
gives 2 / 0 / 0 sync invocations for a task file, a
scripts/file,and an empty path.
(
opencode run, freemuse-spark-1.3-contributor-freetier) changedthe file, and
file.editedreached the script: both sync scripts ranand the missing-binary path exited 0. Payload shape also matches the
installed SDK type defs (
EventFileEdited.properties.file).Confirmed opencode emits
file.editedfrom inside thewrite/edit/patch tools, not from a file watcher — so the plugin
tracks agent edits rather than any change on disk.
three surfaces (probed by creating one), with both tracked symlinks
still at mode 120000.
sanitize.shcatches a self-exempting# see AGENTS.mdprobe lineand stays clean on the real tree.
paths:key (reversemutation: with the guard deleted, all three new cases fail with
DID NOT RAISE).
Harness differences measured (documented in
AGENTS.md)Layer and scoped rules are auto-loaded only under Claude Code. Tested
on a synthetic tree: Codex never reads a
CLAUDE.mdbelow the repo root —not even with the working directory inside that folder — and renaming the
same file
AGENTS.mdloads it immediately; opencode only walks upwardfrom the working directory. So for those harnesses the 16 layer and scoped
rules are a pointer to follow, not an injection.
Checklist
Required (all PRs)
type(scope): description)AI-Generated Code - <model> (<provider>)in module docstringcore/replacement each in
post_edit_checks.py, and no other caller read them🤖 Generated with Claude Code