fix(mcp): stop unsandboxed tests writing the real needs-auth cache - #994
Merged
Merged
Conversation
`mark_and_clear_needs_auth_round_trip` drove `mark_needs_auth` / `clear_needs_auth`, which write `mcp-needs-auth-cache.json` under `ORGII_HOME`, without holding a `test_env::sandbox()` guard. That violates the invariant `test_helpers::test_env` documents: every test mutating or reading `ORGII_HOME`-backed state must go through the canonical home lock. Two consequences, one visible and one not: - Under `cargo test`'s parallelism the unlocked test wrote "srv" into whichever sandboxed test's tempdir `ORGII_HOME` pointed at. When that happened to be `concurrent_sets_all_persist`, its `assert_eq!(cache.len(), 8)` saw a foreign entry and failed. This is the intermittent CI failure at needs_auth_cache.rs:280 — the "concurrent" test was never at fault; its own writes are already serialized by `write_lock()`. - With `ORGII_HOME` unset, `cache_path()` fell back to `dirs::home_dir()`, so running the suite wrote to and deleted the developer's real `~/.orgii/mcp-needs-auth-cache.json`. Fixes both at the boundary that resolves the path: `cache_path()` now panics under `#[cfg(test)]` when `ORGII_HOME` is unset instead of silently reaching for the real home, so a future test cannot recreate the invalid state without failing loudly. The offending test takes a sandbox guard. Production behavior is unchanged — the `cfg(not(test))` arm keeps the existing `~/.orgii` fallback, and `#[cfg(test)]` covers only agent-core's own unit-test build.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
specialization::mcp::needs_auth_cache::tests::concurrent_sets_all_persistfails intermittently in CI — most recently on
#992 (3180 passed, 1 failed,
panicking at
needs_auth_cache.rs:280), a PR that changed zero Rustfiles. Re-running the job passed, which is the signature of a race
rather than a broken assertion.
The "concurrent" test was never at fault. Its own eight writes are
already serialized by the module's
write_lock(), so they cannot loseeach other. The interference came from outside the test.
specialization::mcp::manager::tests::mark_and_clear_needs_auth_round_tripcalls
mgr.mark_needs_auth(...)/mgr.clear_needs_auth(...), which gothrough
needs_auth_cache::set_entry/remove_entryand writemcp-needs-auth-cache.jsonunderORGII_HOME. It held notest_env::sandbox()guard, violating the invariant thattest_helpers::test_envdocuments in its module docs:ORGII_HOMEis process-global, so undercargo test's parallelism thatunlocked test wrote the key
"srv"into whichever sandboxed test'stempdir the variable happened to point at. When that was
concurrent_sets_all_persist, itsassert_eq!(cache.len(), 8)saw nineentries and failed.
There is a second consequence that never showed up as a test failure:
when
ORGII_HOMEwas unset,cache_path()fell back todirs::home_dir(), so running the test suite wrote to and deleted thedeveloper's real
~/.orgii/mcp-needs-auth-cache.json.Solution
Fixed at the boundary that resolves the path, not at the assertion that
noticed the symptom.
needs_auth_cache::cache_path()now panics under#[cfg(test)]whenORGII_HOMEis unset, instead of silently reaching for the real home.A future test cannot recreate the invalid state without failing loudly
and with a message that names the fix. This is the regression guard:
it makes the producing condition impossible rather than asserting on
its downstream effect.
mark_and_clear_needs_auth_round_triptakes atest_helpers::test_env::sandbox()guard.concurrent_sets_all_persistis deliberately left exactly as it was. Itwas correct; weakening its assertion or retrying it would have hidden
the defect instead of removing it.
Production behavior is unchanged: the
cfg(not(test))arm keeps theexisting
~/.orgiifallback verbatim, and#[cfg(test)]applies only toagent-core's own unit-test build — integration tests under
tests/compile against the non-test build and are unaffected.
Potential risks
cfg(test)panic could surface other unsandboxed callers. Thatis the intent, but it does mean a test elsewhere in agent-core reaching
an MCP lifecycle path without a sandbox would now fail loudly instead
of silently touching the real home. The full agent-core suite was run
to bound this: 3181 passed, 0 failed, and the new panic fired on
nothing else. The blast radius is exactly the one test fixed here.
#[cfg(test)], so it cannot affect the shipped binary or anyintegration test.
cache_path()ispub(crate), so no other crate can reach it.the needs-auth cache specifically. The same class of bug — a test
touching process-global env state without
lock_home()— could existin other modules; this PR does not audit for that.
Verification
Run in a clean worktree off
develop(Darwin,cargo test -p agent_core):Full agent-core suite: 3181 passed, 0 failed, 2 ignored. CI's
failing run was 3180 passed + 1 failed = the same 3181 tests, so the
counts line up.
MCP subset (
cargo test -p agent_core specialization::mcp): 103passed, 0 failed.
cargo clippy -p agent_core --all-targets -- -D warnings: clean.(The unrelated pre-existing
block v0.1.6future-compat dependencywarning is not introduced here.) The pre-commit hook's scoped clippy
run also passed.
Root cause proven, not assumed. Temporarily removing only the new
sandbox guard makes the offender fail deterministically and
immediately:
This confirms that test really was resolving the cache path outside any
sandbox. The guard was restored before committing.
Not run: the original flake is probabilistic, so "the suite passed once"
is not by itself proof it is gone. The deterministic demonstration above
is the load-bearing evidence — it identifies the specific writer and
shows the enforcement catches it at the moment it happens, rather than
relying on repeated green runs.