Conversation
Add ApplyFixThenCompileTests with an end-to-end test that applies a code fix via apply_fix, then calls al_compile (onlyErrors: false) and asserts the fixed diagnostic is gone. Characterization result: 10/10 pass with no settle delay — almcp's FileSystemWatcher + WaitForProcessingAsync drains before compile on this machine. A skipped al_getdiagnostics variant documents that it returns cached (zero) results instead of re-analyzing. AlMcpProxyTests.CreateProxy() gains an optional fixtureName parameter so the new fixture can reuse the helper. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Update the apply_fix and apply_fix_all tool descriptions to direct callers to verify with al_compile (onlyErrors: false) instead of al_getdiagnostics. Add a "Verifying a fix" section to the README and a bullet in AGENTS.md explaining the difference: al_compile awaits almcp's FileSystemWatcher drain, while al_getdiagnostics returns cached results without re-analyzing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…kipped al_getdiagnostics test - Pass Cts.Token to ApplyFixTool.ApplyFix so the call respects the test's 90-second cancellation timeout (CONFIRMED review finding). - Remove ApplyFix_ThenAlGetDiagnostics_ReportsStaleDiagnostics: its only assertion (expects LC0020) contradicted its own skip reason (reports zero diagnostics). The knowledge is preserved in the class-level summary (PLAUSIBLE review finding). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
apply_fix→al_compileagainst the realalmcpchild process, confirming fixed diagnostics disappear.al_getdiagnosticsto verify fixes.What changed
Test files:
ApplyFixThenCompileTests.cs(new) —ApplyFixAlMcpFixturelaunches the full server, applies a code fix, then callsal_compile(onlyErrors: false)and asserts the fixed diagnostic is gone. A skippedal_getdiagnosticsvariant documents why that endpoint cannot verify fixes.AlMcpProxyTests.cs—CreateProxy()gains an optionalfixtureNameparameter so the new fixture can reuse the helper.Docs & tool descriptions:
ApplyFixTool.cs/ApplyFixAllTool.cs— tool descriptions now say: "Verify withal_compile(onlyErrors: false)."README.md— new "Verifying a fix" section after theonlyErrorsnote.AGENTS.md— new bullet in "Tool patterns" explainingal_compilevsal_getdiagnosticsafter a fix.Reproduction outcome
Characterization (10 consecutive runs, no settle delay): 10/10 PASS.
This is the "passes every run" branch of the decision tree. The race between our file write and
almcp'sProjectWatcher(FileSystemWatcher) is not reproducible on this machine:al_compileawaitsWaitForProcessingAsyncbefore compiling, and the OS delivers the change event quickly enough that the old text is never compiled.No runtime code change was needed. The test stays as a regression guard.
The
al_getdiagnosticsvariant confirmed the watcher-drain analysis: it returns{"succeeded":true,"diagnostics":[],"errorCount":0}— zero diagnostics — because it reads cached compilation results rather than re-analyzing. This test is[Skip]-ped with documentation.Likely sources of the original report: using
al_getdiagnostics(returns cached, never re-compiles), a child that had runal_downloadsymbols(disposes the workspace), or a project outside the child's--projectsset.Test plan
dotnet build --configuration Release— 0 warnings, 0 errorsdotnet test --configuration Release— 69 passed, 0 failed, 1 skipped (theal_getdiagnosticsvariant, intentionally skipped)ApplyFix_ThenAlCompile_NoLongerReportsFixedDiagnostic— 10/10 pass with no settle delayCorrects PR #20's known-issue note
PR #20 noted that
al_compilemight return stale diagnostics afterapply_fix, suggesting callers add a delay or callal_getdiagnostics. This was backwards: the childalmcpdoes not serve a cached compilation foral_compile. It has a per-projectFileSystemWatcher(ProjectWatcher) whose pending eventsal_compiledrains viaWaitForProcessingAsyncbefore compiling, so it always picks up on-disk changes.al_getdiagnostics, on the other hand, returns the last cached compilation result without re-analyzing — it is the endpoint that misses changes. Only a full child process restart produces a truly "clean" state, butal_compileis fresh enough for verification after a fix.🤖 Generated with Claude Code