Skip to content

The incomplete-review bundle says so in a field, not just in prose - #96

Merged
jschatz1 merged 1 commit into
mainfrom
fix/incomplete-review-flag
Sep 9, 2026
Merged

The incomplete-review bundle says so in a field, not just in prose#96
jschatz1 merged 1 commit into
mainfrom
fix/incomplete-review-flag

Conversation

@jschatz1

@jschatz1 jschatz1 commented Sep 8, 2026

Copy link
Copy Markdown
Member

#90 taught a review that ran out of road to report how far it got instead of vanishing. The prose it writes is honest. The bundle around it is not yet — incomplete stayed a local bool and never left this process.

Why that matters downstream

kai-server picks the review's headline from the finding's counts. An incomplete finding carries no risks, no decisions and an unknown intent, because this command deliberately leaves match and readiness Unknown rather than guess a verdict:

It never guesses a verdict. The caller leaves match and readiness Unknown, which is the one value that means "I have no opinion" rather than any point on the merge/do-not-merge scale.

That is arithmetically identical to a review that read the whole change and liked it. So both landed in the same branch, and kai-desktop#304 (9m59s, 27 turns) and kai-server#186 (9m47s, 29 turns) each opened with

Read through this. Nothing jumped out, though I couldn't tie it to a stated goal…

three lines above their own "This review did not finish."

The renderer could not have known. The one process that did know was this one, and it dropped the fact on the floor.

The change

incomplete goes into the emitted bundle.

omitempty keeps a finished review's JSON byte-identical to what it was before this field existed: the flag is present only when it is true, so no complete review can be misread as an incomplete one, and a server that predates the field ignores it.

Companion

kaicontext/kai-server#210 consumes this. Until the pinned reviewer image carries this build, it falls back to matching the prose — so the user-visible fix does not wait on a re-pin.

Verified

TestIncompleteBundleCarriesTheFlag covers both directions: the flag is present when the run stopped, and absent from a finished review's bundle. go test ./cmd/... green (exit 0).

🤖 Generated with Claude Code

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jschatz1 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 75c6922d-3f2f-4131-9676-c2869e0fdb7f


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kaicontext kaicontext Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kai review

Kai Summary

Read through this one. 2 things worth your eyes before it merges. 👇

Review

Scope: kai-cli repo, working tree at commit 62f3000. The change touches cmd/kai/review_commit.go and cmd/kai/review_commit_incomplete_test.go. The finding.Finding type lives in the kai-engine external dependency (v0.6.56), whose source is not vendored in this repo, so I could not inspect that type directly — one concern below rests on that.

The rest of the read-through

What the change does: Adds Incomplete bool json:"incomplete,omitempty" to the anonymous struct that runReviewCommit marshals to stdout in JSON mode, feeding it the existing local incomplete variable (set at review_commit.go:325-332 only when a review produced no parseable prose, no risks, no decisions, and unknown intent). The omitempty keeps a completed review's JSON byte-identical to its pre-change form. The production edit is correct and well-scoped: the variable is in scope, the field placement mirrors the existing Review/Depth pattern, and the exit-non-zero-after-emit ordering is preserved.

The test does not cover the fix. TestIncompleteBundleCarriesTheFlag (review_commit_incomplete_test.go:101-132) defines its own local bundle struct (lines 102-106) and hands that to json.Marshal. It never calls runReviewCommit, never references the production anonymous struct at review_commit.go:454-459, and never exercises the JSON branch of the command. Consequently:

  • If you revert the review_commit.go half of this diff entirely, this test still passes — it marshals a struct it defined itself, not the one the command builds.
  • What the test actually asserts is that encoding/json emits a field tagged json:"incomplete,omitempty" when the value is true and omits it when false. That is a property of the standard library, guaranteed regardless of whether the command carries the flag.

This is the test that passes on the unfixed code: the behavioral claim — "an incomplete review's serialized bundle carries incomplete:true, and a complete review's does not mention it" — is never exercised against the code that produces the bundle. The fix is real in the production code but unverified. A test that caught the regression would need to reach the marshaling path: either drive runReviewCommit in JSON mode (the existing review_commit_fast_test.go:233 already calls it, so the harness is there) or, more cheaply, extract the bundle struct + marshal into a small named helper and assert on its output for the incomplete and complete cases. As written, removing the field from review_commit.go:458 would leave the suite green.

Pre-existing design boundary (not a defect this diff introduces): incomplete is only set true when the review is completely hollow — no prose, no risks, no decisions, unknown intent (review_commit.go:326). A review that ran out of time but still emitted some risks or a partial verdict stays incomplete == false and renders as a finished review. That's the PR#89/#90 design, unchanged by this diff; I note it only because the intent text frames the flag as "a review that stopped early," and a partially-stopped review is the case the flag does not cover. Worth confirming that's the boundary you want, but it is not something this change altered.

Unverifiable external type: The anonymous struct embeds finding.Finding (from kai-engine v0.6.56) and adds Incomplete as a promoted sibling to Review and Depth. If finding.Finding already declares a field whose JSON tag is incomplete, Go's encoding/json would silently drop both on a name collision (no error). I could not read that type from this repo. The existing Review/Depth fields would already have hit this if Finding had either name, so the collision risk is low and follows an established pattern — but the correctness of the serialization rests on it, and I could not confirm it.

Cross-repo coordination: The intent notes a server-side companion consumes this flag and, until a matching reviewer image ships, falls back to prose matching. The companion and the pinned image are outside this repo, so I could not verify either exists or is wired to read "incomplete". If the companion isn't shipped or doesn't read the field, the flag round-trips harmlessly (older servers ignore unknown JSON keys) but the contradictory-headline problem persists — the prose fallback is the only thing still working. Nothing charges a customer or changes access here; it's an internal delivery detail.

Overall: The production change is correct and minimal. The test is the gap: it's a tautology that would pass with the fix removed, so the fix ships unverified. Small fix to the test, then this is ready.

Important files changed
File Change
cmd/kai/review_commit.go modified · +11 −3
cmd/kai/review_commit_incomplete_test.go modified · +41 −0

Decisions

Correct as written, but somebody should say yes to these:

  • The incomplete flag is only set for fully-hollow reviews (no prose, no risks, no decisions, unknown intent); a partially-stopped review that still emitted findings is indistinguishable from a completed one. This is the pre-existing PR#89/#90 design, not altered by this diff, but confirm that's the boundary you want before relying on the flag to mean "stopped early" generally.

+52 −3 · 2 files · reaches 9 · the full analysis
💬 Reply to any of my comments and I'll answer, or say @kaicontext anywhere on this PR — a question, or "take another look at the retry logic".

// this field the server has only a literal sentence to match on, and two
// timed-out reviews shipped with "Nothing jumped out" as their opening line
// (kai-desktop#304, kai-server#186, 2026-09-08).
func TestIncompleteBundleCarriesTheFlag(t *testing.T) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test defines and marshals its own local bundle struct instead of exercising the production struct in runReviewCommit, so it passes on the pre-change code and does not verify the fix; drive the JSON branch of the command (or extract the marshal into a testable helper) and assert on its output.

#90 taught a review that ran out of road to report how far it got instead
of vanishing. The prose it writes is honest. The bundle around it is not
yet: `incomplete` stayed a local bool and never left this process.

Downstream, kai-server picks the review's headline from the finding's
counts — and an incomplete finding has no risks, no decisions and an
unknown intent, because this command deliberately leaves match and
readiness Unknown rather than guess a verdict. That is indistinguishable
from a review that read everything and liked it, so kai-desktop#304 and
kai-server#186 both opened with "Nothing jumped out" three lines above
their own "This review did not finish" (2026-09-08).

The renderer could not have known. The one process that did know was this
one, and it dropped the fact on the floor.

So it goes in the bundle. `omitempty` keeps a finished review's JSON
byte-identical to what it was before this field existed — the flag is
present only when it is true, so no complete review can be misread as an
incomplete one, and an older server that does not know the field ignores
it.

The server-side companion consumes this and, until the pinned reviewer
image carries this build, falls back to matching the prose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jschatz1
jschatz1 force-pushed the fix/incomplete-review-flag branch from 62f3000 to ad09c26 Compare September 9, 2026 06:18

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jschatz1 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@kaicontext kaicontext Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kai review

Kai Summary

Nothing new since my last review — the point already on this PR still stands. 👇

Where I'd land: 3/5 — small fixes first.

I reviewed this branch (repo kai, files cmd/kai/review_commit.go and cmd/kai/review_commit_incomplete_test.go). I could not read the embedded finding.Finding struct — it lives in the external module github.com/kaicontext/kai-engine v0.6.56, and there is no vendor directory in this workspace, so that struct's actual fields are outside my reach.

What the change does

runReviewCommit already salvaged an incomplete review's prose and set a local incomplete bool, but that bool never left the process. This change adds an Incomplete bool json:"incomplete,omitempty" field to the anonymous struct the command marshals at review_commit.go:456, threading the local incomplete through as the fourth positional value. The intent is sound and the wiring is correct — four positional values matching the embedded finding.Finding plus Review, Depth, Incomplete. With omitempty, a completed review serializes with no incomplete key at all; only a timed-out run carries "incomplete":true, so the byte-identical-backward-compatible claim holds by construction. The non-zero-exit-after-emit ordering is preserved (lines 482-501 still return rcErrIncompleteReview after printing).

The real concern

TestIncompleteBundleCarriesTheFlag (review_commit_incomplete_test.go:101) declares its OWN local bundle type and marshals THAT. It asserts that a hand-rolled struct with json:"incomplete,omitempty" emits "incomplete":true when true and omits it when false — which is a test of encoding/json's tag handling, not of runReviewCommit. If you deleted the Incomplete field from the production struct at review_commit.go:460, this test would still pass, because it never touches the production struct. I checked every test that calls runReviewCommit (review_commit_fast_test.go:233, review_commit_incomplete_test.go:108); none sets reviewCommitFormat to "json" or inspects the JSON output, so the reviewCommitFormat == "json" branch (line 428) is unexercised by this diff.

This matters because the whole point is that the downstream renderer gets the flag structurally. If the field were mis-tagged, mis-wired, or dropped in a future refactor, this test would stay green while kai-server went back to deriving "Nothing jumped out" from the counts. The fix is quick: either extract the production anonymous struct into a named type so both the command and the test reference the same definition, or drive runReviewCommit with reviewCommitFormat="json" and assert stdout contains "incomplete":true.

What I could not confirm

I could not read finding.Finding from the pinned kai-engine v0.6.56. The author's framing assumes Finding has no field whose json tag is "incomplete" today. If that assumption is false and a future kai-engine bump adds an Incomplete field tagged "incomplete" but without omitempty, the outer field here would shadow it, and complete reviews would silently drop a key they used to emit — breaking the byte-identical claim. The author owns that repo too, so the practical risk is low, but the boundary I verified is this repo only.

Nothing here charges a customer, caps usage, sends, publishes, deletes, or changes access — it's a diagnostic flag — so no DECISIONS.

The change itself is correct and does what it intends. The one gap is that the test insulates itself from the code it claims to cover. Fix the test to exercise the production struct and this is ready to merge.

Important files changed
File Change
cmd/kai/review_commit.go modified · +11 −3
cmd/kai/review_commit_incomplete_test.go modified · +41 −0

+52 −3 · 2 files · reaches 10 · the full analysis
💬 Reply to any of my comments and I'll answer, or say @kaicontext anywhere on this PR — a question, or "take another look at the retry logic".

@jschatz1
jschatz1 merged commit dc489e2 into main Sep 9, 2026
8 checks passed
jschatz1 pushed a commit that referenced this pull request Sep 9, 2026
The facts were already there. rcRunReviewAgent gathers them on every run —
its own comment says so, "collected unconditionally and cost nothing" —
and then throws them away unless the review died.

So a review that finished said nothing about its own coverage, and the
only account of scope on a healthy review was the model's "Scope:"
paragraph. That is prose, and prose can be wrong about itself: reviews on
kai-desktop#288 and #300 named the repository they were reading as
kai-engine and kai-server respectively.

rcCoverage is the half that cannot hallucinate — files opened, turns,
seconds — and it now rides in the bundle on every grounded run.

Nil in, nil out: the fast pass makes one call over the diff and opens
nothing, so `omitempty` drops the field entirely rather than publishing a
record that claims zero files were read. An absent manifest is not a claim
of no coverage, and the server-side companion treats it that way.

Stacked on #96 — same bundle struct, and that one should land first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant