Skip to content

Review pipeline: per-tool deadlines + fail-fast budget so one stalled query can't kill a review #46

Description

@jschatz1

Problem (2026-08-27 post-mortem, P3)

Review runs died of their own infrastructure repeatedly today:

  • A single kai_callers graph query stalled 5+ minutes inside a review, ate the 5-minute model-time budget, and the salvage conclusion call then exceeded its own 3-minute deadline → fail-loud with zero findings (run 12fa021e, kai-desktop PR chore(ci): run ci tests on branch push #10 head 1bb6d88).
  • The same signature killed kai-desktop PR Update billing to work with seat based pricing model #12's review three times on unchanged content (runs af6fc236, 56a04336, f07686d1) — multi-minute provider gaps between tool batches, budget expiry, conclusion-call timeout.
  • Six parallel reviews from one stack push contended and produced same-sha runs with contradictory pass/fail outcomes.

kai_grep already has a 120s wall clock (grepToolWallClock); kai_callers and the other graph tools do not.

Task

  • Hard per-tool-call deadline for every review-context tool (kai_callers, kai_impact, symbol lookups) — a stalled query returns a labeled partial, never eats the budget.
  • Fail-fast budget accounting in kai review-commit: when >N% of budget went to tool stalls rather than model output, conclude early from what exists instead of riding to expiry.
  • Give the conclusion call a provider-latency-aware deadline (today's 3min was exceeded by a single model call three separate times) or one retry on a fresh connection.

Goal

Reviews are a reliable gate: a slow graph or a slow provider degrades a review to smaller-but-honest, never to dead.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions