Skip to content

review/scan: reasoning models can return empty content — misleading 'EOF while parsing' error #536

Description

@ajianaz

Observed

codecoradev/cosy#63 (comment 5447415658): `cora review` failed with

failed to parse LLM JSON response: parse failed (original: EOF while parsing a value at line 1 column 0, after repair: …)

while the model (glm/glm-5.3 via Bifrost) had actually produced a valid findings JSON array — visible in backend logs.

Root cause

Default `max_tokens: 4096` is consumed by the model's `reasoning_content` (chain-of-thought) before the final answer; provider returns HTTP 200 with `content: ""` and `finish_reason: "length"`. cora:

  • never reads `finish_reason` or `reasoning_content` (`src/engine/llm.rs` `ChatResponse`)
  • silently defaults empty `content` to `""` and feeds it to the JSON parser, producing the misleading `EOF while parsing a value at line 1 column 0`
  • retries only re-send the same 4096 budget, so the retry always fails identically

Fix direction (productivity over token frugality)

  1. Raise default `max_tokens` 4096 → 8192.
  2. Parse `finish_reason` + `reasoning_content` from the response.
  3. Empty content + `finish_reason=length` → auto-retry with doubled budget (up to 32768) inside `chat_completion`.
  4. Last resort: if `reasoning_content` contains JSON, use it as the raw response.
  5. Empty raw at parse layer → explicit "provider returned an EMPTY response" error instead of serde EOF noise.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions