A refusal whose ledger says the messages fit lowers the completion ceiling to what the window leaves, and history trimming leaves a list shorter than the stored history unchanged

[Maya-Kid]

The two follow-ups from #973's review.

Problem

• Context::apply replaces the first N non-system messages of rig's list with the trimmed history, N being the size of the stored history. It trusted that rig's list holds the whole prior history. If the list ever held fewer messages, the current question would have been counted as history and dropped, silently.
• A refusal caused by our own max_tokens on a small-window model has been read as a context-window refusal since Long conversations shed old exchanges and recover from context refusals #973. An example: "maximum context length is 8192 tokens. However, you requested 69536 tokens (4000 in the messages, 65536 in the completion)".
  • Only the pipeline calls send that ceiling: extraction, alignment, time resolution and the other callers of chat_retrying_rate_limits_at. Chat sends none.
  • On those calls the retry at a lower ceiling was skipped, and the failure was worded as a prompt that exceeds the window.
  • Before Long conversations shed old exchanges and recover from context refusals #973 the retry took the window itself as the ceiling. The messages plus the window still exceed the window, so that retry failed as well.

Change

Trimming falls back. apply counts the non-system messages it was given; when there are no more of them than the stored history holds, it returns the list unchanged and logs it. An oversized request then fails at the endpoint, as it would have, instead of going out without its question.

Which part overflowed. utopia-llm reads the ledger some endpoints put in the refusal: the window, the tokens the messages take and the tokens the completion asked for (OpenAI, DeepSeek and vLLM word it this way; DeepSeek's was confirmed in #973).

• When the messages fit and the window leaves at least 1,024 tokens for an answer, the refusal is not a context refusal: stated_completion_ceiling returns what the window leaves, and the existing retry sends the request once more at that ceiling and remembers it on the client.
• When the messages themselves fill the window, or the refusal carries no ledger, it is a context refusal as before, with the stated window.

Verification

First verified on 2026-09-28 against dev 0815a41 in containers on a Linux host (workspace 1,255 passed). It was then rebased onto dev 798fb24 without conflicts: since 0815a41, only #996 touched utopia-llm/src/lib.rs, in other lines. It was verified again there, on macOS (arm64): Rust 1.98.1, PostgreSQL 16 with pgvector 0.8.2, and Poppler 26.06.

Tests

• chat_context.rs: a_list_shorter_than_the_history_is_sent_unchanged.
• utopia-llm, a_completion_overflow_is_a_ceiling_to_lower_and_not_a_context_refusal:
  • the same ledger is tried with the generic code and with context_length_exceeded;
  • a ceiling is only lowered;
  • messages that fill the window, or leave it under 1,024 tokens, stay a context refusal with the window.
• utopia-llm, a_completion_overflow_is_retried_at_what_the_window_leaves: a scripted endpoint refuses 65,536 with a ledger of 200 in the messages, the retry asks for 7,992, and the answer comes back.
• The existing ceiling test now expects 32,668 for the message it used to expect nothing for. It also has a message without a ledger, which still yields nothing.

On dev (798fb24) with the new tests

The control is dev plus the tests, with the code reverted:

• In utopia-llm, three tests fail: a_completion_overflow_is_a_ceiling_to_lower_and_not_a_context_refusal, a_completion_overflow_is_retried_at_what_the_window_leaves and the existing ceiling test. 42 pass.
• In chat_context.rs, a_list_shorter_than_the_history_is_sent_unchanged fails; 4 pass.

This branch

• utopia-llm: 45 passed.
• cargo fmt --all --check and cargo clippy --locked --workspace --all-targets -- -D warnings are clean.
• UTOPIA_TEST_REQUIRE_DB=1 UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspace: 1,323 passed, 0 failed, 6 ignored.
• No web file changes.

Known limits

• Only a refusal that states the ledger is told apart; "maximum context length is N tokens; request has M tokens" stays a context refusal.
• The ceiling learned this way is what the window left for that request's messages; a later request with more messages can be refused again and lowers it further, which is the existing one-way behaviour.

[WaylandYang]

Approving. Thanks @Maya-Kid. Reviewed at df4e879: the ledger is read only in the one wording OpenAI, DeepSeek and vLLM share, a refusal whose messages fit lowers the ceiling to what the window leaves instead of being read as a context refusal, and apply sends a list shorter than the stored history unchanged rather than swallowing the question. The two-response test pins the whole retry path.