Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,25 @@ This project follows pre-1.0 SemVer: minor versions may still include breaking c

### Fixed

- [fixed] The later model calls of a local goal no longer run short of
output after short replies. Each call was charged its whole output limit
even when the model wrote far less, and each call keeps half of what is
left for later ones, so calls 5 to 8 of an 8-call goal got 1,024, 512, 256
and 256 tokens. In a laptop run of 20 exercises with `qwen3.5:4b`, edits
that finished wrote a median of 1,057 output tokens, half of them between
about 880 and 1,310, and three of the four goals that failed ended with
their last calls cut off at 256 to 512 tokens. A call is now charged the
output tokens the runtime reports once it answers, and a reply without a
count stays charged its whole limit. The receipt's
`generated_tokens_reserved` total follows the same rule.
- [fixed] A restart edit no longer stops when its strategy's one-line source
anchor sits inside a small symbol that the edit prompt offers as one
whole-symbol choice. The edit call usually has a larger output limit than
the strategy call, so the symbol the strategy saw as separate lines could
become one choice, the anchor line was no longer a choice of its own, and
the goal ended with "Strategy source anchor was omitted from the edit
choices". The edit is now bound to the smallest choice that contains the
anchor, and the candidate's decision reason says so.
- [fixed] A local goal on a model that reasons no longer stops after one model
call when the first reply runs out of output. The baseline restart that
follows asks for a strategy, a JSON answer of about 100 tokens under a
Expand Down
4 changes: 3 additions & 1 deletion docs/local-harness-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,9 @@ creation probe and 256 for the tool-call probe, or a quarter of the context
(at most 2,048) when the model may think first. For coding goals, the saved profile
reserves up to one quarter of that context for each complete edit response;
the goal's shared token budget can reduce it further. When later attempts are
allowed, the runner keeps half the remaining token budget for them. This is a
allowed, the runner keeps half the remaining token budget for them. Each call
is charged the output tokens the runtime reports once it answers, not its
whole limit; a reply without a count stays charged its limit. This is a
generation allowance, not a measured claim that the model can produce a file
of that size. Existing profiles keep their saved ceiling until recalibrated.
Calibration first requests `think:false` so a thinking model can spend the
Expand Down
14 changes: 11 additions & 3 deletions docs/local-harness.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,10 @@ The small probe output caps do not set the ceiling for a coding goal. The
saved profile allows up to one quarter of its calibrated context for one
complete edit or new-file response, then the shared goal budget can reduce
that reservation. The worker keeps half the remaining token budget for a
later permitted model call. This is a context-based allowance, not measured
later permitted model call. A call is charged its whole limit while it runs,
then the output tokens the runtime reports once it answers, so a short reply
leaves the rest for later calls; a reply without a count stays charged its
limit. This is a context-based allowance, not measured
model generation capacity. A truncated runtime response is still rejected.
Previously saved profiles retain their smaller ceiling until recalibrated.
If neither edit format passes, calibration still saves and prints the raw probe
Expand Down Expand Up @@ -712,8 +715,13 @@ and following edit each reserve one of the same `generations` model-attempt
budget, even if admission later stops a chat request; the
strategy is a proposal, and only the resulting candidate bytes and fixed checks
can justify review. The edit's actual returned path and source anchor are
checked before materialization. An interrupted strategy remains pending evidence, not a
candidate.
checked before materialization. The edit call usually has a larger output
limit than the strategy call, so a small symbol the strategy saw as separate
lines can be one whole-symbol choice in the edit prompt. A one-line anchor
inside such a symbol binds the edit to the smallest choice that contains it,
and the candidate's decision reason says so. An anchor that no edit choice
contains still ends the search. An interrupted strategy remains pending
evidence, not a candidate.
Cancellation retains evidence. Reopening an unfinished receipt labels it
interrupted; it does not silently resume or mark it complete. Inspect the selected
candidate and its diff before applying it.
Expand Down
4 changes: 3 additions & 1 deletion phonton-types/src/local_run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,9 @@ pub struct LocalRunReceipt {
pub hypotheses: Vec<HypothesisEvidence>,
pub selected_candidate: Option<u32>,
pub checks_used: u32,
/// Missing runtime counters are unknown; admission reserves max output.
/// Output tokens charged to the shared budget: each call's whole output
/// limit while it runs, then the runtime's reported count once it
/// answers. A call without a reported count stays charged its limit.
pub generated_tokens_reserved: u64,
/// All reserved model attempts, including those stopped before chat transport.
#[serde(default, alias = "inference_calls_used")]
Expand Down
Loading
Loading