Skip to content

Repeated malformed tool calls never self-correct #65

Description

@micpapal

Summary

Once a model calls a tool with malformed arguments, it tends to keep calling
it the same wrong way, turn after turn, instead of adapting to the error
because each retry is appended to history on top of the previous (also
broken) attempt.

Example

get_metrics: {"payment_db": null, "window": "15m"}

should have been

get_metrics: {"service": "payment_db", "window": "15m"}

The model drops the service key and promotes its intended value into a bare
key with a null value. It gets back "required parameter 'service' is missing", and on the next turn issues the exact same malformed call again.

Confirmed directly: diffing the assembled context between two consecutive
repeats of the loop shows the entire prior (broken) exchange preserved
untouched, with the new (also broken) attempt appended on top:

messages[200][:199] == messages[198]   # True

Root cause

library-standard/src/mas/library/standard/plugins/context/assembler.py,
commit 247131e ("fix(context): provider-safe assembly after history trim
(supersedes #55) (#62)"):

- turn_msgs = [m for m in messages if m.get("role") in ("user", "assistant")]
+ turn_msgs = [m for m in messages if m.get("role") in ("user", "assistant", "tool")]

Before this change, tool-role messages were excluded from turn_msgs, and
since the assembled output is rebuilt from turn_msgs as
system_msgs + managed_past + current_and_after, tool results were
effectively dropped once history management ran. That was itself a real bug
(it could leave an assistant's tool_calls without their paired tool
results), and this commit correctly fixes it — but as a side effect, a
model's own broken call and the identical error it produced now persist in
every subsequent turn for the rest of the run, instead of aging out. Before
#62, that unintentional dropping acted as a de facto reset.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions