Skip to content

feat(agents): promote tool-call JSON repair to a ToolCallJsonRepair capability - #841

Open
mpfaffenberger wants to merge 2 commits into
mainfrom
feature/tool-call-json-repair-capability
Open

feat(agents): promote tool-call JSON repair to a ToolCallJsonRepair capability#841
mpfaffenberger wants to merge 2 commits into
mainfrom
feature/tool-call-json-repair-capability

Conversation

@mpfaffenberger

@mpfaffenberger mpfaffenberger commented Aug 21, 2026

Copy link
Copy Markdown
Owner

ToolCallJsonRepair capability

Thirteenth in the capability series (#828#836, #838#840). Converts the tool-call JSON repair feature — until now an eager monkeypatch of ToolManager.validate_tool_call in pydantic_patches.py — into a first-class pydantic-ai capability on the before_tool_validate seam. This is the first conversion in the series that retires monkeypatch surgery rather than a constructor kwarg or a run-site closure.

What the feature does

LLMs sometimes emit slightly broken JSON in tool-call arguments (trailing commas, missing quotes, unclosed braces). Repairing the raw args before validation via json_repair avoids burning a retry round-trip.

How it moved

  • New code_puppy/agents/_json_repair.py — stateless ToolCallJsonRepair(AbstractCapability):
    • before_tool_validate repairs string args and mirrors the repair onto the live ToolCallPart (call.args = repaired) — the same in-place custody the patch performed, so message history records the JSON the tool actually ran with. Verified empirically: the part passed to the seam is the object recorded in run state.
    • build_tool_call_json_repair() returns [] when the optional json_repair dependency is absent (mirrors the patch's quiet skip and the build_tool_output_limits conditional-splice pattern).
    • Stateless plain dataclass → spec-constructible with inherited defaults.
  • Both construction sites (_builder.py, subagent_invocation.py) splice the capability. Position is inert — the tool-validate seam never interacts with the history hooks.
  • The patch is demoted to a guest-agent fallbackexplicit-when-ours, fallback-for-guests, the same split feat: deliver Logfire tracing as an explicit Instrumentation capability #838 used for Logfire instrumentation. The patched validate_tool_call walks the run's ToolManager.root_capability tree with the public AbstractCapability.apply visitor and steps aside when ToolCallJsonRepair is present. Raw pydantic-ai agents built by plugins keep eager repair — the audit found one real dependent: wiggum's judge, which registers genuine read-only tools. btw's side-query agent has no tools (inert either way); shell_safety's agent builds through our builder (gets the capability).
    • The gate fails open: if the leaf-walk ever crashes, the patch repairs eagerly — old behavior, and idempotent alongside the capability (repairing repaired JSON is a no-op).

Parity audit

  • Seam placement: before_tool_validate is the only hook receiving raw pre-validation args; it fires exactly once per validate_tool_call (allow_partial=False only — no streaming-partial hazard).
  • Ordering vs the cp_ normalization patch: runtime order changes from repair args → normalize name → resolve to normalize name → resolve → repair args. Name normalization and arg repair are independent — inert reordering.
  • Bounded divergence (documented + pinned): calls to unknown/unavailable tools no longer get their recorded args repaired — _resolve_tool raises ModelRetry before the seam fires, where the patch repaired first and failed after. The call fails identically either way; history now keeps the model's true emitted bytes (arguably more honest). Pinned by test_unknown_tool_args_stay_raw_in_history.
  • Archaeology of the round: on pydantic-ai 2.31.0, tool-based structured output validates through validate_output_tool_call — a method the patch never wrapped. Output tools were never covered by this feature, so pydantic-ai's kind == 'output' hook carve-out changes nothing. (An output-tool repair via before_output_validate would be new behavior, deliberately out of scope — noted for a future PR if wanted.)

Tests

17 contract tests in tests/agents/test_tool_call_json_repair_capability.py:

  • direct-seam semantics: repair + custody, valid/dict/empty pass-throughs (object identity), swallowed repair failures, missing-dependency no-ops;
  • end-to-end FunctionModel runs: tool executes with repaired args, zero retries burned, history carries repaired bytes;
  • wire parity: guest-agent-under-patch vs capability-agent produce identical recorded args and output, plus retry-feedback parity on the unknown-tool path;
  • patch gating both ways: repair invoked exactly once with the capability present (patch stepped aside), guests still repaired; _run_owns_json_repair leaf-walk incl. CombinedCapability nesting;
  • wiring pins for both capabilities=[...] blocks + a seam-signature pin so a pydantic-ai hook-contract change fails loudly.

Full suite: 7615 passed, 0 failed (28 skipped, 1 xpassed). Reviewed by code-puppy clone: pass 1 APPROVE with 1 non-blocking + 1 nit (both applied in 103571e), pass 2 APPROVE with zero findings.

Merge-order note

Touches the same two capabilities=[...] blocks as the other twelve open capability PRs — whichever lands last eats a trivial rebase.

…apability

Convert the patch_tool_call_json_repair monkeypatch (eager
ToolManager.validate_tool_call surgery) into a first-class pydantic-ai
capability on the before_tool_validate seam.

- New code_puppy/agents/_json_repair.py: stateless ToolCallJsonRepair
  repairs malformed JSON string args via json_repair, mirrors the repair
  onto the live ToolCallPart (same history custody as the patch's
  in-place mutation), and returns the repaired args for validation.
  build_tool_call_json_repair() returns [] when json_repair is absent,
  mirroring the patch's quiet optional-dependency skip.
- Both construction sites (_builder.py + subagent_invocation.py) splice
  the capability; position inert (tool-validate seam never interacts
  with the history hooks).
- The monkeypatch is demoted to a guest-agent fallback: it steps aside
  when the run's root_capability tree contains ToolCallJsonRepair
  (public AbstractCapability.apply leaf-walk), and keeps repairing for
  raw pydantic-ai agents built by plugins (wiggum's tool-wielding
  judge). Explicit-when-ours, fallback-for-guests, same split as
  Logfire instrumentation.
- Bounded divergence, documented + pinned: unknown/unavailable tool
  calls no longer get their recorded args repaired (resolution raises
  before the seam fires); the call fails with the identical ModelRetry
  and history keeps the model's true emitted bytes.
- Archaeology: on pydantic-ai 2.31.0 output tools validate through
  validate_output_tool_call, which the patch never wrapped - output
  tools were never covered, so the seam's kind=='output' carve-out
  changes nothing.

16 contract tests: seam semantics, end-to-end FunctionModel runs with
history-custody pins, wire parity vs the eager patch, patch gating both
ways, unknown-tool divergence pin, wiring + seam-signature pins.
- Pin retry parity for the unknown-tool divergence: identical
  RetryPromptPart feedback and recovery under the eager patch and the
  capability; only the recorded args differ (raw vs repaired), which is
  the entire documented divergence. Note that unavailable tools share
  the same raise-before-seam code path.
- Strengthen the seam-signature pin to compare parameter names, kinds,
  and defaults (annotations excluded deliberately: the seam spells
  RawToolArgs, we spell the underlying union).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant