fix(openai-compatible): send list tool return as native list content

[Million-mo]

Problem

pydantic-ai's OpenAIChatModel always serializes list-type ToolReturnPart content as a JSON string, instead of using the OpenAI SDK's native list[ChatCompletionContentPartTextParam] format.

This breaks OpenAI-compatible models (e.g. GLM-5) whose chat templates branch on tool role content being a string vs list:

• String branch (current bug): one <|tool_return|> block with escaped JSON text
• List branch (fixed): multiple <|tool_return|> blocks, each with a native result

Closes #112.

Solution

Subclass OpenAIChatModel as OpenAICompatibleModel, overriding _map_user_message to send list content as native list[ChatCompletionContentPartTextParam] when:

1. ToolReturnPart.content is a list
2. No multimodal files were extracted
3. Tool return was not a failure (errors need wrapping)

All other cases (string/dict/int content, multimodal files, failed returns) fall back to the parent's string serialization.

Affected providers

All OpenAI-compatible providers now use OpenAICompatibleModel:

• deepseek:
• grok:
• openrouter:
• perplexity:
• lm-studio:
• zen:
• copilot:

openai: (Responses API) and openai-chat: (pydantic-ai built-in) are unaffected.

Changes

• New: src/wolfharness/models/openai_compatible.py — OpenAICompatibleModel class
• Modified: src/wolfharness/utils/model_helpers.py — use OpenAICompatibleModel in _get_openai_based_model() and copilot: prefix
• New: tests/unit/test_openai_compatible_model.py — 13 unit tests
• New: changelog/unreleased/2026-08-28-openai-compatible-tool-return-list-content.md

Verification

• ✅ 13/13 unit tests pass
• ✅ Ruff lint + format clean
• ✅ mypy type check clean
• ✅ All pre-commit hooks pass

[github-actions]

Code Review — PR #398

Reviewed the diff against the pinned pydantic-ai 2.17.0 (verified byte-for-byte against the upstream _map_user_message it forks). I independently ran the suite in a sandbox with the pinned versions: 13/13 unit tests pass, ruff check + ruff format --check are clean with the repo config, and mypy --strict is clean on the new module. No blockers.

Important

1. openai-chat: — the actual GLM route — does not receive the fix. The PR body says openai-chat: is "unaffected," which is true, but the repo itself configures the target model via this prefix: tests/conftest.py:60 defaults to openai-chat:svc/glm-4.7, and the OpenCode server resolves openai-chat:svc/glm-4.7 (config_routes.py:308, acp_agent.py:1001). _infer_single_model (src/wolfharness/utils/model_helpers.py:156) routes openai-chat: through pydantic-ai's built-in infer_model_ → stock OpenAIChatModel, which still serializes list tool returns as JSON strings. So a user writing model: "openai-chat:glm-5" (no base_url) gets the pre-fix behavior, and issue pydantic-ai OpenAI tool return list content 被序列化为 JSON 字符串，与 OpenAI-compatible 模型（如 GLM-5）的 chat template 不兼容 #112's target is only fixed when GLM is reached via one of the 7 named prefixes or a StringModelConfig with an explicit base_url (model_configs.py:344). Suggest either adding an openai-chat: branch to _infer_single_model returning OpenAICompatibleModel (safe: existing VCR cassettes only record string tool returns, so replay is unaffected), or explicitly documenting this gap in the changelog.

2. No L3 VCR test. tests/AGENTS.md ("Which Layers Do I Need") requires L3 VCR for a bug fix in the model/protocol layer, and lists "No VCR test for model-touching code" as an anti-pattern. Mitigation: the fix is pure client-side serialization and the L1 tests assert the exact payloads, but the most defensible addition is a test that drives the full request() path with a mocked AsyncOpenAI transport and asserts the serialized tool-message content is a list of ChatCompletionContentPartTextParam.

Minor

3. Dead code: create_openai_compatible_model (openai_compatible.py:132-147) is never used — model_helpers.py constructs OpenAICompatibleModel directly. Remove it or use it.

4. Fork-drift risk + untested forked branches. The override re-implements the whole parent method (needed to intercept the ToolReturnPart branch), but the RetryPromptPart, SystemPromptPart (all three roles), and UserPromptPart branches are now forked and untested, and pyproject.toml allows pydantic-ai-slim>=2.12.0. A parity test that feeds identical non-list messages to OpenAIChatModel vs OpenAICompatibleModel and asserts identical output would both close the branch gap and guard against silent divergence on the next lock bump. Worth noting the pinned pydantic-ai version in the module docstring.

5. Behavior scope is broader than the changelog states. _get_openai_based_model is also used by StringModelConfig.get_model() for any identifier with an explicit base_url (model_configs.py:337-348, described there as "the most common pattern for custom/private model providers") — not just the 7 prefixes listed. This is desirable, but worth a line in the changelog.

6. Empty-string list elements (content=["", "x"]) yield {"type":"text","text":""} parts; a template rendering each part as a separate <|tool_return|> block would emit an empty block. Consider filtering empty strings in the comprehension.

7. Nits: the is_multi_modal_content(item) guard in the comprehension (openai_compatible.py:93) is provably dead on that path (the not tool_file_content precondition already excludes files) — isinstance(item, str) alone suffices. Tests use list[Any]/dict[str, Any] (test_openai_compatible_model.py:12,40-41) where precise types are available, per tests/AGENTS.md.

Docs

• No required changes to root AGENTS.md: models/ has no subsystem AGENTS.md and isn't in the Context Loading table (pre-existing; the change is a single new file + one-line class swap, so OpenSpec isn't required either). Changelog entry conforms to changelog/README.md.
• Nice-to-have, pre-existing but the natural moment to fix: docs/how-to/advanced/models.md:43-45 lists only 3 of the 7 affected providers and has a typo (deepsek:deepsek-chat → deepseek:deepseek-chat); docs/explanation/module-structure.md:14 describes models/ as "Pydantic data models and configuration schemas," which now also hosts a model wrapper.

Verified positives

• The fork matches the 2.17.0 parent byte-for-byte outside the new branch; failed-return {"error": ...} wrapping and the trailing file-content user message are preserved.
• Fix applies to both streaming and non-streaming paths (both iterate _map_user_message).
• Edge cases behave correctly: empty list → string fallback, single-element list stays native, dicts/ints → per-element text, denied/interrupted → native list (consistent with pydantic-ai's "sent as ordinary results" semantics), multimodal-in-list and failed returns → parent path.
• No regression risk: tests/conftest.py:237-241 remaps openai: → openai-chat: (stock model, cassettes unchanged), and test_model_capabilities.py:107-109 mocks _get_openai_based_model.

[github-actions]

Posted the synthesized review: #398 (comment)

Summary of findings:

Verified (ran in sandbox against pinned pydantic-ai 2.17.0): 13/13 tests pass, ruff clean, mypy --strict clean on the new module, and the _map_user_message fork matches upstream byte-for-byte outside the new list branch.

Key findings:

• Important: The fix misses openai-chat: — the prefix the repo itself uses for GLM (tests/conftest.py:60), which falls through to the stock OpenAIChatModel at model_helpers.py:156. Issue pydantic-ai OpenAI tool return list content 被序列化为 JSON 字符串，与 OpenAI-compatible 模型（如 GLM-5）的 chat template 不兼容 #112's target model is only fixed via the 7 named prefixes or an explicit base_url.
• Important: No L3 VCR test, which tests/AGENTS.md requires for model-layer bug fixes.
• Minor: dead create_openai_compatible_model factory; fork-drift risk from untested forked branches; changelog omits the StringModelConfig(base_url) blast radius; empty-string elements produce empty text parts.
• Docs: no required changes; flagged pre-existing stale provider list + typo in docs/how-to/advanced/models.md:43-45.

opencode session  |  github run