Skip to content

fix(adapters/langgraph): honour the declared output format, not its shape - #42

Merged
spichen merged 1 commit into
spichen:oi-forkfrom
MAabdelmoumen:fix/OA-1681-json-single-field
Sep 22, 2026
Merged

spichen merged 1 commit into
spichen:oi-forkfrom
MAabdelmoumen:fix/OA-1681-json-single-field

Conversation

@MAabdelmoumen

Copy link
Copy Markdown

Fixes the case where an agent that asks for JSON output gets prose instead.

The problem

is_single_string_output treats one string output as the agent's free text and skips structured generation. It cannot distinguish two different intents, because both produce an identical outputs list:

text agent              outputs: [{ title: "output",    type: "string" }]
JSON, one string field  outputs: [{ title: "sentiment", type: "string" }]

So a caller declaring a one-field JSON schema gets no structured-output tool in the request at all — the model is never told a schema was wanted, and answers in prose. Adding a second field changes nothing about the model, endpoint or prompt, yet works.

Verified against a vLLM deployment (nvidia/GLM-5.3-NVFP4) which supports json_schema, json_object and forced tool calls:

schema tool in request result
1 field, String none prose
1 field, Enum none prose
2 fields AgentOutputModel JSON, both fields

In a flow it is worse than cosmetic: the declared property is absent, so the downstream node raises Expected node ... to have a value for property ..., but none was found and the flow dies.

The change

Gate on the caller's declared format when one is given. metadata["outputFormat"] carries TEXT or JSON.

When no format is declared the previous shape heuristic still applies, so existing callers keep today's behaviour.

Why weak models stay safe

The heuristic exists so a string output works on models without structured-output support. That protection is unchanged: a single string output still satisfies is_single_string_output, so extract_outputs_from_invoke_result still falls back to the final message when structured generation returns nothing. The worst case is the prose returned today — never an empty output.

Note that oracle/agent-spec main has no such heuristic; it always builds the response model and forces ToolStrategy. This moves the fork back toward that behaviour for callers that declare a format.

Refs OA-1681

…hape

A single string output is treated as the agent's free text and skips
structured generation. That heuristic cannot tell two different things
apart, because both produce the identical outputs list:

  text agent            outputs: [{title: "output",    type: "string"}]
  JSON, one string field outputs: [{title: "sentiment", type: "string"}]

A caller that declares one JSON field therefore gets prose, with no tool
registered and nothing in the request telling the model a schema was
wanted. Adding a second field changes nothing about the model or the
prompt, yet works, which is what makes the failure so hard to place.

Gate on the caller's declared format when there is one. `metadata`
carries `outputFormat` (`TEXT` or `JSON`); when it is absent the previous
shape heuristic still applies, so callers that declare no format keep
today's behaviour.

Weak models stay protected: a single string output still satisfies
is_single_string_output, so extract_outputs_from_invoke_result continues
to fall back to the final message when structured generation returns
nothing. The worst case is the prose that is returned today, never an
empty output.

Refs OA-1681
@MAabdelmoumen

Copy link
Copy Markdown
Author

Closing — wrong base. This was branched from c83e113 (the commit agentfabric pins), which is not an ancestor of main, so the diff swept in unrelated commits. Reopening against the correct base shortly.

@MAabdelmoumen MAabdelmoumen reopened this Sep 22, 2026
@MAabdelmoumen
MAabdelmoumen changed the base branch from main to oi-fork September 22, 2026 11:06
@MAabdelmoumen MAabdelmoumen reopened this Sep 22, 2026
@spichen
spichen merged commit f111e7a into spichen:oi-fork Sep 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants