fix(py/plugins/openai): emit content, reasoning, and tool-call deltas that share a chunk - #6282
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the OpenAI model implementation to support emitting and converting reasoning content, text content, and tool calls together in a single stream chunk or message, rather than treating them as mutually exclusive. Previously, tool calls took precedence over reasoning and text content; the changes ensure all three parts are preserved and ordered correctly (reasoning, then text, then tool calls). Unit tests have been added and updated to verify these streaming and conversion behaviors. I have no feedback to provide as there are no review comments.
|
One thing worth being explicit about before I approve. From what I can see this brings Python in line with Go (go/plugins/compat_oai/generate.go:827-845 non-stream, :552-570 stream), which already emits reasoning, text and tool calls together, and #6233 says as much. JS Small knock-on: on the non-stream path |
b83bd33 to
2546985
Compare
2546985 to
1beae60
Compare
|
Yeah, a JS issue can follow. On the DeepSeek json-mode + tool call part comment, I ran it: ai.generate with an output_schema and a tool against deepseek-chat in json mode, transport mocked so round 1 returns fenced JSON content and a tool call in one message. round 1 model message: [TextPart('{"city": "NYC", "temp_c": 0}'), ToolRequestPart(get_weather)] get_weather runs round 2: .output = Weather(city='NYC', temp_c=7), finish_reason=stop Fences stripped, the tool part carried through untouched, raw and custom kept. Pinned in TestCleanJsonResponse.test_keeps_tool_parts_beside_cleaned_text. I found out that to_openai splits a mixed model message into two assistant messages, one with content and one with tool_calls, where Go builds a single assistant message carrying both (generate.go:87-103). It was unreachable before this PR because to_genkit returned either tool calls or text, never both. I have not seen a provider reject it, so I would rather file it than grow this PR. |
1beae60 to
535f759
Compare
535f759 to
556f708
Compare
cabljac
left a comment
There was a problem hiding this comment.
Nice, this is the right shape: three independent ifs, one callback per chunk, and a single order that to_genkit and the stream loop now share. The if parts: guard is the detail I like most. Refusal-only and heartbeat deltas emitted nothing under the old if/elif chain by accident, and a naive rewrite would have started sending empty chunks; you kept it deliberately.
I ran the suite on the branch (257 pass) and broke each behaviour the PR claims to check the tests bite. They all do, including the ordering and the null-fragment case.
One thing worth digging into on the null-arguments thread, then two inline.
556f708 to
d9642de
Compare
d9642de to
7275c15
Compare
… that share a chunk
…ext, tool-call order
7275c15 to
c87b4ea
Compare
Closes #6233
Summary
The streaming loop now reads content, reasoning and tool-call deltas independently and appends them to a single outgoing chunk, in reasoning, text, tool-call order. The chunk is emitted only when it carries something. A tool-call fragment that shares a chunk with text reaches the accumulator, so the aggregate response is complete as well.
MessageConverter.to_genkitgets the same treatment on the non-streaming path: reasoning, text, then one part per tool call, rather than tool calls to the exclusion of both.A text delta converts straight to a text part instead of going through
to_genkit, which would otherwise return the reasoning and tool-call parts riding on the same delta a second time.