fix(server-utils): Deduplicate Google GenAI streaming tool calls - #23432
andreiborza merged 4 commits into
Conversation
The streaming handler recorded every tool call twice: once from `chunk.functionCalls` and again from the `functionCall` parts of each candidate. `functionCalls` is an SDK getter over those same parts, so one real tool call produced two span entries with mismatched shapes (one keyed by the non-spec `args`, one by `arguments`). Take tool calls only from `chunk.functionCalls`, the same source the non-streaming path uses, so each call is recorded once in one shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
8 similar comments
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
|
👋 @logaretm, @stephanie-anderson — Please review this PR when you get a chance! |
Co-Authored-By: GPT-6 <codex@openai.com>
andreiborza
left a comment
There was a problem hiding this comment.
Hi @zkasuran, thanks for fixing this! Sorry for the late reply. I just made a couple of small changes, but LGTM!
Co-Authored-By: GPT-6 <codex@openai.com>
This PR adds the external contributor to the CHANGELOG.md file, so that they are credited for their contribution. See #23432 Co-authored-by: andreiborza <168741329+andreiborza@users.noreply.github.com>
The streaming instrumentation for
@google/genairecords every tool call twice.handleCandidateContentinpackages/server-utils/src/ai/google-genai/streaming.tspushes tool calls to the span from two sources on the same chunk:chunk.functionCalls, spread straight intostate.toolCallspart.functionCall, mapped again while iteratingcandidate.content.partschunk.functionCallsis a getter on the@google/genairesponse that is itself derived from those same candidate parts (it filterspartsforfunctionCallthen maps them). So a single real tool call lands ingen_ai.response.tool_callsas two entries. The two do not even share a schema: the getter keeps the SDK-native{ id, name, args }while the parts loop emits{ type, id, name, arguments }. One value ends up keyedargs, the otherarguments.The non-streaming path (
addResponseAttributes) reads tool calls fromresponse.functionCallsand records one entry per call, so streaming and non-streaming disagreed on both count and shape.Fix: take streaming tool calls only from
chunk.functionCalls, the same accessor the non-streaming path uses, then drop the second push. Each call is now recorded once, in one shape. The streaming output matches the non-streaming output.I kept
chunk.functionCallsrather than the parts loop so both code paths share one source of truth and emit the SDK-native shape. This mirrors the OpenAI and Anthropic integrations, whose streaming paths reconstruct the exact tool-call shape their non-streaming paths produce. The deprecatedgen_ai.response.tool_callsexample shows{ name, arguments }, but no provider integration normalizes to that literally (Anthropic keepsinput, OpenAI nests underfunction), so consistency between a provider's streaming and non-streaming output was the stronger property to preserve here.Root cause:
chunk.functionCallsandpart.functionCallare two views of the same data. Both were being written to the span.Verified against a real gemini-3.6-flash streaming response that returns one
controlLighttool call (response idIuF-auTiDpW2g8UPnNDKsAI). The identical response was replayed through the instrumentation before and after the change.before (
gen_ai.response.tool_calls):[ { "id": "call_2079699", "args": { "colorTemperature": "warm", "brightness": 30 }, "name": "controlLight" }, { "type": "function", "id": "call_2079699", "name": "controlLight", "arguments": { "colorTemperature": "warm", "brightness": 30 } } ]after:
[ { "id": "call_2079699", "args": { "colorTemperature": "warm", "brightness": 30 }, "name": "controlLight" } ]A unit test in
packages/server-utils/test/ai/lib/tracing/google-genai-streaming.test.tscovers the single-call case, multiple calls across chunks and therecordOutputs: falsecase, then asserts the non-streaming path still records one entry in the same shape. It fails ondevelop(two entries) and passes with this change.yarn lint) & (yarn test).AI assistance (Claude, Anthropic) was used in developing this change. The design, review and verification were done by the author. Verified locally before submitting: the new and existing
@sentry/server-utilsunit tests (377 passing),oxlint,oxfmt --checkand the TypeScript type-check all pass, plus a real gemini-3.6-flash streaming run captured before and after.