Skip to content

[opentelemetry-instrumentation-genai-openai-agents] Record gen_ai.tool.call.arguments on execute_tool spans - #588

Merged
lmolkova merged 3 commits into
open-telemetry:mainfrom
sfc-gh-zeningchen:fix/ob-66104-tool-call-arguments
Sep 3, 2026
Merged

[opentelemetry-instrumentation-genai-openai-agents] Record gen_ai.tool.call.arguments on execute_tool spans#588
lmolkova merged 3 commits into
open-telemetry:mainfrom
sfc-gh-zeningchen:fix/ob-66104-tool-call-arguments

Conversation

@sfc-gh-zeningchen

@sfc-gh-zeningchen sfc-gh-zeningchen commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Description

execute_tool spans never carried gen_ai.tool.call.arguments, even with
content capture enabled. The processor read FunctionSpanData.input in
on_span_start, but the agents library assigns it inside the
with function_span(...) block, so the read always saw None. This reads it in
on_span_end instead, next to the existing output read, and deserializes it
to the object semconv specifies.

input stays None when a run sets RunConfig.trace_include_sensitive_data=False,
so the attribute is absent for those runs. A string that does not parse to a
JSON object is recorded as-is.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How has this been tested?

New test_runner_records_tool_call_arguments drives Runner.run against a
cassette and asserts the attribute on the exported span. New parametrized
test_tool_arguments_value_shapes covers the argument string shapes providers
produce, plus the None and empty cases. Three existing tests pre-set
FunctionSpanData.input at construction, which the real SDK never does and
which is why they passed while the attribute was missing; they now assign it
after on_span_start.

  • openai-agents suite: 30 passed at the declared floor
    (openai-agents==0.3.3) and at latest (openai-agents==0.22.0)
  • py314-test-instrumentation-genai-openai_agents-conformance: 1 passed
  • tox -e precommit (ruff, ruff-format, rstcheck) and pyright clean

Checklist

  • Followed the style guidelines of this project
  • Changelog updated if the change requires an entry
  • Unit tests added
  • Documentation updated

@sfc-gh-zeningchen
sfc-gh-zeningchen requested a review from a team as a code owner September 3, 2026 06:42
Copilot AI lite review requested due to automatic review settings September 3, 2026 06:42
@linux-foundation-easycla

linux-foundation-easycla Bot commented Sep 3, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: sfc-gh-zeningchen / name: Zening Chen (2e1dd59)

@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Sep 3, 2026

Copy link
Copy Markdown

Pull request dashboard status

Merged · refreshed 2026-09-03 23:16 UTC

Status above doesn't look right?
  • Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

…tool.call.arguments` on execute_tool spans

The processor read `FunctionSpanData.input` in `on_span_start`, but the
agents library assigns it inside the `with function_span(...)` block, so
the read always saw `None`. Read it in `on_span_end` instead, next to the
existing `output` read, and deserialize it to an object.

Assisted-by: Claude Opus 5
@sfc-gh-zeningchen
sfc-gh-zeningchen force-pushed the fix/ob-66104-tool-call-arguments branch from 8b0f2f9 to 2e1dd59 Compare September 3, 2026 06:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The processor currently performs JSON parsing / result assignment even when span content capture is disabled, creating avoidable overhead on every tool invocation.

Pull request overview

This PR updates the OpenAI Agents GenAI instrumentation to reliably record gen_ai.tool.call.arguments on execute_tool spans by reading tool input at span end (when the agents library has populated it) and normalizing JSON-object argument strings to a consistent representation.

Changes:

  • Move tool input (FunctionSpanData.input) handling from on_span_start to on_span_end to match the agents library’s lifecycle.
  • Normalize tool argument values by JSON-deserializing object-shaped inputs (while preserving non-object / non-JSON inputs as strings).
  • Add/adjust unit and VCR-backed integration tests validating span-end capture and argument-shape behavior.
File summaries
File Description
instrumentation/opentelemetry-instrumentation-genai-openai-agents/src/opentelemetry/instrumentation/genai/openai_agents/processor.py Reads tool input at span end and normalizes JSON-object argument strings before assigning to ToolInvocation.arguments.
instrumentation/opentelemetry-instrumentation-genai-openai-agents/tests/test_processor.py Updates existing unit test to reflect span-end population and adds new tests for capture timing and argument-shape normalization.
instrumentation/opentelemetry-instrumentation-genai-openai-agents/tests/test_instrumentor.py Adds a VCR-backed runner test asserting gen_ai.tool.call.arguments and result are exported on the execute_tool span.
instrumentation/opentelemetry-instrumentation-genai-openai-agents/tests/cassettes/test_runner_records_tool_call_arguments.yaml Adds cassette data for the new integration test.
instrumentation/opentelemetry-instrumentation-genai-openai-agents/.changelog/588.fixed Adds a changelog fragment documenting the fix.
Review details

Suppressed comments (1)

instrumentation/opentelemetry-instrumentation-genai-openai-agents/src/opentelemetry/instrumentation/genai/openai_agents/processor.py:148

  • on_span_end deserializes tool arguments and assigns tool_result even when content capture on spans is disabled, which adds avoidable overhead (JSON parsing/stringification) for every tool call. Since ToolInvocation already snapshots should_capture_content_on_span, gate the expensive work on that flag.
            arguments = span.span_data.input
            if arguments:
                invocation.arguments = _tool_arguments(arguments)
            output = span.span_data.output
            if output is not None:
                invocation.tool_result = (
                    output if isinstance(output, str) else str(output)
                )
  • Files reviewed: 5/5 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

util-genai recommends checking the flag in the instrumentation before
assigning `arguments` and `tool_result`, since both can be expensive to
serialize. `_apply_finish` checks it again before writing the attributes,
so the exported spans are unchanged.

Assisted-by: Claude Opus 5
if (
isinstance(invocation, ToolInvocation)
and isinstance(span.span_data, FunctionSpanData)
and invocation.should_capture_content_on_span

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JFYI - we should probably switch to more generic check, but let's get this one in and I'll update #593

@lmolkova
lmolkova added this pull request to the merge queue Sep 3, 2026
Merged via the queue into open-telemetry:main with commit f157a49 Sep 3, 2026
73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants