Refactor metrics recording to invocations and add tool/workflow duration metrics - #615
Open
lmolkova wants to merge 21 commits into
Open
Refactor metrics recording to invocations and add tool/workflow duration metrics#615lmolkova wants to merge 21 commits into
lmolkova wants to merge 21 commits into
Conversation
lmolkova
added a commit
that referenced
this pull request
Sep 4, 2026
Pull request dashboard statusWaiting on maintainers · refreshed 2026-09-04 23:17 UTC Resolve merge conflicts, then merge when ready. Status above doesn't look right?
|
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
It removes public metric APIs without deprecation and repeats histogram creation in the streaming hot path.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Refactors GenAI metric recording into invocation classes and introduces semconv-aligned tool duration metrics.
Changes:
- Replaces
InvocationMetricsRecorderusage with directMeteraccess. - Adds dedicated tool metrics and aligns workflow metric attributes/buckets.
- Updates metric tests and changelog fragments.
File summaries
| File | Description |
|---|---|
.changelog/615.added |
Documents metric changes. |
.changelog/615.removed |
Documents removed helpers. |
_agent_invocation.py |
Uses shared client metric method. |
_embedding_invocation.py |
Uses shared client metric method. |
_fetch_response_invocation.py |
Uses shared client metric method. |
_inference_invocation.py |
Passes Meter directly. |
_invocation.py |
Creates and records client metrics. |
_retrieval_invocation.py |
Uses shared client metric method. |
_tool_invocation.py |
Adds tool duration recording. |
_workflow_invocation.py |
Aligns workflow duration recording. |
handler.py |
Passes Meter to invocations. |
instruments.py |
Removes histogram factories. |
metrics.py |
Removes the metrics recorder. |
test_handler_metrics.py |
Updates tool metric assertions. |
test_handler_workflow.py |
Updates workflow attributes. |
Review details
Suppressed comments (2)
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py:1
- Deleting this module removes
InvocationMetricsRecorder, which was exposed via__all__from a non-private module. Existing consumers importing it will fail immediately; the util compatibility policy requires public objects to be deprecated before removal (.github/instructions/util-genai.instructions.md:36-40). Keep a compatibility module/object with a deprecation note for a release cycle, or defer removal to a breaking release.
util/opentelemetry-util-genai/src/opentelemetry/util/genai/instruments.py:1 - These helpers are public imports from a non-private module, and repository release notes explicitly advertised
create_duration_histogramandcreate_token_histogramfor reuse (instrumentation/opentelemetry-instrumentation-genai-openai/CHANGELOG.md:114-116). Removing the module breaks downstream imports; retain deprecated compatibility wrappers first or remove them only in a breaking release.
- Files reviewed: 15/15 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…system messages in input_messages
- Align GenericPart with semconv by removing the value field - Extract non-text Bedrock system content blocks as GenericPart - Check system is None in Anthropic message extractors - Retain list[MessagePart] in CompletionHook signature - Add tests for GenericPart serialization and system instruction span attributes
lmolkova
added a commit
to lmolkova/opentelemetry-python-genai
that referenced
this pull request
Sep 4, 2026
…ResponseInvocation
lmolkova
added a commit
that referenced
this pull request
Sep 4, 2026
lmolkova
force-pushed
the
lmolkova-metrics-refactor
branch
from
September 4, 2026 05:56
7e63ff9 to
93acaa8
Compare
lmolkova
force-pushed
the
lmolkova-metrics-refactor
branch
from
September 4, 2026 06:09
93acaa8 to
b23f3f7
Compare
lmolkova
marked this pull request as ready for review
September 4, 2026 06:17
DylanRussell
approved these changes
Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change do?
Refactors metric recording directly into invocation classes (removing
InvocationMetricsRecorder,instruments.py, andmetrics.py), and adds duration metrics for tool and workflow invocations.Semantic convention changes
Added metrics:
gen_ai.execute_tool.duration(s): Measures tool execution duration.gen_ai.tool.name,gen_ai.tool.type,gen_ai.agent.name,error.type.gen_ai.invoke_workflow.duration(s): Measures workflow execution duration.gen_ai.workflow.name,error.type.Changed metric attributes & buckets:
gen_ai.invoke_workflow.duration: Updated advisory bucket boundaries to workflow-specific scale ([1, 5, ... 7200]s).gen_ai.operation.namefrom tool and workflow duration metrics in compliance with the semconv metrics spec.Why?
Aligns tool and workflow metrics with GenAI semantic conventions (metrics spec) and removes an unnecessary indirection layer.