fix(agent): classify LLM provider 400s as USER with an actionable detail - #1072
fix(agent): classify LLM provider 400s as USER with an actionable detail#1072ionut-mihalache-uipath wants to merge 1 commit into
Conversation
An LLM gateway 400 was falling into the unclassified branch of `_classify`, so it reached telemetry as HTTP_ERROR / UNKNOWN with `detail` set to the HTTP reason phrase — the two words "Bad Request". That is the largest slice of the fleet's two-word failure messages (PC-5002), and it hides causes the customer can actually fix, such as the `max_tokens=65535` that Agent Builder itself wrote into the model settings. 400 now maps to LLM_PROVIDER_BAD_REQUEST / USER. Where the gateway supplies a first-party ProblemDetails `detail`, that still wins; otherwise the error carries a canned message pointing at the agent's model settings. The provider body is deliberately not read out — it may carry customer PII and is already recorded on the tenant-scoped LLM call span. 404 is left in UNKNOWN on purpose: every LLM-gateway 404 seen in prod over 30 days was a missing or unreachable deployment (BYO relay not connected, Azure DeploymentNotFound, a retired Bedrock model), which is Deployment rather than User, and deserves its own decision. Also fixes a stale file reference in the develop-agent-module skill (`exceptions/licensing.py` → `exceptions/llm.py`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZ5LHC1Xhp1zBRfYnYENwv
There was a problem hiding this comment.
🟢 Approval recommended
The change is well-scoped, avoids provider-body leakage by construction, and is backed by targeted unit + node-level tests that pin the new 400 classification and fallback-detail behavior.
Pull request overview
This PR improves how LLM-gateway HTTP 400 responses are mapped into AgentRuntimeError so customers see an actionable, provider-safe message instead of the unhelpful HTTP reason phrase (“Bad Request”), while keeping other 4xx statuses classified as UNKNOWN per the stated scope.
Changes:
- Classifies HTTP 400 as
LLM_PROVIDER_BAD_REQUESTwithUiPathErrorCategory.USERand a stable canneddetailwhen the gateway provides no ProblemDetailsdetail. - Extends
_classify()to return afallback_detailso only explicitly-handled statuses avoid reason-phrase fallback. - Adds/updates tests to pin 400 behavior (including “no provider body leakage”) and to ensure other 4xx remain
UNKNOWN; updates the agent module skill doc reference.
File summaries
| File | Description |
|---|---|
src/uipath_langchain/agent/exceptions/llm.py |
Adds 400 classification with provider-safe fallback detail and threads fallback_detail through raise_for_provider_http_error(). |
src/uipath_langchain/agent/exceptions/exceptions.py |
Introduces AgentRuntimeErrorCode.LLM_PROVIDER_BAD_REQUEST. |
tests/agent/test_llm.py |
Expands contract tests for 400 mapping, detail precedence, and “no provider body quoted back”; pins unclassified 4xx behavior. |
tests/agent/react/test_llm_node.py |
Adds end-to-end node-level assertions for 400 mapping for both new client and legacy OpenAI exception paths. |
.claude/skills/develop-agent-module/SKILL.md |
Fixes the referenced exceptions module path for LLM provider errors. |
Review details
- Files reviewed: 5/5 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|



Problem
An LLM gateway
400fell into the unclassified branch of_classify, so it surfaced asHTTP_ERROR/UNKNOWNwithdetailset to the HTTP reason phrase — the two words "Bad Request".That is the single largest contributor to the fleet's two-word failure messages (PC-5002), and it buries causes the customer can actually act on. The motivating case: a job that failed 192/192 runs on
gpt-4.1-mini-e2e-custombecausemax_tokens=65535exceeded the model's 32768 completion-token limit — a value Agent Builder itself had written into the model settings. The customer saw "Bad Request", categorized Unknown.Change
400now maps toLLM_PROVIDER_BAD_REQUEST/UiPathErrorCategory.USER, with a title of "LLM provider rejected the request".For
detail, precedence is:detail— first-party UiPath text, and more specific, so it still wins.The provider body is deliberately not read out into the error. It may carry customer PII, and it is already recorded on the LLM call span, which is tenant-scoped.
USERis not in_SHOULD_WRAP_CATEGORIES, so the canned detail has to stand on its own — tests pin that it isn't prefixed with the generic wrapper._classifynow returns a fourth element,fallback_detail.Nonemeans "fall back to the HTTP reason phrase", so only statuses whose cause we cannot name are left with it.What is deliberately not in scope
404staysUNKNOWN. Every LLM-gateway 404 observed in prod over 30 days was a missing or unreachable model deployment — BYO relay not connected, AzureDeploymentNotFound, a retired Bedrock model — which is Deployment, not User. It needs its own decision on its own evidence rather than being folded into this change.test_unclassified_4xx_remains_unknowndocuments that and covers 404/408/413/422/429.Tests
detail,str(error)andrepr(error).detailtakes precedence over the canned text.UiPathAPIError) and the legacy rawopenai.BadRequestErrorpath.uv run pytest tests/agent/test_llm.py tests/agent/react/test_llm_node.py— 54 passed. ruff check, ruff format, the httpx-client AST linter, and mypy all clean.Drive-by
Fixed a stale file reference in the
develop-agent-moduleskill:exceptions/licensing.py→exceptions/llm.py.🤖 Generated with Claude Code
https://claude.ai/code/session_01UZ5LHC1Xhp1zBRfYnYENwv