Skip to content

fix(provider): 5xx 视为容量拒绝——快速切换成员并冷却死通道 - #392

Merged
tall-1997 merged 2 commits into
mainfrom
fix/relay-5xx-capacity-switch
Nov 18, 2026
Merged

tall-1997 merged 2 commits into
mainfrom
fix/relay-5xx-capacity-switch

Conversation

@tall-1997

Copy link
Copy Markdown
Owner

根因(实测,非推断)

针对报告里的中转站 token.zlshag.xyz 做了 4 组对照实验:

实验 结果 说明
4 种请求体(裸 / +system / +tools / +tools+system+stream_options) 同样的瞬时 503,<1s,body 完全一致 与请求内容无关 → 是通道判定
同 key 不同模型 glm-5.3 3/3 成功;deepseek-v4-pro、deepseek-v4.1-flash 各 1/3 通道独立失效,不是 key 失效
顺序 12 次 12/12 成功 单发稳定,问题不在请求构造
并发突发(2/3/4/5/8) 瞬时 429/502/503,成功率 50% 左右 中转站按通道/密钥做即时准入拒绝
448KB / 147k tokens 大上下文 200 CLEAN(含 usage、[DONE]) 不是体积或上下文超限

结论:503 来自中转站自身的通道准入/上游冷却({"error":{"message":"Service temporarily unavailable","type":"api_error"}},0.4–1.2s 内返回,无上游尝试),不是 app 的请求或网络。glm 通道当时健康、deepseek 通道整段不可用。

app 侧为什么会放大成「老是断开」

  1. mapHttpErrorShared 把 5xx 归为 TransientError(TRANSIENT_HTTP_STATUSES),但 isFallbackable 只认 ProviderError 的 [5xx] → 503 永不触发组内回退。
  2. 于是同一次失败在同一个模型上烧掉 5 次重试(1/2/4/8/16s ≈ 31s),才把错误抛给用户;组里明明有健康成员也不用。
  3. 失败后不给该成员打冷却 → 下一轮又选中同一个死通道,表现为「老是断开」的循环。
  4. 同时 ChatViewModel.HTTP_5XX_STATUS_RE 与 LLMError 各有一份状态码正则,判定逻辑三处各写一遍。

修复

  • LLMError 新增 httpStatus(统一解析 [NNN] 令牌,TransientError / ProviderError / RateLimited 共用)与 isCapacityRejection;isFallbackable 改为经它判定,5xx 的 TransientError 也能回退。看门狗 stall 不带状态码,resume 语义([T-stall-resume])不受影响。
  • Agent loop:容量类 5xx 在「组内还有候选」时预算降为 1 次(与既有 429 策略一致),把时间花在切换而不是空转 31s。
  • Agent loop:容量类 5xx 触发 markRoutingCooldownFor(默认 30s,有 Retry-After 则以其为准),下一轮路由绕开死通道。
  • 移除 ChatViewModel.HTTP_5XX_STATUS_RE,只留单一解析入口。

测试

LLMErrorClassificationTest 新增:状态码解析(Transient/Provider/RateLimited/无码)、5xx→可回退、429 与看门狗 stall 的边界、ProviderError 5xx 行为保持。本地沙箱无 Android 工具链,编译与单测交由 CI。

…s fast

Root cause measured against token.zlshag.xyz (the relay in the report),
not inferred:

* Every payload shape (plain / +system / +tools / +tools+system+
  stream_options) gets the SAME instant 503
  {"error":{"message":"Service temporarily unavailable","type":"api_error"}}
  in <1s, so the status is a channel verdict, not a request complaint.
* glm-5.3 answered 3/3 while deepseek-v4-pro and deepseek-v4.1-flash
  answered 1/3 each on the same key — channels fail independently.
* Sequential 12/12 succeeded; parallel bursts returned 429/502/503
  instantly, so the relay also admits ~1 concurrent per channel.
* A 448KB / 147k-token prompt returned 200 CLEAN, so neither body size
  nor context length triggers it.

The app made that relay behaviour expensive: a 5xx maps to TransientError
(mapHttpErrorShared's TRANSIENT_HTTP_STATUSES), isRetryable was true but
isFallbackable was false, so the loop burned 5 same-provider retries
(1/2/4/8/16s ≈ 31s) and then failed the turn even when the group had a
healthy member, and the dead entry was never parked — the next turn
re-picked it. That is the "Transient error: [503] ... 老是断开" loop.

- LLMError gains `httpStatus` (one parser for the bracketed status the
  mappers stamp into `detail`) and `isCapacityRejection`; `isFallbackable`
  now routes TransientError 5xx and ProviderError 5xx/429 through them.
  Watchdog stalls carry no status token and keep their resume semantics.
- Agent loop: a capacity 5xx with another group member gets the same
  one-attempt budget 429 already had, so the turn switches instead of
  hammering a dead channel for 31s.
- Agent loop: capacity 5xx parks the member (markRoutingCooldownFor,
  30s default, Retry-After wins), so the next turn routes around it.
- ChatViewModel.HTTP_5XX_STATUS_RE removed — duplicate of the new parser.
- Tests pin status parsing, capacity classification, fallback, and the
  "stall is not a capacity rejection" boundary.
@tall-1997
tall-1997 merged commit b8b23fe into main Nov 18, 2026
1 check passed
@tall-1997
tall-1997 deleted the fix/relay-5xx-capacity-switch branch November 18, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant