fix: an exhausted bandwidth allowance is not a retryable rate limit - #18
Merged
Conversation
FMP overloads HTTP 429 for two conditions that need opposite handling, and the
only thing distinguishing them is the response BODY:
{"Error Message": "Limit Reach . Please upgrade..."}
{"Error Message": "Bandwidth Limit Reach . Please upgrade..."}
The first is per-minute pacing and clears in seconds. The second means the
plan's rolling BYTE allowance is spent, and nothing will succeed until the
window rolls over -- typically days.
`_handle_response` discarded the body and raised `FMPRateLimitError` for both,
so an exhausted allowance entered the transparent-retry arm and looped at
5s/10s/20s per request, forever. Observed in a real deployment: a view rebuild
sat in that loop for ten hours, fetching nothing, while every request in it was
guaranteed to fail.
`FMPBandwidthError` is deliberately NOT a subclass of `FMPRateLimitError` --
subclassing would put it straight back inside the `except FMPRateLimitError`
arm, which is the bug. It is also distinct from `FMPBudgetError`, which is the
harvester's own client-side cap rather than a server-side refusal.
An unreadable 429 body falls back to the rate-limit reading: that costs a retry,
where the reverse would turn a transient hiccup into a permanent failure.
Tested for the split rather than just the new class: a bandwidth 429 is
attempted exactly once, a per-minute 429 still gets its full retries, and the
two exception types are asserted not to be related by inheritance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
FMP overloads HTTP 429 for two conditions that need opposite handling, and the only thing distinguishing them is the response body:
{"Error Message": "Limit Reach . Please upgrade..."} {"Error Message": "Bandwidth Limit Reach . Please upgrade..."}The first is per-minute pacing — it clears in seconds and is worth retrying. The second means the plan's rolling byte allowance is spent, and nothing will succeed until the window rolls over, typically days later.
_handle_responsediscarded the body and raisedFMPRateLimitErrorfor both, so an exhausted allowance entered the transparent-retry arm and looped at 5s/10s/20s per request indefinitely.Observed impact
A view rebuild in a real deployment sat in that loop for ten hours, logging
while every request in it was guaranteed to fail. Nothing was fetched, nothing was written, and the only signal was a log line that reads like ordinary throttling.
The change
Reads the body on 429 and raises the new
FMPBandwidthErrorwhen it names a bandwidth limit.Two deliberate decisions worth review:
FMPRateLimitError. Subclassing would put it straight back inside theexcept FMPRateLimitErrorarm of the retry loop — the exact bug being fixed. The existing loop catches onlyTimeoutError,ClientError,FMPRateLimitErrorandFMPServerError, so the new type propagates to the caller, matching the documented intent that "other FMP* exceptions propagate immediately so callers can apply the right per-exception policy."It is also distinct from the existing
FMPBudgetError, which is the harvester's own client-side cap rather than a server-side refusal.Tests
Covers the split, not just the new class:
FMPRateLimitErrorFull tracked suite passes (939 tests);
ruff checkandruff format --checkclean.🤖 Generated with Claude Code