Skip to content

Identify the caller by counting proxy hops, not by trusting the first header entry - #337

Open
crtahlin wants to merge 1 commit into
devfrom
fix/trusted-proxy-hops
Open

Identify the caller by counting proxy hops, not by trusting the first header entry#337
crtahlin wants to merge 1 commit into
devfrom
fix/trusted-proxy-hops

Conversation

@crtahlin

Copy link
Copy Markdown

Closes #333.

What was wrong

Three controls key on the caller's address: the global per-IP rate limit, the x402 free-tier rate limit, and the daily spend budget (#102). A caller who can choose that value is not limited by any of them.

Both copies of get_client_ip took the first entry of X-Forwarded-For:

return forwarded_for.split(",")[0].strip()

X-Forwarded-For is a list appended to by each proxy it passes through, so the leftmost entry is the one furthest from us — and it is whatever the client sent if any proxy appends rather than replaces. A caller rotating the header would get a fresh rate-limit window and a fresh spend budget on every request. That is the same failure the pool allowance had when every distinct Origin minted a new budget.

It was not exploitable, which is why this is worth doing now rather than urgently

Verified against staging on 2026-09-09 — free-tier limit exhausted, then retried with three forged values:

1..3          -> 422   (through)
4..5          -> 429   (limit reached)
XFF=9.9.9.9   -> 429
XFF=8.8.8.8   -> 429
XFF=7.7.7.7   -> 429

Caddy replaces the header rather than appending, so nothing got through. But that safety lived entirely in the proxy's configuration: nothing in this repository recorded the dependency, and no test pinned it. One trusted_proxies line, a CDN in front, or a load balancer that appends would have made all three controls bypassable, and nothing would have failed — the limits would just stop limiting.

The fix

Count back from the right by TRUSTED_PROXY_HOPS (default 1). The rightmost entry is the one our own proxy added, and is the only entry we know to be truthful.

header "1.2.3.4"                    (proxy replaced)  -> 1.2.3.4
header "9.9.9.9, 1.2.3.4"           (proxy appended)  -> 1.2.3.4
header "9.9.9.9, 8.8.8.8, 1.2.3.4"  (forged chain)    -> 1.2.3.4

Correct either way, which is the property worth having — it stops depending on behaviour nobody here controls.

A chain shorter than the hop count falls back to the direct peer rather than clamping to index 0. Clamping is the obvious implementation and it would hand back the leftmost entry — exactly the value an attacker controls — so the naive fix reintroduces the flaw in the misconfigured case. A test asserts the fallback never returns a header value.

0 hops ignores forwarding headers entirely rather than half-trusting them, for a directly exposed deployment.

One implementation

app/core/client_ip.py, re-exported from both previous homes so the five calling modules (rate_limit, x402/middleware, x402/dependency, stamps, chunks, stamps_for_owner) are untouched. A test asserts they are all the same object — two copies existed and both had the same flaw, so divergence means one control gets fixed and another silently does not.

Tests

19 new, 100% of the new module, 1184 passed, 25 skipped overall.

The ones that matter: a forged prefix cannot change the answer at one or two hops; rotating the header twenty times yields one identity rather than twenty; a replacing proxy and an appending proxy agree; the short-chain fallback never returns a header value; and — because a correct helper nothing calls would fix nothing — the spend budget is actually charged to the real caller rather than the forged prefix, driven through the live endpoint.

test_forwarded_for_header asserted the old behaviour, so it now asserts the new one with the reasoning in its docstring. It encoded the flaw.

Deployment

deploy.yml writes TRUSTED_PROXY_HOPS=1 for both environments, documented in .env.example.

The Caddyfile now records the dependency next to the existing X-Forwarded-For note, on both sites, because that file is where someone would break it: adding a proxy in front without raising the hop count makes every request share one budget, and lowering it below the real number lets callers pick their own identity.

… header entry

Three controls key on the client address: the global per-IP rate limit, the x402
free-tier rate limit, and the daily spend budget. A caller who can choose that
value is not limited by any of them.

Both copies of get_client_ip took the FIRST entry of X-Forwarded-For. That is
the entry furthest from us, and it is whatever the client sent if any proxy in
the chain appends rather than replaces. A caller rotating the header would get a
fresh rate-limit window and a fresh spend budget on every request — the same
failure the pool allowance had when every distinct Origin minted a new budget.

It was NOT exploitable. Verified against staging on 2026-09-09: the free-tier
limit was exhausted, then retried with three forged values, all still refused,
because Caddy replaces the header rather than appending. But that safety lived
entirely in the proxy's configuration, nothing in this repository recorded the
dependency, and no test pinned it. Setting trusted_proxies, adding a CDN, or
moving to a load balancer that appends would have made all three controls
bypassable with nothing failing to say so.

Now counts back from the RIGHT by TRUSTED_PROXY_HOPS, default 1. The rightmost
entry is the one our own proxy added and is the only entry we know to be
truthful. This is correct whether the proxy replaces or appends, which is the
property worth having: it stops depending on behaviour nobody here controls.

A chain shorter than the configured hop count falls back to the direct peer
rather than clamping to index 0. Clamping would hand back the leftmost entry —
exactly the value an attacker controls — so the obvious implementation of the
fix would have reintroduced the flaw in the misconfigured case. A test asserts
the fallback never returns a header value.

0 hops ignores forwarding headers entirely rather than half-trusting them, for
a directly exposed deployment.

One implementation, in app/core/client_ip.py, re-exported from both previous
homes so the five calling modules are untouched. A test asserts they are all the
same object: two copies existed and both had the same flaw, so divergence means
one control gets fixed and another silently does not.

The Caddyfile now records the dependency next to the X-Forwarded-For note, on
both sites, because that file is where someone would break it.

test_forwarded_for_header asserted the old behaviour and now asserts the new,
with the reasoning in its docstring — it encoded the flaw.

19 new tests, 100% of the new module. 1184 passed, 25 skipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant