Make the web-search fingerprint configurable and track curl_cffi's newest Chrome - #16
Open
guuse wants to merge 1 commit into
Open
Make the web-search fingerprint configurable and track curl_cffi's newest Chrome#16guuse wants to merge 1 commit into
guuse wants to merge 1 commit into
Conversation
…west
The web-search fallback added in 3.1.5 builds its session with a hardcoded
`impersonate="chrome124"`. Chrome 124 is from April 2024, and Akamai no longer
accepts that fingerprint, so the fallback fails with:
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR
Measured from a residential NL IP on curl_cffi 0.15.0:
chrome124 + HTTP/2 curl (92) INTERNAL_ERROR (0.1s)
chrome124 + HTTP/1.1 curl (28) timeout (25s)
chrome146 + HTTP/2 HTTP 200, payload parses (0.5s)
chrome146 + HTTP/1.1 HTTP 200, payload parses (0.5s)
Only the fingerprint matters. Forcing HTTP/1.1 does not help, and priming
Akamai's cookies by loading funda.nl first is not needed either.
Rather than bump the pin to another release that will age out too, default to
curl_cffi's own `"chrome"` alias, which resolves to its newest Chrome target. A
`pip install -U curl_cffi` then recovers from the next rotation without a
pyfunda release. `Funda(web_impersonate=...)` pins a specific target when
needed. The alias exists across the whole supported curl_cffi>=0.14.0 range
(0.14.0 -> chrome142, 0.15.0 -> chrome146, 0.16.3 -> chrome150), so the
dependency floor does not need raising.
The mobile transport's FINGERPRINT_POOL is deliberately left alone: it mimics
the Funda app rather than a browser, so a newest-Chrome default would be wrong
there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R1DFuoA2djNRuzmWKLi4Sr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The web-search fallback added in 3.1.5 builds its session with a hardcoded
impersonate="chrome124". Chrome 124 is from April 2024 and Akamai no longer accepts that fingerprint, so the fallback fails with:Since the mobile search API currently returns
401 no token provided(#15), that leaves search with no working path at all.Measurements
From a residential NL IP, curl_cffi 0.15.0:
curl (92) INTERNAL_ERROR— 0.1scurl (28) timeout— 25sOnly the fingerprint matters. Forcing HTTP/1.1 doesn't help, and priming Akamai's cookies by loading
funda.nlfirst isn't needed either.Fix
Default to curl_cffi's own
"chrome"alias, which resolves to its newest Chrome target, instead of pinning a release that will age out again. The next Akamai rotation then only needs apip install -U curl_cffi, with no pyfunda release:The alias exists across the whole supported
curl_cffi>=0.14.0range, so the dependency floor doesn't need raising:"chrome"resolves toNotes
FINGERPRINT_POOLin_transport.pyis deliberately untouched. It mimics the Funda app rather than a browser, so a newest-Chrome default would be wrong there.web_impersonatehas a default and is the last init field, so existing positional and keyword calls are unaffected. The default behaviour does change (chrome124 → newest), which is the point.tests/test_web_session.py.