Fix Ethereum FDV and stop the Beacon fetch from failing - #679
Merged
Conversation
FDV is supply times price and has no dependency on the validator set, but getTvs aborted the whole snapshot whenever the Beacon request came back empty. The supply had already been fetched successfully from Etherscan and CoinGecko at that point and was discarded along with the empty node list, so update-chain-tvs never wrote total_supply and update-fdv kept skipping on its supply <= 0 guard. Ethereum FDV stayed at 0 in the networks table. The Beacon endpoint fails for mainnet because its active-validator response exceeds the V8 string limit (Cannot create a string longer than 0x1fffffe8 characters), so this is the steady state rather than a transient blip. Staking figures now degrade to zero when the validator set is unavailable while supply is still published. Bonded stake exceeding total supply is treated the same way, so a ratio above 1 can never reach the database. The only remaining abort is a missing supply, where returning null correctly preserves the previous snapshot. Verified end to end against a local database: chain-tvls wrote 120682557074402769230357351 wei and update-fdv computed 231410010015.74 USD at a price of 1917.51.
The mainnet validator endpoint returns ~1 GB in a single response, so res.json() could never materialise it and threw "Cannot create a string longer than 0x1fffffe8 characters" — the V8 string limit is 512 MB. Three retries burned 313 seconds every hour and always ended with an empty list, which is why Ethereum had no validators, no TVS and no APR. Server-side filtering does not help: ?status=active and ?status=active_ongoing both return the full set byte for byte (1059431887 vs 1059448434), so the node ignores the parameter. Of the 2.32M entries returned, 1.43M are withdrawal_done and only ~890k are active. Fold the response incrementally with stream-json instead. Memory stays flat — 15 MB heap and 156 MB RSS over the full gigabyte — and a live run now completes in 107 seconds where it previously spent 313 seconds failing. Mainnet no longer indexes validators one by one. jobs/get-nodes.ts issues database queries per node, so a successful parse would mean ~890k round trips and rows for a single chain; aggregate stake is streamed instead. Testnets keep the per-node path, their sets being small enough to index. Live verification: 2322293 validators streamed, 892032 bonded, bonded stake 41452485.77 ETH against a supply of 120682557.07 ETH, TVS 34.35%. Note for review: the bonded figure follows the pre-existing definition in this code (active* plus withdrawal_possible, using current rather than effective balances). It could not be cross-checked against a public source — beaconcha.in and ultrasound.money both require an API key now — and 34.35% sits above the commonly quoted staked share, so the definition is worth a second look.
The tab lists transfers in and out of the wallet, so "Tokens" read as a holdings view and set the wrong expectation. Its page title said "Token Portfolio and Assets Holdings", which described a portfolio the tab never showed. Renames the tab label and the page title across all three locales. The route stays /address/<addr>/tokens and the AccountPage.Tokens message namespace is unchanged, so existing links keep working and the diff stays confined to the strings a user actually sees. Verified in a local render for en, ru and pt: tab reads Movements / Движения / Movimentos and the title reads Token Movements / Движения токенов / Movimentos de Tokens, with no missing-message fallbacks.
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
task #486
Ethereum FDV was stuck at $0. getTvs aborted the whole snapshot whenever the
Beacon request returned nothing — discarding the supply it had already fetched
successfully from Etherscan/CoinGecko — so total_supply was never written and
update-fdv kept skipping on its supply <= 0 guard.
The Beacon request itself always failed: the mainnet validator endpoint returns
~1 GB in one response and res.json() cannot exceed the V8 512 MB string limit.
Three retries burned 313s every hour and always ended empty. Server-side
filtering does not help — ?status=active and ?status=active_ongoing return the
full set byte for byte. Now folded incrementally with stream-json: 15 MB heap
over the full gigabyte, 107s to a real result.
Mainnet no longer indexes validators one by one — jobs/get-nodes.ts queries the
database per node, so ~890k active validators would mean ~890k round trips.
Aggregate stake is streamed instead. Testnets keep the per-node path.
Also renames the account Tokens tab to Movements (en/ru/pt) — it lists wallet
transfers, not holdings. Route and message namespace unchanged.