Stop the map throwing away the ground you just looked at - #162
Conversation
Three faults behind "it loads piece by piece, and going somewhere else makes the first place load all over again". **It deleted what it had.** The desktop trimmed with `trimTiles(tiles, visibleChunks())` — keep the current viewport, drop everything else — so panning one screen deleted the screen you came from and panning back re-fetched all of it. The web kept an 8-chunk margin, which only moved the edge. Replaced by `tilesToDrop` in @shared/livemap: drop nothing until the cache is over its limit, then drop FARTHEST FROM THE VIEW first, so what survives is a ring of recently visited ground. The limit was also wrong in a way that guaranteed churn: 2048 tiles against a viewport that requests up to 4096, so a single view could evict itself and re-fetch on the next draw. It is now 8192, and the invariant that it must exceed the viewport cap is asserted rather than remembered. **It asked in bands.** 64 chunks a request, one in flight, is 64 sequential round trips for a full viewport. Now 512 — eight rounds. Both HTTP surfaces already gzip tile responses, so that is ~82 KB on the wire against ~10 KB. **It re-asked for what it was already waiting for.** The viewport is walked in order, so the chunks whose region was still being parsed were always at the front of the next request: the map spun on one band while the rest stayed blank. A chunk that comes back neither drawn nor listed empty is now left alone for 400 ms, and the request moves on to the rest of the view. `MAX_TILES_PER_REQUEST` is one constant in @shared/livemap, imported by the main process and interpolated into MAP_JS. It was two literals and a constant, and they have to agree: a client asking for more than the server reads gets a response that mentions neither the extra chunks nor a reason, and both handlers treated that silence as "empty" via `!pending` and would have blanked them permanently. That inference is gone — `empty` is the server's actual answer, and every requested chunk comes back drawn, listed empty, or still pending. Verification, all proved failable first (MSMS_SMOKE_WEB, by exit code): - restoring the viewport-only rule fails with "the cache was not trimmed to its limit: 2500" - inverting the distance order, which still trims to exactly the limit, fails with "a tile in view was dropped: 0,0" - desyncing MAP_JS back to 64 fails with "the web map asks for 64 but the server reads 512" The retention fixture is 10000 tiles, past the 8192 limit, because the body does not run below it — and it asserts the pan property directly: after panning one screen, at least 2000 of the previous screen's 2500 tiles are still held. Gates green: MSMS_SMOKE, _WORLDS, _MODUPDATE, _WEB. Closes #159
There was a problem hiding this comment.
Pull request overview
This PR fixes live map client-side tile churn by changing cache eviction from “keep only current viewport” to a size-based eviction policy that drops the farthest tiles first, while also increasing per-request chunk caps and removing a dangerous “pending === 0 implies empty” inference that could permanently blank unexamined chunks.
Changes:
- Centralizes map tuning constants in
@shared/livemap(request cap, viewport cap, cache keep limit) and updates clients/server/docs to stay in sync. - Replaces viewport-only trimming with distance-ranked eviction that only activates after exceeding the cache limit.
- Improves fetch behavior by increasing request batch size and adding a short per-chunk retry backoff while regions are still pending.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/shared/mapUi.ts | Updates web map JS generation: request caps, pending backoff, and farthest-first trimming. |
| src/shared/livemap.ts | Adds shared constants and the tested tilesToDrop eviction policy helper. |
| src/shared/apiSurface.ts | Updates endpoint docs to reflect new map/tiles request cap. |
| src/renderer/src/components/LiveMap.tsx | Updates desktop map client to use shared constants, tilesToDrop, and pending backoff. |
| src/main/smoke.ts | Adds smoke assertions for eviction determinism and client/server cap sync. |
| src/main/core/worldTiles.ts | Removes duplicated request-cap constant; re-exports shared constant for server parsing. |
| docs/openapi.json | Updates OpenAPI description to match the new request cap. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const k = c.cx + ',' + c.cz | ||
| return !tiles.current.has(k) && (waiting.current.get(k) ?? 0) <= now | ||
| }) | ||
| .slice(0, MAX_TILES_PER_REQUEST) | ||
| if (!want.length) return |
| var now=Date.now(); | ||
| var want=chunks.filter(function(c){ | ||
| var k=mapTileKey(c.cx,c.cz); | ||
| return MAP_TILES[k]===undefined&&!(MAP_TILE_WAIT[k]>now)}); | ||
| if(!want.length)return; | ||
| mapTrimTiles(); |
| if(MAP.loadAhead&&chunks.length){ | ||
| var xs=chunks.map(function(c){return c.cx}),zs=chunks.map(function(c){return c.cz}); | ||
| var x0=Math.min.apply(null,xs)-2,x1=Math.max.apply(null,xs)+2; | ||
| var z0=Math.min.apply(null,zs)-2,z1=Math.max.apply(null,zs)+2; | ||
| if((x1-x0+1)*(z1-z0+1)<=4096){ | ||
| chunks=[]; |
The backoff added in the parent commit can empty the request list completely — every chunk on screen is usually in the same few regions, so they all back off together. The retry only fires after a RESPONSE, and in that state there is no response coming, so the effect returned having scheduled nothing and the view stopped filling until the operator moved it. On the desktop that is indefinite; on the web it recovered on the 2-second refresh. Either way it looks exactly like the bug this branch is fixing. Both clients now schedule their own return, at the soonest backoff still outstanding. Driven for real rather than reasoned about: MAP_JS is run in a vm with a recording setTimeout and a host whose every response says "still reading". The first pass asks, the second finds everything backing off, and the test asserts a wake-up was scheduled — and that the second pass did NOT re-ask, which is the behaviour the backoff exists for. It also asserts something actually ended up backing off, so it cannot pass by never reaching the case. Removing the wake-up fails it with "a fully-backed-off view scheduled no wake-up; the map would stall". Also: trimTiles dropped tiles and marks but not the backoff entry, so that map kept one entry per chunk ever looked at for the life of the window. Gates green: MSMS_SMOKE, _WORLDS, _MODUPDATE, _WEB, _ANALYSIS, _AUDIT.
Self reviewRead the diff back. One real regression, one leak — both on the branch ( 1. The backoff could stall the map completelyThis is the one that mattered. The new backoff stops the client re-asking for chunks whose region is still being parsed. But every chunk on screen usually belongs to the same handful of regions, so they all back off together and the request list comes out empty: const want = visibleChunks().filter(/* not held, not backing off */)
if (!want.length) return // ← scheduled nothingThe only retry is Both clients now schedule their own return at the soonest outstanding backoff. Tested by driving it, not by reasoning about it.
Removing the wake-up fails it: 2. The backoff map outlived its tiles
Checked and left alone
DisclosedThe desktop client is React and is not driven by the vm harness — the equivalent stall fix there is verified by inspection and by the shared logic ( Gates green by exit code: |
Fixes the reported "harita parça parça yükleniyor, başka yeri yükleyince diğer yerin tekrar yüklenmesini bekliyoruz". #157 made the parse 9.7x faster; this is the client, which was throwing the results away.
Three faults
1. It deleted what it had. The desktop trimmed with
trimTiles(tiles.current, visibleChunks())— keep the current viewport, drop everything else. Pan one screen and the screen you came from was gone; pan back and all of it was re-fetched. That is the reported bug, exactly. The web kept an 8-chunk margin, which only moves the edge.Replaced by
tilesToDropin@shared/livemap: nothing is dropped until the cache is over its limit, and then the tiles farthest from the view go first. What survives is a ring of recently visited ground.2. The limit guaranteed churn. 2048 tiles against a viewport that requests up to 4096 — so a single view could exceed the cache, evict tiles it had just fetched, and re-fetch them on the next draw. Now 8192, with
TILE_KEEP_LIMIT > MAX_VIEWPORT_CHUNKSasserted rather than remembered.3. It asked in bands, and re-asked for what it was already waiting on. 64 chunks per request with one in flight is 64 sequential round trips for a full viewport. Now 512 — eight rounds. And because the viewport is walked in order, the chunks whose region was still being parsed were always at the front of the next request too, so the map spun on one band while the rest stayed blank; a chunk that comes back neither drawn nor listed empty is now left alone for 400 ms while the request moves on.
The trap this had to avoid
MAX_TILES_PER_REQUESTwas a constant inworldTiles.tsplus two hardcoded64s in the clients. Raising only the clients would have been silently destructive: the server reads its cap and the response then mentions neither the excess chunks nor a reason, and both handlers read that silence as "empty" —With a cache-hit response (
pending === 0) every unexamined chunk would have been marked permanently blank. So the constant now lives once in@shared/livemap, imported by the main process and interpolated intoMAP_JS, and the!pendinginference is gone —emptyis the servers actual answer, and every requested chunk comes back drawn, listed empty, or still pending.Verification
All three checks were proved failable before being trusted (
MSMS_SMOKE_WEB, by exit code):the cache was not trimmed to its limit: 2500a tile in view was dropped: 0,0MAP_JSback to 64the web map asks for 64 but the server reads 512The retention fixture holds 10000 tiles, past the 8192 limit — the function early-returns below it, so a small fixture would step over the body entirely and pass with the whole policy deleted. It asserts the reported property directly: after panning one screen right, at least 2000 of the previous screens 2500 tiles are still held. It also asserts determinism, since a redraw between two calls must not change what is kept.
Gates green:
MSMS_SMOKE,MSMS_SMOKE_WORLDS,MSMS_SMOKE_MODUPDATE,MSMS_SMOKE_WEB.Costs, stated
drawableChunksiterates every held key and splits it per draw, so that loop is now up to 8192 entries instead of 2048 — around a millisecond per pan event. Both would be fixed properly by baking one canvas per region instead of one per chunk (512x512 rather than 1024 small ones), which is worth doing but is a separate change to the draw path.Closes #159