One canvas per region, not one per chunk - #164
Conversation
The ground still vanished when you panned away and came back, and a page with a lot of it loaded went slow. Both come from the same choice: the client baked a 16x16 canvas PER CHUNK. A viewport is up to 4096 chunks, so holding even two screens meant thousands of canvas elements. Object overhead — not pixels — is what forced the cache limit to stay low, so #159's distance-ordered eviction was doing the right thing with far too little room: 8192 chunks is eight regions, and panning past that dropped ground that then had to be re-read. Drawing cost a walk of every held key plus up to 4096 drawImage calls, per frame, which is the "web starts to lag once a lot is loaded" half of the same report. The pixels are the same bytes either way. A region canvas is 512x512 and holds 1024 chunks, so the storage is unchanged and everything around it collapses by three orders of magnitude: MAP_REGIONS['rx,rz'] = { cv, st: {chunk: 1|0}, mk: {chunk: [...]} } 48 regions is about 48 MB and roughly twelve million blocks of ground — far more than a session pans over, which is what "it stays until I reload the page" actually needs. A viewport draws at most nine drawImage calls. Chunks are STAMPED IN as they arrive rather than baked once. A region arrives a few hundred chunks at a time, so baking it when its first chunk landed would leave it permanently mostly transparent — a half-loaded map, which is the thing being fixed. Per-chunk bookkeeping stays per chunk; only the pixels moved. Keeping the state map inside the region entry makes eviction correct by construction: dropping the canvas drops the claim to have drawn those chunks in the same statement, so the client can never believe it holds a tile it cannot draw. Markers moved in beside them for the same reason. Both clients, because they share the eviction policy and its limit — the desktop would otherwise have trimmed to 48 CHUNKS. Verification, each proved failable first (MSMS_SMOKE_WEB, by exit code): - MAP_JS is driven in a vm against a recording canvas. One response carries two chunks of one region, one of another, and one WEST OF ZERO, and the test asserts three canvases, four stamps, and the exact local offsets 0/16/128/496. A raw `cx % 32` gives -16 for the western chunk — outside the canvas, silently drawing nothing — and fails with the offsets named. - A second response must paint into the region that already exists: making regionFor rebuild instead fails with "a region did not record both of its chunks". - The retention fixture is 64 regions against the limit of 48, deliberately over but not wildly over, and asserts that panning one screen sideways keeps EVERY region of the screen you came from. Gates green: MSMS_SMOKE, _WEB, _WORLDS, _MODUPDATE, _ANALYSIS.
A chunk that came back READ AND EMPTY created its region entry with the canvas already allocated. So a region of nothing but ungenerated chunks — an ocean, the edge of the explored world, the void past a border — took a megabyte for pixels nothing would ever be drawn into, and took a slot in a 48-slot cache, evicting terrain that had really been read. Panning across empty ground was a way to lose the ground behind you. The entry still has to exist: it is what remembers the chunk is empty, and without it the client asks for it forever. Only the canvas is deferred, to the first chunk that actually needs one. Both clients. Proved failable: allocating on the empty path again fails with "a region with nothing drawn in it allocated a canvas". The stamping fixture also gained the empty case and a wider, shorter test viewport — the viewport is walked row by row and capped at 512, so the tall one spent its whole budget above z=0 and the fixture chunk at x=80 was never requested at all.
Self reviewOne real defect, found by asking "what does the empty path allocate". An ocean could evict the ground you actually loadedA chunk that came back read and empty created its region entry through the same constructor as a drawn one — canvas included: else if(known[k]){ mapRegionFor(cx,cz,true).st[k]=0; ... } // ← allocated 512x512So a region of nothing but ungenerated chunks — an ocean, the edge of the explored world, the void past a world border — took a megabyte for pixels nothing would ever be drawn into, and a slot in a 48-slot cache. Panning across empty ground was a way to lose the ground behind you, which is the exact complaint this PR is answering. The smaller the cache is in regions, the more this bites: at 8192 chunks the waste was diluted across thousands of entries, and consolidating made it sharp. The entry itself must stay — it is what remembers the chunk is empty, and without it the client asks forever (#136). Only the canvas is deferred, to the first chunk that needs one. Proved failable: allocating on the empty path again fails with The stamping fixture was not testing what it claimedAdding the empty case surfaced that the test viewport was tall and narrow. The viewport is walked row by row and capped at 512 chunks, so it spent its whole budget on rows above z=0 and the fixture chunk at x=80 was never requested — the response was about a chunk nobody asked for, and the assertion would have passed on any implementation. Now wide and short (126 x 4 = 504, just under the cap) so every fixture chunk is genuinely in the request. Considered and left
DisclosedThe desktop client is not driven by the vm harness — its equivalent paths are verified by inspection plus the shared policy ( Gates green by exit code: |
1 participant
The ground still vanished when you panned away and came back, and a page with a lot of it loaded went slow. Both are the same choice: the client baked a 16x16 canvas per chunk.
A viewport is up to 4096 chunks, so holding even two screens meant thousands of canvas elements. Object overhead — not pixels — is what forced the cache limit low, so #159s distance-ordered eviction was doing the right thing with far too little room: 8192 chunks is eight regions, and panning past that dropped ground that then had to be read again. Drawing cost a walk of every held key plus up to 4096
drawImagecalls per frame, which is the "web starts to lag once a lot is loaded" half of the same report.The pixels are the same bytes either way
A region canvas is 512x512 and holds 1024 chunks, so storage is unchanged and everything around it collapses by three orders of magnitude.
drawImageper frame48 regions is about 48 MB and far more ground than a session pans over, which is what "loaded areas stay until I reload the page" actually requires.
Two things that had to be right
Chunks are stamped in as they arrive, not baked once. A region arrives a few hundred chunks at a time. Baking a region canvas when its first chunk landed would leave it permanently mostly transparent — a half-loaded map, which is the thing being fixed.
Eviction is correct by construction. The per-chunk state map lives inside the region entry, so dropping the canvas drops the claim to have drawn those chunks in the same statement. The client can never believe it holds a tile it cannot draw. Markers moved in beside them for the same reason — they can no longer outlive the terrain they annotate.
Both clients changed, because they share the eviction policy and its limit: the desktop would otherwise have trimmed to 48 chunks.
Verification
Each proved failable before being trusted (
MSMS_SMOKE_WEB, by exit code):cx % 32instead of the wrapped modulochunks were stamped at the wrong offsets: -16:0 0:0 128:0 16:0regionForrebuilds instead of reusinga region did not record both of its chunksa region in view was dropped: 0,0MAP_JSis driven in avmagainst a recording canvas. One response carries two chunks of one region, one of another, and one west of zero, and the test asserts three canvases, four stamps, and the exact local offsets0/16/128/496. The western chunk is not decoration: chunk -1 belongs to region -1 at local 31, and a raw modulo gives -1 there — painting outside the canvas and silently drawing nothing. Without it in the fixture the wrap could be deleted and the test would still pass.The retention fixture is 64 regions against a limit of 48 — deliberately over but not wildly over, because at three times the limit everything evicts everything and the pan property stops being a property of the policy and becomes one of the fixture size. It asserts that panning one screen sideways keeps every region of the screen you came from.
Gates green:
MSMS_SMOKE,MSMS_SMOKE_WEB,MSMS_SMOKE_WORLDS,MSMS_SMOKE_MODUPDATE,MSMS_SMOKE_ANALYSIS.Closes #164