The expert-cache question was already answered in mi210-vllm, and the guide that framed it was wrong

[Andrei-Dr]

A recurring proposal — trace the router, simulate an LRU expert cache, then implement one — came round again. It should not be run, and the reason is already on disk. docs/60, plus a correction to guides/moe-expert-cache-vllm.md.

1. The study already ran. mi210-vllm/docs/ROUTING.md, 2026-08-06, on glm52-int4int8: route_probe_sitecustomize.py instruments the router, drive_routing.py drives a six-language mixed workload, analyze_routing.py replays it through static top-K and LRU with prefill/decode separated and K swept. That is phase 1 plus most of phase 2 of the proposal, already built.

metric	value
per-layer normalized entropy	~0.91 (near-uniform over 256 experts)
per-layer Gini	~0.51
top-10% of experts	42% of accesses
LRU vs static top-K	tracks it

At 8-of-256 experts per token the cost is 8 experts x ~76 layers crossing PCIe whichever ones they are, and there is ~2.6 GiB free VRAM per card after the offload window. Verdict there: "caching will not help."

2. The deployment the proposal optimizes is gone. The framing is "25 of 48 expert layers on CPU." /models/mimo-v25/... does not exist, no MiMo weights remain on /mnt/llm-storage, llama-swap is not running, and the live litellm config serves coder / deephat / glm-5.2 / laguna. The README's architecture section still describes the MiMo split — flagged, not rewritten here, since that is your call.

3. Guide corrected, on both counts, precisely.

• --moe-expert-cache-size is documented as shipped. It is not — open PR [MoE][Offload] Run MoE models exceeding VRAM via expert CPU offloading with GPU cache (--moe-expert-cache-size) vllm-project/vllm#37190. But "unmerged" is not the objection: mi210-vllm/patches/registry.yaml carries five unmerged vLLM branches on purpose and aiter-cdna2 goes much further. The objections are that #37190 needs --enforce-eager (surrendering the 99.9%-coverage graph decode of docs/42, whose eager control was 3.5x slower wall) and is BF16 / no EP>1, so it cannot serve Q4_K at all — and that the trace says it would not pay anyway.
• The VLLM_USE_AITER=0 advice: true of stock AITER, false as a hardware claim. Upstream really does ship gfx942/gfx950/gfx1250 objects only and gate by arch name — aiter-cdna2's README says VLLM_ROCM_USE_AITER=1 "has no effect whatsoever" on an unpatched MI210. What makes it false here is aiter-cdna2 repatching 242 of 1,422 objects and opening on_mi3xx(). The note is now conditional on which image you are running, since the guide is public and most readers do not have the patched one — and it carries the caveat that ROCM_AITER_FA silently corrupts sliding-window attention on gfx90a, so "AITER on" is not unconditionally right even here.

4. Two claims corrected that keep being cited as successes. The vLLM PrefetchOffloader crashed in ncclAllReduce (docs/29 §2; round18_prefetch_eager.sh is the retry and its header opens "WHAT FAILED"; the 695.9 t/s is a target string). Selective UVA was never measured — round20 states predictions, round15 refuses to run without ALLOW_UVM_HANG=1, and the only UVA result is a VM fault at weight load.

Still open, narrowly: does some other MoE route more skewed than entropy 0.91? That is a port of the existing tooling, not a new simulator, and it needs a resident MoE to point at.

Two ledger entries added. No production change.