Repository navigation

serve: forward-serve warms the weight cache before it says ready (44.5 s it was hiding) - #1621

Merged

bong-water-water-bong merged 5 commits into

mainfrom

1bit/serve-warmup

Nov 9, 2026

Merged

serve: forward-serve warms the weight cache before it says ready (44.5 s it was hiding)#1621
bong-water-water-bong merged 5 commits into
mainfrom
1bit/serve-warmup

Commits on Nov 9, 2026
- npu: dump each layer's residual stream so a per-layer cosine is possible at all
  
  pi-npufeat
  authored and
  bong-water-water-bong
  committed
  75ef4b3
- npu: the profiler dumps for a prefill-only run (it required a decode launch first)
  pi-npufeat
  committed
  70bb369
- serve: forward-serve warms the weight cache before it says "ready"
  pi-npufeat
  committed
  013468b
- serve: the warm-up's log line names what it does not cover (routed MoE experts)
  pi-npufeat
  committed
  4157226
- serve: announce the warm-up, and raise the readiness helper's budget to cover it
  pi-npufeat
  committed
  3983734