Building fast LLM inference at Coral Bricks. I fix things in the engines we run in production.

vLLM _{· 2 merged · 0 open · 1 reported}
🟣 #58457 Catch a block size a backend rejects, at startup
🟣 #49845 Start mixed-attention models without --block-size
🐞 #49844 Garbage output with PP=2 on GLM-MoE-DSA when torch.compile and CUDA graphs combine

SGLang _{· 1 merged · 6 open · 5 reported}
🟣 #38932 Fix NVFP4 crash with adaptive speculative decoding
🟢 #44940 Function_call: Stop re-scanning the open DSML invoke on every streamed delta (DeepSeek V3.2/V4/V4.1)
🟢 #44923 Refresh is_all_greedy / is_any_greedy when filter_batch removes rows
🟢 #42812 Fix a DeepSeek-V4 hang with HiCache and DSPARK
🟢 #41379 Test image tokens in usage on the chat path
🟢 #41324 Stop one logprobs stream from stalling the server
🟢 #39240 Fix Gemma 4 tool calls with mangled arguments
🐞 #44941 DeepSeek V3.2/V4/V4.1 streaming tool-call parser: pre-existing bugs found while fuzzing
🐞 #42548 DeepSeek-V4 HiCache: the SWA host pool is too small, so idle prefixes miss the host tier
🐞 #42465 DeepSeek-V4 + HiCache: TP ranks deadlock under concurrent long prefills
✅ #38795 NVFP4 + adaptive speculative decoding fails CUDA-graph capture
✅ #32459 EAGLE speculative decoding silently collapses multi-turn prefix reuse

LMCache _{· 3 merged · 8 open · 2 reported}
🟣 #4830 Fix silent KV cache corruption
🟣 #4079 Share KV copy code across transfer modes
🟣 #4015 Free a dead client's pinned GPU memory
🟢 #4944 Fix flaky port collisions in MQ tests
🟢 #4410 Hybrid-attention models in engine-driven mode
🟢 #4072 Survive a cache server restart without crashing vLLM
🟢 #4068 Release locks on skipped cache writes
🟢 #3992 Correct the hybrid-models docs
🟢 #3932 Keep serving when the S3 tier hits its cap
🟢 #3909 Rebuild S3 tier state after a restart
🟢 #3892 One S3 bucket per tenant
🐞 #4071 Design proposal: hybrid-attention (multi-group KV) support for engine-driven transport
✅ #4014 The server keeps a dead vLLM client's KV pool memory until it restarts

FlashInfer _{· 0 merged · 1 open · 1 reported}
🟢 #4885 Stabilize JIT cached ops across environments
🐞 #4884 JIT cached ops rebuild needlessly (PATH-dependent CUDA home, mtime staleness)

_{🟣 merged · 🟢 open · ⚪ draft · 🐞 issue reported · ✅ issue resolved · updated automatically every day}