A from-scratch LLM inference engine for Qwen3, built to understand how vLLM-style serving works under the hood. The model, paged KV cache, continuous batching scheduler, prefix cache, and sampler are written in plain PyTorch, and the attention hot path runs on hand-written CUDA kernels.
-
Updated
Nov 10, 2026 - Python