AI Max+ 395 acceleration: measured heterogeneous GPU PD and asynchronous fused-layer pipeline experiments with RTX 3060.
-
Updated
Sep 5, 2026 - Python
#
prefill-decode
Here are 12 public repositories matching this topic...
KVTide is a Kubernetes-native LLM serving system exploring cache-aware scheduling and proactive peer-to-peer KV mobility.
scheduler distributed-computing inference golang-server mlsys ai-infra dynamic-batching llm-serving llm-inference connectrpc prefix-cache prefill-decode
-
Updated
Sep 4, 2026 - Go
Peer-to-peer L3 KV-cache storage for SGLang HiCache with RDMA zero-copy — no centralized master.
-
Updated
Sep 2, 2026 - Python
🏆[ICML 2026 Spotlight] Official implementation of "DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions"
protocol large large-language-models mechanistic-interpretability activation-steering prefill-decode
-
Updated
Jun 21, 2026 - Python
Trace-driven Python simulator for LLM inference economics: continuous batching, prefill/decode disaggregation, speculative decoding, MoE expert balancing, tiered HBM/DRAM/NVMe KV caching, GPU cost, margins, and payback.
python simulation gpu nvme mixture-of-experts kv-cache llm-inference speculative-decoding deepseek continuous-batching prefill-decode inference-economics
-
Updated
Jul 24, 2026 - Python
Mini-Dynamo: disaggregated prefill/decode LLM inference platform with KV-cache-aware routing and observability (simulated, no GPU needed)
python kubernetes redis distributed-systems grafana prometheus observability inference-server fastapi opentelemetry kv-cache llm-inference continuous-batching nvidia-dynamo disaggregated-inference prefill-decode
-
Updated
Aug 30, 2026 - Python
A curated map of AFD, PD disaggregation, KV-cache systems, MoE serving, and re-aggregation baselines for LLM serving.
moe awesome-list disaggregation afd mixture-of-experts kv-cache llm-serving inference-systems prefill-decode
-
Updated
May 15, 2026 - Python
PyTorch GPU inference runtime for disaggregated LLM serving, with Qwen3 support, KV-cache management, two-phase scheduling, and prefill/decode workers.
-
Updated
Sep 1, 2026 - Python
DiSpec — a from-scratch LLM inference engine: paged attention, continuous batching, CUDA-graph decode, speculative decoding, and prefill/decode disaggregation
cuda inference kv-cache llm-inference speculative-decoding paged-attention cuda-graphs disaggregated-inference prefill-decode continuous-batch
-
Updated
Jun 24, 2026 - Python
Kubernetes control plane for disaggregated LLM serving: prefix-aware routing, prefill/decode scheduling, and KV-cache orchestration for prism-infer workers.
python kubernetes distributed-systems helm nats model-serving fastapi kv-cache llm-serving prefill-decode
-
Updated
Sep 2, 2026 - Python
A datacenter-scale simulation framework for energy- and SLO-aware LLM inference serving — non-invasive extension of CloudSim Plus
simulation gpu cloud-computing splitwise autoscaling cloudsim datacenter kv-cache large-language-models llm carbon-aware prefill-decode
-
Updated
Aug 25, 2026 - Java
Measuring the KV-transfer tax of disaggregated prefill/decode LLM serving (vLLM + NIXL, 4x A10G): on PCIe-only hardware, disagg loses to plain data-parallel replication — measured, committed, reproducible.
kv-cache inference-benchmark llm-serving vllm llm-inference nixl disaggregated-serving prefill-decode goodput
-
Updated
Jul 13, 2026 - Python
Add this topic to your repo
To associate your repository with the prefill-decode topic, visit your repo's landing page and select "manage topics."