PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.
-
Updated
Jan 7, 2026 - Rust
#
kv-cache-offload
Here are 3 public repositories matching this topic...
Unofficial fork of FreeToken (FlashML-org/FreeToken) with KV cache tiering: swap to host RAM / SSD, cross-restart session persistence (incl. QSA index tiers), and Windows-focused fixes. Personal experiments — upstream first.
-
Updated
Oct 25, 2026 - Python
On-device inference engine running Qwen MoE models larger than the phone's RAM - offline, exact, at usable speed.
android android-application smartphone quantization inference-optimization mixture-of-experts edge-ai kv-cache local-llm llm-inference qwen speculative-decoding qwen3 kv-cache-offload consumer-ai weight-streaming
-
Updated
Oct 26, 2026 - C++
Add this topic to your repo
To associate your repository with the kv-cache-offload topic, visit your repo's landing page and select "manage topics."