Pinned Loading

1. rednote-machine-learning/Inference-autopilot rednote-machine-learning/Inference-autopilot Public

   Evidence-driven SGLang inference deployment optimization.

   Python 142 1

2. sgl-project/sglang sgl-project/sglang Public

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python 35.7k 8.7k

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 91.4k 22k

4. LightLLM LightLLM Public

   Forked from ModelTC/LightLLM

   LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

   Python

5. TensorRT-LLM TensorRT-LLM Public

   Forked from NVIDIA/TensorRT-LLM

   TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

   C++

6. 3D_cardiac_multi-structure_segmentation 3D_cardiac_multi-structure_segmentation Public

   Python