The Marin Project

A high-throughput and memory-efficient inference and serving engine for LLMs

A machine learning compiler for GPUs, CPUs, and ML accelerators

Automatic evals for LLMs

TPU inference for vLLM, with unified JAX and PyTorch support.

Harbor is a framework for running agent evaluations and creating and using RL environments.