Adaptive sliding window query debouncing and micro-batch coalescing engine optimizing LLM and tool call throughput
Adaptive sliding window query debouncing and micro-batch coalescing engine optimizing LLM and tool call throughput
Ranabhum is a highly scalable, event-driven benchmark and real-time telemetry pipeline designed to compile, isolate, stress-test, and score high-performance financial order matching engines submitted by contestants.
Distributed real-time log ingestion and analytics engine using Redpanda, ClickHouse, FastAPI, and Grafana with micro-batching and DLQ isolation.
Production-grade AI security middleware with async micro-batching, prompt injection detection, RBAC enforcement, PII redaction, cost tracking, and concurrency benchmarking.
Inference gateway for batching AI model requests through bounded micro-batching, queue backpressure, per-request result routing, and reproducible performance evaluation.
Production-grade ML model serving platform — versioned registry, async micro-batching, canary routing, prediction logging, and PSI drift detection. FastAPI · PostgreSQL · Redis · Docker · Kubernetes.
High-throughput, distributed real-time chat platform engineered with Socket.IO, Redis Streams, MongoDB, and Next.js. Features sub-millisecond delivery (p50: 1.0ms), asynchronous micro-batch persistence, atomic Lua rate limiting, memory backpressure circuit breakers, and zero-downtime graceful shutdown.
Per-tenant SQLite (WAL) multi-tenant e-commerce platform — isolation, micro-batching, in-memory cache & resource footprint. Rust + Actix-Web + k6.
ONNX Runtime image classifier with micro-batching, an LLM gateway that measures TTFT and tokens/s, and the load generator that benchmarks them; Docker, Kubernetes, CI
Benchmark harness that turns model-serving configs into dollars per 1M requests. Serves a 4.9M-param intent classifier as fp32, dynamic int8, and TorchScript variants with and without micro-batching, measures p50/p95/p99, req/s, and peak RSS, and prices throughput against a dated AWS snapshot.
To associate your repository with the micro-batching topic, visit your repo's landing page and select "manage topics."