Ultra-low bitrate speech codec (0.27-1 kbps) with cross-modal alignment and real-time capabilities
-
Updated
Aug 27, 2025 - Python
Ultra-low bitrate speech codec (0.27-1 kbps) with cross-modal alignment and real-time capabilities
[FG 2024] Finite Scalar Quantization as Facial Tokenizer for Dyadic Reaction Generation - Winning Solution in REACT@FG24 Challenge
1.09 kbps neural speech codec for speech language models and TTS. Two FSQ token streams, 24 kHz output. Strongest on Indian languages.
Low-bitrate factorized speech codec on top of FACodec: FSQ distillation to replace RVQ (WER, mel and speaker similarity vs bitrate, with bootstrap CIs) and a learned reverberation/noise "environment" factor (partial result, with limitations).
面向低资源方言开放词表检索的跨模态对齐与路由陷阱实验室:组合式零样本 OOV、语速不变跨模态注意力、类条件域对齐与 FSQ 离散晶格。Modular cross-modal dialect alignment benchmark with a live white-box Lab.
A short path to a world model in your LLM — few-shot fine-tune it to predict in latent space (emit a set of next states, a JEPA world model). Also makes bespoke embedding models. lang·set·fit.
Track how discrete representations evolve during neural network training — lifecycle events, phase transitions, ontology discovery, and causal verification
To associate your repository with the fsq topic, visit your repo's landing page and select "manage topics."