Skip to content
#

model-sharding

Here are 9 public repositories matching this topic...

PyTorch implementation of Zero-Gated Cross-Attention (IGCA), a zero-initialized bridge that lets a frozen LLM take in images without changing its outputs at the start of training. Runs on a 4-bit Falcon-40B across multiple GPUs, verifies an exact 0.00e+00 initialization identity, and cuts held-out loss by 48.9%.

  • Updated Nov 5, 2026
  • Python

Add this topic to your repo

To associate your repository with the model-sharding topic, visit your repo's landing page and select "manage topics."

Learn more