Swarms and flocks in a virtual world — 25,000 entities at 90 ticks/second. Built specifically to break a toolchain.
Migratory is an asyncio world server for boid-style simulation at scale: GPU compute (CUDA kernels, Numba JIT), decoupled event loops connected by queues, a binary WebSocket protocol, and a fixed-timestep tick loop with off-loop simulation execution. A Three.js client renders it at 120fps.
▶ Watch it run — the simulation requires a GPU-capable machine on both ends, so a capture shows it better than a clone-and-run.
My first attempt at a large trading-data platform bogged down — async code fighting synchronous packages, tooling that couldn't hold the complexity. Instead of grinding at a system I cared about, I built one whose job was to fail: a simulation demanding enough to expose every weakness in the toolchain (Context Forge + Squadron) in a controlled setting.
It started as a Claude artifact simple enough to run in a chat window. Eight major iterations later it ran tens of thousands of entities — and getting there forced the hard architecture. Every major component below exists because a simpler version broke first:
- Decoupled event loops connected by queues — producer and consumer loops fully separated to prevent loop starvation; the single-loop version died first
- CUDA GPU kernels for the core simulation math; Numba JIT kernels where CUDA was the wrong tool
- Binary WebSocket protocol — JSON serialization was an early casualty at 25K entities per tick
- Graduated client backpressure — slow clients get bounded queues and eviction policy, not server stalls
Then, toolchain hardened, I went back and built the real thing.
Full story: From a Few Prompts to Two Billion Rows
Demonstrated: 25,000 entities at 90 ticks/second server-side; 120fps client rendering. The CI load tier gates conservatively at the v1 operating envelope (5K entities × 60 Hz × 5 clients) — see Tests.
Built with Context Forge and Squadron; the complete process record — slices, tasks, reviews, findings — ships in project-documents/. The receipts are inspectable.
uv run python -m migratory_server --config config/default.yaml
Controls how the static terrain grid is encoded and delivered on the wire.
Defaults match the slice 317 design: f32 + zstd, single-shot for grids
under 4 MiB post-compression and chunked above.
server:
terrain_wire:
dtype: f32 # f32 (default), uint16, f64
compression: auto # auto = zstd; explicit zstd | lz4 | none
chunk_transport_threshold_bytes: 4194304 # 4 MiB; above this → chunked
chunk_target_bytes: 1048576 # 1 MiB; uncompressed chunk size
quantization_range: auto # auto (computes min/max from grid) | fixed
quantization_min: null # required if quantization_range=fixed
quantization_max: null # required if quantization_range=fixed
LZ4 is an optional extra: uv sync --extra terrain-lz4. Selecting compression: lz4 without it installed raises a clear startup error.
Controls per-client outbound queue depth and the graduated slow-client eviction
policy. A client that fills its queue grace_overflow_events times without an
intervening successful send is disconnected with the configured close code.
Terrain messages are non-droppable: a client that cannot accept terrain within terrain_enqueue_timeout_seconds is evicted immediately.
server:
client_backpressure:
send_queue_max_messages: 16 # per-client queue depth cap
grace_overflow_events: 20 # consecutive overflows before eviction
eviction_close_code: 1013 # IANA "Try Again Later"
eviction_close_timeout_seconds: 1.0 # close-handshake timeout before transport.abort()
terrain_enqueue_timeout_seconds: 2.0 # per-message put timeout during terrain delivery
log_overflow_at_warn: true # log drops at WARN (false → DEBUG)
No protocol change — viewer and other clients are unaffected. Counters
(state_updates_dropped, overflow_events, queue_depth_hwm) are logged at
INFO on every disconnect.
The canonical TERRAIN v2 wire-format spec (single-shot 0x03,
TERRAIN_CHUNK 0x04, TERRAIN_HEADER 0x05) lives at project-documents/reference/terrain-wire-protocol-v2.md.
Client implementers (browser viewer, future tooling) should code against
that document.
uv run pytest # unit tests (load tier excluded by default)
uv run pytest -m load # run the opt-in load tier
Load tests run against an in-process server at the v1 operating envelope (5K entities × 60 Hz × 5 clients) and are gated in CI as a separate step.
MIGRATORY_DEBUG=1 uv run python -m migratory_server --config config/default.yaml
Enables asyncio debug mode with slow_callback_duration=0.05s. Use
when investigating suspected event-loop-blocking regressions. Off by
default; ~10–30% scheduling overhead when on.
MIT — see LICENSE.