Low-latency trading components in C++20, and the lessons that led to them.
The goal is a port of OpenHFT-Lab — a market data and market making engine originally written in C#/.NET — to C++, starting from the pieces where the language actually pays for itself: the lock-free ring buffer, the order book, and the hot path between them.
The C# version sustains ~3,500 market data events per second with ~200 µs end-to-end latency. The target here is to beat both, and to understand exactly why the difference exists rather than assuming C++ is faster.
Status: the core pipeline is ported and measured. What remains is live exchange connectivity (the WebSocket transport itself) and the risk layer.
Tick-to-decision — JSON parse → ring buffer → book update → strategy → quote — measured over 500k simulated Binance messages (2.1M events) flowing through two real threads:
| Percentile | This engine | C# baseline* |
|---|---|---|
| p50 | 417 ns | ~200,000 ns |
| p90 | 576 ns | — |
| p99 | 8.8 µs | — |
| p99.9 | 15.8 µs | ~800,000 ns |
Sustained throughput: 11.4 M events/sec (the C# engine processed 3,500/sec).
* The comparison is close but not exact: the C# figure includes the network hop to the exchange; this measures everything after the socket. The tail (p99+) comes from scheduler preemption — WSL2 offers no CPU isolation, and the run uses no thread pinning.
All measured on the reference hardware below, -O2, sanitizer-clean:
| Component | Operation | Cost |
|---|---|---|
LockFreeRingBuffer |
write + read, 64-byte event | 2.8–3.1 ns |
OrderBook |
level update (insert/update/delete mix, ~100 levels) | 31 ns |
OrderBook |
spread + mid read | 0.4 ns |
OrderBook |
depth imbalance, top 5 levels | 1.9 ns |
BinanceParser + pipeline |
full tick-to-decision | 417 ns p50 |
The C# targets were < 1 µs per book update and ~200 µs end-to-end.
ThreadSanitizer is silent across the full two-thread pipeline; every test also passes under AddressSanitizer + UBSan.
include/openhft/ the library (header-only; templates need to be visible)
core/ ring buffer, market event, timestamps
book/ order book, price levels
feed/ market data adapters
src/ non-template implementation, when there is any
tests/ test_*.cpp, run by ctest
bench/ bench_*.cpp, always built with -O2
learning/ the six C++ lessons - practice code, not the engine
docs/ specifications
The split matters: learning/ is where I worked out how the language behaves,
and it is kept because the measured numbers there are what the engine's design
decisions rest on. It is not part of the library.
cmake -S . -B build
cmake --build build -j
ctest --test-dir build --output-on-failureDefaults to Release — at -O0, any number this project reports is
meaningless.
| Option | Default | Effect |
|---|---|---|
HFT_BUILD_LEARNING |
ON | Build the lesson programs |
HFT_BUILD_TESTS |
ON | Build the tests |
HFT_BUILD_BENCH |
ON | Build the benchmarks |
HFT_ASAN |
OFF | AddressSanitizer + UBSan |
HFT_TSAN |
OFF | ThreadSanitizer |
HFT_NATIVE |
OFF | -march=native |
# hunting memory errors
cmake -S . -B build-asan -DHFT_ASAN=ON && cmake --build build-asan -j
# hunting data races - the only tool that finds a wrong memory order
cmake -S . -B build-tsan -DHFT_TSAN=ON && cmake --build build-tsan -jHFT_ASAN and HFT_TSAN are mutually exclusive; the build stops if both are
set.
C++20 compiler (g++ 13+ or clang 16+), CMake 3.20+, pthreads.
Every measurement in this repository comes from:
AMD Ryzen 9 9900X3D · 12 cores / 24 threads
L1d 576 KiB · L2 12 MiB · L3 96 MiB (3D V-Cache)
Cache line: 64 bytes
Ubuntu 24.04 (WSL2) · g++ 13.3 · C++20
Numbers are hardware-specific and worth re-measuring elsewhere. The false sharing penalty here is 2.8x, against a commonly quoted 5–10x — the large V-Cache absorbs much of the coherence traffic.
Six lessons from zero C++ to the ring buffer:
learning/.
Headline results, measured:
- 582x between walking a contiguous array and a linked list — same data, same additions, same O(n)
- 33% of a struct's size saved by declaring fields largest-first
- ~100x cost of sharing one counter across 4 threads vs thread-local
- 2.8x from a single
alignas(64)
And the finding that shaped how the rest of this is written: a formally
incorrect relaxed ordering passed 20 of 20 runs on x86-64. Only
ThreadSanitizer caught it. Bugs at this level produce no signal — correct
output, clean build, passing tests, and broken on ARM.
Pull requests welcome, including from people learning C++ alongside this — the
lessons exist for that reason. main is protected, so everything goes through
a PR. See CONTRIBUTING.md for the build checks a change has
to pass, and the list of what needs doing.
- C++ fundamentals through atomics and memory ordering
-
LockFreeRingBuffer<T, N>— SPSC, spec -
MarketEvent— 64 bytes, one cache line, fixed-point prices -
SymbolTable— per-symbol price/quantity scales -
OrderBook— L2 sorted-array book, 31 ns per update -
BinanceParser— hand-rolled depthUpdate/trade parsing straight to ticks, sequence-gap detection per the Binance continuity rule - Market making strategy — adaptive spread, inventory skew, position limits, post-only discipline
- Engine loop + simulated feed + end-to-end benchmark: 417 ns p50 tick-to-decision vs the ~200 µs C# baseline
- Live WebSocket transport (Boost.Beast + TLS) feeding the same parser
- Gap recovery via REST snapshot (parser already flags the gap)
- Risk manager port (position/order/rate limits before the gateway)
- Per-symbol strategy state
- CI: build + ctest + ASan/TSan on every PR
MIT