A bounded lock-free single-producer / single-consumer ring buffer in C++20, built for market-data style workloads where the producer must never block.
Both push() and pop() are wait-free: they complete in a bounded number
of steps regardless of what the other thread is doing, with no locks, no spins
and no syscalls. When the buffer is full the producer drops the new item and
counts it rather than stalling — the behaviour a feed handler needs, where
blocking the socket thread is worse than losing a tick.
#include <spsc/ring_buffer.hpp>
spsc::RingBuffer<Tick, 4096> ring;
// feed thread
if (!ring.push(tick)) { /* consumer fell behind; see ring.dropped() */ }
// processing thread
while (auto tick = ring.pop()) handle(*tick);Measured: 462M items/s across threads, 25x a mutex-guarded queue.
- Why lock-free here
- How it works
- Memory ordering
- False sharing
- Overflow policy
- API reference
- Building
- Testing
- Benchmarks
- Results
- Live validation: Binance BTC/USDT
- Limitations
The SPSC constraint — exactly one producer thread, exactly one consumer thread — is what makes this both simple and fast. With a single writer per index, neither index needs a compare-exchange, a retry loop, or any contention handling: each thread owns one index and only reads the other.
That is a real restriction, not a technicality. Two producers on the same ring corrupts it silently. If you need multiple producers, this is the wrong structure.
What it buys, measured below: 0.54 ns per push/pop pair single-threaded and 462M items/s across two threads, versus 18.3M/s for the same handoff through a mutex.
A fixed array of slots plus two monotonically increasing indices.
graph LR
subgraph ring["RingBuffer<T, 8>"]
S0["slot 0<br/>consumed"]
S1["slot 1<br/>consumed"]
S2["slot 2<br/>LIVE"]
S3["slot 3<br/>LIVE"]
S4["slot 4<br/>LIVE"]
S5["slot 5<br/>free"]
S6["slot 6<br/>free"]
S7["slot 7<br/>free"]
end
R["read_idx = 2<br/>consumer owns"] --> S2
W["write_idx = 5<br/>producer owns"] --> S5
style S2 fill:#e6f4ea,stroke:#34a853,color:#000
style S3 fill:#e6f4ea,stroke:#34a853,color:#000
style S4 fill:#e6f4ea,stroke:#34a853,color:#000
style R fill:#e8f0fe,stroke:#4285f4,color:#000
style W fill:#fef7e0,stroke:#f9ab00,color:#000
Two design choices carry their weight here:
Indices are never wrapped. They grow forever and are masked only when
addressing a slot: storage_[i & (Capacity - 1)]. This is why write - read is
a correct count even after unsigned overflow, and why all Capacity slots are
usable — a design that wraps the indices themselves has to waste one slot to
tell "full" from "empty".
Capacity must be a power of two, enforced by static_assert, so the mask
replaces a division.
sequenceDiagram
participant P as Producer thread
participant M as Memory
participant C as Consumer thread
Note over P: w = write_idx (relaxed - we own it)
P->>M: read read_idx (ACQUIRE)
alt full
P->>P: ++dropped, return false
else has room
P->>M: construct element at slot(w)
P->>M: write_idx = w+1 (RELEASE)
Note over P,C: release/acquire pair:<br/>the element is visible<br/>before the index that points at it
end
Note over C: r = read_idx (relaxed - we own it)
C->>M: read write_idx (ACQUIRE)
alt empty
C->>C: return nullopt
else has data
C->>M: move out of slot(r), destroy
C->>M: read_idx = r+1 (RELEASE)
Note over C,P: producer may now reuse the slot
end
The correctness of the whole structure rests on two release/acquire pairs. Getting these wrong produces a bug that is invisible in testing and shows up as corrupted data under load on someone else's machine.
| Operation | Ordering | Why |
|---|---|---|
Producer reads its own write_idx |
relaxed |
Nobody else writes it |
Producer reads read_idx |
acquire | Must see the consumer's slot destruction before reusing the slot |
Producer writes write_idx |
release | Publishes the constructed element; it must be visible first |
Consumer reads its own read_idx |
relaxed |
Nobody else writes it |
Consumer reads write_idx |
acquire | Pairs with the producer's release — seeing the index means seeing the element |
Consumer writes read_idx |
release | Frees the slot; destruction must be visible first |
The failure mode if the producer's store were relaxed: the consumer can
observe the incremented index before the element it points at has been fully
written, and read a half-constructed object.
This is verified, not assumed. Changing that one store to relaxed and
re-running under ThreadSanitizer produces a data race report:
WARNING: ThreadSanitizer: data race
SUMMARY: ThreadSanitizer: data race in std::_Optional_payload_base<P>::_Storage
The correct version is clean. That negative control matters: a green TSan run only means something if you have confirmed TSan would fail on the broken version.
tests/test_concurrent.cpp also carries a payload with internal redundancy
(seq, seq_copy, checksum) so that a torn read is detectable even when TSan
is not running.
write_idx_ is written by the producer and read by the consumer; read_idx_ is
the reverse. If they shared a cache line, every push would invalidate the
consumer's cached copy of read_idx_ and vice versa — the line would bounce
between cores on every single operation.
Each index therefore gets its own cache line via alignas(64).
graph TB
subgraph bad["Unpadded - one cache line"]
direction LR
BL["write_idx | read_idx"]
BP["producer core"] -.->|"writes"| BL
BC["consumer core"] -.->|"writes"| BL
BL -.->|"line bounces"| BP
end
subgraph good["Padded - separate lines"]
direction LR
GL1["write_idx + 56B pad"]
GL2["read_idx + 56B pad"]
GP["producer core"] -->|"writes"| GL1
GC["consumer core"] -->|"writes"| GL2
end
style bad fill:#fce8e6,stroke:#ea4335,color:#000
style good fill:#e6f4ea,stroke:#34a853,color:#000
Measured cost, 100k items across two threads:
| Layout | Throughput |
|---|---|
| Padded (this implementation) | 389.6 M items/s |
| Unpadded, same logic otherwise | 336.6 M items/s |
1.16x — worth having, but smaller than the folklore suggests. The commonly cited 2-3x speedups come from tighter loops than this one; here the 4096-element buffer already separates the hot data and the hardware prefetcher hides part of the cost. The padding is free in every sense except a few bytes of memory, so it stays, but it is not where the 25x over a mutex comes from.
BM_FalseSharingPadded vs BM_FalseSharingUnpadded measures this directly —
bench/bench_ring.cpp contains a deliberately unpadded ring for the comparison.
Padding solves false sharing — two unrelated variables colliding on one line.
It does nothing about the true sharing underneath: push() genuinely has to
know where the consumer is, and reading that atomic pulls a line owned by
another core on every single call.
The fix is to notice that the producer does not need a current value. It only needs to know whether there is room, and usually there is. So each thread keeps a private, non-atomic copy of the opposite cursor and checks that first; the shared atomic is read only when the cached copy says we are stuck, which then either frees us or confirms the stall. The cached value is always conservative — the cursors only ever advance — so it can report "full" when there is room, never "room" when there is none. Each cached copy lives on the same cache line as its owner's index, since it is private to that thread.
Measured as an A/B of the same class with and without the caching:
| Capacity | With caching | Without | Ratio |
|---|---|---|---|
| 16 | 166.8 M/s | 217.1 M/s | 0.77x |
| 64 | 400.4 M/s | 427.7 M/s | 0.94x |
| 256 | 522.4 M/s | 489.7 M/s | 1.07x |
| 1,024 | 521.0 M/s | 415.9 M/s | 1.25x |
| 16,384 | 723.7 M/s | 587.7 M/s | 1.23x |
The technique is not free, and below a certain size it is a net loss. A ring that is usually full makes the cached check fail and fall through to the atomic anyway, so you pay for the extra branch and gain nothing. It pays off when the ring has slack — which is exactly the feed-handler case this buffer is built for, where the consumer keeps up and the buffer is mostly empty.
Worth stating because the first benchmark I wrote for this said the opposite.
It compared against a separate hand-written class that stored elements by
assignment rather than construct_at, so it was measuring the difference in
representation, not in caching. The honest A/B needs the same class both ways.
When the ring is full, push() drops the new item and increments a counter.
It never blocks, never overwrites unconsumed data, and never fails silently.
flowchart TD
P["push(item)"] --> C{"room available?"}
C -->|"yes"| S["construct in slot<br/>publish index"] --> T["return true"]
C -->|"no"| D["++dropped<br/>item is discarded"] --> F["return false"]
style S fill:#e6f4ea,stroke:#34a853,color:#000
style D fill:#fef7e0,stroke:#f9ab00,color:#000
This is the feed-handler contract: a socket thread that blocks backs pressure up
into the network stack, which is far worse than losing a tick. dropped() is the
health metric — a rising count means the consumer cannot keep up.
The alternative policies were considered and rejected: overwrite-oldest complicates the consumer (it can be reading a slot the producer is overwriting), and block until space stops being lock-free on the write path, which was the entire point.
Symmetrically, pop() returns std::nullopt immediately on an empty ring.
Waiting policy belongs to the caller — spin, yield, sleep, or come back
later. Baking a wait into the structure would drag scheduling into a data
structure whose purpose is to avoid the scheduler.
T must be nothrow-move-constructible. Capacity must be a power of two. Both
are enforced by static_assert.
| Member | Thread | Description |
|---|---|---|
push(item) -> bool |
producer | Enqueues. false if full (item dropped). |
emplace(args...) -> bool |
producer | Constructs in place. false if full. |
pop() -> optional<T> |
consumer | Oldest item, or nullopt if empty. Never blocks. |
size() |
either | Snapshot; racy by nature. |
empty(), full() |
either | Snapshots. |
capacity() |
either | constexpr, equals Capacity. |
dropped() |
either | Items lost to a full buffer. |
size(), empty() and full() are inherently approximate: the other thread
may act between the read and your use of the value. They are for metrics and
tests, not control flow.
The destructor destroys any elements the consumer never took.
Header-only. Requires C++20 (GCC 11+, Clang 14+, MSVC 19.30+). Copy
include/spsc/ or use CMake:
add_subdirectory(spsc-ring)
target_link_libraries(my_target PRIVATE spsc::spsc)| Preset | Purpose |
|---|---|
release |
Optimised, benchmarks enabled |
tsan |
ThreadSanitizer — the important one here |
asan |
AddressSanitizer + UBSan |
cmake --preset release && cmake --build build/release -j
ctest --test-dir build/release22 test cases in two files:
| File | Covers |
|---|---|
test_ring_buffer.cpp |
FIFO order, exact-capacity fill, drop counting, index wraparound, element lifetime (no leaks, no double-destroy) |
test_concurrent.cpp |
Real producer/consumer threads: no loss, no reordering, no torn reads, drop accounting under a slow consumer |
ctest --test-dir build/release -j4
cmake --preset tsan && cmake --build build/tsan -j && ctest --test-dir build/tsanClean under Release, ASan and TSan.
./build/release/bench/spsc_bench
./build/release/bench/spsc_bench --benchmark_filter=FalseSharing| Benchmark | Question |
|---|---|
BM_PushPopSingleThread |
Pure instruction cost, no contention |
BM_PushPopSingleThreadMutex |
The same through a mutex |
BM_PopEmpty |
The path a spinning consumer hammers |
BM_ThroughputCrossThread |
Real two-thread throughput |
BM_ThroughputCrossThreadMutex |
The mutex baseline |
BM_FalseSharingPadded / Unpadded |
What the cache-line padding buys |
BM_Capacity<N> |
Throughput against ring size |
24-core machine, WSL2, GCC 13, -O3:
| Benchmark | Throughput | Note |
|---|---|---|
| Push+pop, single thread | 1.76 G items/s | 0.54 ns per pair |
| Push+pop through a mutex | 222.8 M items/s | 4.24 ns per pair — 7.9x slower |
| Pop on empty | 2.65 G items/s | 0.36 ns |
| Cross-thread handoff | 462.3 M items/s | |
| Cross-thread through a mutex | 18.3 M items/s | 25x slower |
Capacity sweep, cross-thread:
| Capacity | Throughput |
|---|---|
| 64 | 326.4 M items/s |
| 1,024 | 349.6 M items/s |
| 16,384 | 459.5 M items/s |
Bigger rings absorb producer bursts, so the producer stalls less waiting for space. The gain flattens once the working set leaves cache.
The examples/feed.cpp handler pushes 1M synthetic ticks: 217 M ticks/s with
10.6% dropped — the producer never once blocked, and the drop counter is what
tells you the consumer was behind.
Measured under WSL2. Absolute numbers on native Linux will differ; the ratios are the durable result.
Synthetic benchmarks prove throughput; they do not prove the design survives a real feed, where arrival is bursty, the producer is a library thread you do not control, and correctness is checkable against an external sequence number.
live/binance_feed.cpp connects to Binance's @depth@100ms WebSocket stream,
maintains a local BTC/USDT order book, and reports microstructure metrics.
cmake -S . -B build/live -DCMAKE_BUILD_TYPE=Release -DSPSC_BUILD_LIVE=ON
cmake --build build/live -j
./build/live/live/spsc_live_binance btcusdt 30Requires libssl-dev. Pulls IXWebSocket via FetchContent; off by default.
graph LR
B["Binance<br/>wss depth@100ms"] -->|"network"| WS["IXWebSocket thread<br/>PRODUCER"]
WS -->|"push raw JSON<br/>never blocks"| R["SPSC ring<br/>8192 slots"]
R -->|"pop"| C["main thread<br/>CONSUMER"]
C --> P["parse JSON"]
P --> OB["apply diffs<br/>to local book"]
OB --> M["spread / mid<br/>imbalance"]
style WS fill:#fef7e0,stroke:#f9ab00,color:#000
style R fill:#e8f0fe,stroke:#4285f4,color:#000
style C fill:#e6f4ea,stroke:#34a853,color:#000
The division of labour is the point: the WebSocket callback timestamps the message, moves it into the ring, and returns. It does not parse, allocate beyond the payload, or take a lock — stalling there would push back-pressure into the socket. Every expensive step happens on the consumer.
--- ring ---
produced : 322 messages
consumed : 319
dropped : 0 (none)
rate : 10.5 msg/s
--- queueing latency (ring push -> pop) ---
mean : 1689.0 us
p50 : 135.2 us
p99 : 332.4 us
--- book ---
diffs applied : 316
stale (pre-snapshot, expected) : 3
sequence gaps : 0 (book is consistent)
final mid : 79850.01
final spread : 0.01
final imbalance: 0.446
Zero sequence gaps across 316 applied diffs is the meaningful correctness
result. Binance stamps each event with a first (U) and final (u) update id;
a gap means a message was lost or applied out of order, and the local book has
silently diverged from the exchange's. Zero gaps means every diff chained onto
the previous one. The full documented sync procedure is implemented: buffer the
diff stream before fetching the REST snapshot, discard events older than it,
and verify the first applied event straddles the snapshot id.
Zero drops at 10.5 msg/s is unsurprising — the ring holds 8192 messages,
about 13 minutes of headroom at this rate. That is the honest reading: this feed
does not stress the ring, it validates correctness under real conditions. The
examples/feed.cpp synthetic producer is what actually saturates it (217 M
ticks/s, 10.6% dropped).
The p50 of 135 µs is not ring latency. BM_PushPopSingleThread puts the
ring's own cost at 0.54 ns — roughly 250,000x smaller. What is being measured is
almost entirely the consumer's wait policy: messages arrive every ~100 ms, so
the consumer is usually asleep when one lands, and it only notices on its next
wake-up.
That is visible in the numbers. An earlier version slept immediately on an empty ring:
| Consumer wait policy | p50 | p99 |
|---|---|---|
sleep_for(200us) immediately |
143.6 µs | 61,323.9 µs |
| Spin ~2000 iterations, then sleep | 135.2 µs | 332.4 µs |
184x better p99 from changing the wait policy alone, with the ring untouched. The p50 barely moved, because the median message was always going to find the consumer asleep — spinning fixes the tail, not the middle.
This is exactly why pop() does not block. Had the waiting been baked into the
ring, this tradeoff would not be the caller's to make. The occasional multi-
hundred-millisecond max is WSL2 descheduling the process, the same effect
measured elsewhere as a ~74 µs sleep(0).
- Strictly SPSC. One producer thread, one consumer thread. More of either corrupts the buffer silently. An MPMC ring is a substantially different design (CAS loops, per-slot sequence numbers).
- Bounded, and drops on overflow. By design. If you cannot afford to lose
items, size the ring for your worst burst and monitor
dropped(). - No blocking pop. Deliberate — see Overflow policy. Waiting is the caller's business.
Tmust be nothrow-move-constructible. A throwing move partway through an index update would leave the ring inconsistent.- Capacity fixed at compile time. The power-of-two mask and inline storage depend on it.