Skip to content

Add allocation-free byte overloads, and complete queued calls with no work item for late continuations - #2

Merged
XeonFX merged 8 commits into
mainfrom
fix/completion-race-span-overloads
Oct 20, 2026
Merged

XeonFX merged 8 commits into
mainfrom
fix/completion-race-span-overloads

Conversation

@XeonFX

@XeonFX XeonFX commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

How queued calls complete

BcryptQueue's pooled requests now complete through an internal CompletionCore instead of ManualResetValueTaskSourceCore. A continuation registered just after completion (the result arrived between the awaiter's look and its registration) runs at once on the registering thread instead of being queued as a pool work item. That work item is an allocation, so this is part of the 0 B per call. The core also claims the continuation slot before it marks the result ready and touches nothing after, so it has no completion window by construction. The flags are still honoured (ExecutionContext, synchronization context, task scheduler), and completions still run in Execute on the pool, never on a bcrypt worker.

1.0.1 was not exposed to a completion race. The runtime core marks the result ready before it claims the slot, but its GetStatus stays Pending until the slot is claimed (checked on .NET 10.0.13), and 1.0.1's request releases only once its status isn't Pending. 20 s of oversubscribed callers against the published 1.0.1 package, reading at once and awaiting, gave 14,454 rounds with no failure. This was the only ManualResetValueTaskSourceCore in the repo.

Byte overloads

A server reads the password as bytes and may keep hashes as bytes. These skip both strings and allocate nothing:

  • Bcrypt.Verify(ReadOnlySpan<byte> passwordUtf8, ReadOnlySpan<byte> hash)
  • Bcrypt.Hash(ReadOnlySpan<byte> passwordUtf8, int cost, Span<byte> destination), which returns the bytes written (60, Bcrypt.HashLength)
  • BcryptQueue.VerifyAsync(ReadOnlySpan<byte>, ReadOnlySpan<byte>, CancellationToken) → ValueTask<bool>
  • BcryptQueue.HashAsync(ReadOnlySpan<byte>, int cost, CancellationToken) → ValueTask<BcryptHash>; BcryptHash holds the 60 bytes inline, with CopyTo, TryFormat (bytes or chars), AsSpan and ToString
  • BcryptQueue.TryVerifyAsync → ValueTask<BcryptStatus> and TryHashAsync → ValueTask<BcryptHashResult>: a full queue is Busy and a fired token Cancelled, with no exception and no allocation

These follow the contest server's own byte API (PasswordCheck, PasswordHash, TryEnqueue in csharp-kestrel-xeonfx), fitted to the package's names. The queue keeps its own lane batching, window and cancellation, which ends a waiting call at once instead of when a worker reaches it. Two things came from the server: BcryptHashResult.CopyTo, which throws when there is no hash, and slot taking by compare-and-swap, so a refused call never briefly holds a slot that another call could use.

The queue copies both spans before the call returns. The password's bytes are the key as they are: UTF-8 gives the string overloads' hashes, and other bytes are accepted as OpenBSD and Go accept them. Over 72 bytes is refused, NUL is allowed, and a malformed hash is false without queueing. The string path now also copies the stored hash's 31 checksum characters into the request and formats new hashes into bytes, making the string on the caller's thread. Every existing API keeps its behaviour. The one source-level change: HashAsync(null, ...) with a literal null is now ambiguous at compile time.

Version 1.1.0, with a README section. Nothing is tagged or published.

Testing

  • CompletionTests pauses the completing thread right after the result is ready (an internal test hook) while the caller reads the result, frees the request and queues its next call on it. It pins CompletionCore's order (claim the slot, mark ready, then nothing) and goes red when that order is reversed. An oversubscribed stress test runs callers that read and reuse requests at once. Further tests check that a late continuation runs inline, that a synchronization context gets the continuation, and that OnCompleted flows the ExecutionContext.
  • A second await of a call still running throws without touching the first await's state (the first await had hung in an earlier version of this branch; the test was red before the fix).
  • The server's scenarios, ported: callers in one batch each get their own answer, malformed hashes are answered at once, a flood past the limit is exactly the excess as Busy, a stale read doesn't pool a request twice, one changed character in a stored hash or its salt doesn't match, a call older than the window runs without waiting again, and 200 logins plus 25 sign-ups allocate 0 B over the whole process.
  • Differential tests against the string API and BCrypt.Net-Next: edge passwords (empty, NUL, exactly 72 and 73 bytes, multi-byte characters ending at byte 72 or 73), an FsCheck property over random passwords with $2a$, $2b$ and $2y$ hashes, random multi-byte text through the queue, the published vectors (including Openwall's and the non-UTF-8 ones, now checked through the public byte API), the original $2$ revision, and malformed hashes (bytes past ASCII, too long or short, stray checksum bits, $2x$, bad cost).
  • Concurrency: buffers overwritten right after the call returns, cancellation before a worker takes the call (and during linking), a full queue answered as Busy, Dispose, and every kind of call sharing the pool under load.
  • Allocation: 0 B per call once warm for each new overload, including Busy and an already-cancelled token. The queue ones are measured over the whole process.
  • Each test was shown to go red by breaking the code on purpose: the core's order reversed, ignored scheduling or execution context, a queued late continuation, a Try cancellation that throws, Busy as an exception, a 72-byte password refused, the stored hash not copied.
  • Benchmarks ([MemoryDiagnoser]) next to the existing ones: VerifyUtf8, Queue5Utf8, HashUtf8, QueueHash and QueueHashUtf8. ShortRun at cost 10 on an M-series Mac: Hash 64.5 ms and 144 B, HashUtf8 64.1 ms and none; QueueHash 207 B, QueueHashUtf8 63 B (the remainder is harness noise that the string Queue5 shows too).
  • All 299 tests pass locally on the net10.0 build, and on the net8.0 and net9.0 builds rolled forward to the .NET 10 runtime (the Mac has only .NET 10). CI runs .NET 8, 9, 10 and 11 on x64 and arm64.

BcryptQueue's requests completed through ManualResetValueTaskSourceCore, which marks a
result ready before it claims the continuation slot. A caller that read the result in that
window released the request, and the completion then landed in its next call. CompletionCore
claims the slot first and touches nothing after; a late continuation runs inline.

New: Bcrypt.Verify/Hash over bytes, BcryptQueue.VerifyAsync/HashAsync over bytes,
TryVerifyAsync/TryHashAsync with a BcryptStatus instead of exceptions, and BcryptHash,
which holds a hash's 60 bytes inline. Version 1.1.0.
…l revision for real, bound the Busy test's wait
…ee, CopyTo on the hash result, its test scenarios
…t the docs on late continuations, NUL bytes and Dispose
The queue reads arrival and window times from an internal TimeProvider (the system clock
outside the tests). The test moves the clock past a 30 s window between stamping a call's
arrival and linking it, so the worker always finds it too old to wait for. It failed CI on
a loaded runner with its 150 ms bound.
The runtime core's GetStatus stays Pending until its continuation slot is claimed, and the
queue releases a request only once it isn't Pending, so 1.0.1 never reached the window.
What the new core gives is a late continuation run inline instead of a pool work item (an
allocation), and no window by construction. The paused test is named for that order.
@XeonFX XeonFX changed the title Fix BcryptQueue's completion race and add allocation-free byte overloads Add allocation-free byte overloads, and complete queued calls with no work item for late continuations Oct 20, 2026
…e least

A single window failed once in CI on .NET 11 with 1,160 B of the process's own allocations.
Noise only ever adds, while an allocation per call shows in every window, as the queue's
other allocation tests already assume. A failure now lists every window's bytes.
@XeonFX
XeonFX marked this pull request as ready for review October 20, 2026 09:09
@XeonFX
XeonFX merged commit df2c1b0 into main Oct 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant