diff --git a/ROADMAP.md b/ROADMAP.md index 798f7cf..93135a3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -138,15 +138,21 @@ API. The decision record must compare wheel size, license/attribution, build complexity, sanitizer coverage, syscall overhead, and the unsafe surface. Raw syscalls require correctly implementing the acquire/release ordering on - shared ring heads and tails that liburing already handles. + shared ring heads and tails that liburing already handles. The + [Phase 1 backend decision](docs/phase1-native-ring-decision.md), accepted on + 2026-08-01, selects the statically linked, vendored liburing route and + records the measurements and required follow-up. + - Move request lifetimes into native structs. A request owns every kernel-referenced buffer, sockaddr, msghdr, and iovec. + - Use a refcounted, multi-completion request state machine rather than "release on the first CQE." At minimum it must model submitted, cancel-pending, operation-complete, notification-pending, and released states. `SEND_ZC` normally produces an operation CQE followed by a notification CQE, and asynchronous cancellation can race with a successful operation completion. + - Add a bounded provided-buffer pool with explicit memory accounting and backpressure. Be precise about copying: @@ -158,8 +164,10 @@ API. - Size the CQ explicitly for multishot workloads, detect overflow, drain promptly, and test re-arming when the kernel terminates a multishot operation by clearing `IORING_CQE_F_MORE`. + - Expose a small, GIL-conscious batch API: prepare operations, submit once per loop tick, and reap a batch of completions in one native call. + - Add EINTR handling, runtime opcode/feature probing, and registered-ring-fd support where probing and measurement justify it. diff --git a/docs/phase1-native-ring-decision.md b/docs/phase1-native-ring-decision.md new file mode 100644 index 0000000..c64fcb9 --- /dev/null +++ b/docs/phase1-native-ring-decision.md @@ -0,0 +1,136 @@ +# Phase 1 native ring backend decision + +**Status:** accepted\ +**Date:** 2026-08-01 + +## Decision + +Use the pinned, vendored liburing as the implementation layer for the native +ring and link it statically into the extension. The production module will use +the canonical `_uringcore` name; `_uringcore_liburing` is only the +comparison-spike name. + +The raw-syscall implementation will remain available only until a follow-up +change moves the selected implementation to `_uringcore` and removes the two +experimental backends. The project will not maintain both implementations in +production. + +## Context + +Phase 1 required lifecycle spikes for both viable implementation routes: + +- direct `io_uring_setup` and `mmap` calls; and +- a statically linked, vendored liburing. + +Both spikes expose the same `Ring` construction, introspection, deterministic +close, context-manager, and deallocation behavior. Neither submits or reaps +requests, so the measurements below compare only the lifecycle boundary. In +particular, they cannot establish submit/reap throughput. + +## Measurements + +Measurements were taken from commit `467543c` on CPython 3.12.3, x86-64, +GCC 13.3.0, GNU binutils 2.42, and Linux +6.18.33.2-microsoft-standard-WSL2. The pinned liburing revision was +`f4e42a515cd78c8c9cac2be14222834be5f8df2b` (liburing 2.5). + +### Correctness boundary + +The two extensions have matching tests for: + +- queue-size validation; +- kernel-reported SQ/CQ sizes and features; +- deterministic, idempotent `close()`; +- context-manager cleanup and closed-ring rejection; and +- equivalent public ABI-version reporting. + +This demonstrates equivalent behavior at the spike boundary, not equivalence +for request submission, completion, cancellation, or shared-ring ordering. + +### Wheel contribution + +`uv build --wheel` used the interpreter's normal extension flags, including +`-O2 -g`; the wheel was not stripped. The table reports each wheel member, +not an estimated whole wheel containing only that backend. + +| Route | Wheel member, unpacked | Wheel member, compressed | `strip --strip-unneeded` copy | +| --- | ---: | ---: | ---: | +| Raw syscalls | 35,152 B | 13,099 B | 15,536 B | +| Static liburing | 109,840 B | 45,476 B | 27,792 B | +| Static-liburing increase | 74,688 B | 32,377 B | 12,256 B | + +The complete comparison wheel, which contains both backends and the existing +CFFI extension, was 170,228 bytes compressed. The static extension's dynamic +section lists only `libc.so.6`; it does not require a system `liburing.so` at +runtime. + +The size cost is acceptable for avoiding a substantially larger +correctness-sensitive implementation. Release manylinux and musllinux wheel +sizes must still be recorded when those builds exist. + +### Lifecycle syscalls + +`strace -c` around 1,000 construct/close iterations reported identical calls +for both routes: + +| Syscall | Raw syscalls | Static liburing | +| --- | ---: | ---: | +| `io_uring_setup` | 1,000 | 1,000 | +| `mmap` | 2,080 | 2,080 | +| `munmap` | 2,008 | 2,008 | +| `close` | 1,147 | 1,147 | + +The totals include interpreter startup and imports, but those totals are the +same for both processes. Neither spike calls `io_uring_enter`. Repeated timing +on WSL2 varied by more than 4x within each route, so the observed lifecycle +timings are not used to select a backend. + +Submission and completion benchmarks remain mandatory once the native batch +API exists. liburing helpers are largely inline, but that is not evidence that +their hot-path cost is zero. + +## Tradeoff comparison + +| Criterion | Raw syscalls | Static, vendored liburing | +| --- | --- | --- | +| Wheel size | Smaller | About 32 KB more compressed in this comparison build | +| Runtime dependency | None beyond libc and the kernel ABI | None beyond libc and the kernel ABI | +| Build complexity | Ordinary extension build | Must build and statically link the pinned submodule | +| Source packaging | Self-contained now | Vendored sources are not yet present in the sdist | +| License work | No additional bundled-library notice | MIT notice must remain in binary/source distributions | +| Sanitizers | Extension flags cover all project-owned ring code | liburing must also be rebuilt with matching sanitizer flags | +| Lifecycle syscall count | Identical | Identical | +| Unsafe surface | Project owns mappings and all future shared-ring ordering | liburing owns mappings and established SQ/CQ ordering helpers | +| Maintenance | Must track kernel ABI details directly | Must update and test a pinned upstream dependency | + +The current raw spike is 344 C lines, compared with 256 C lines for the +liburing wrapper. The more important difference is future code: the raw route +would make this project responsible for acquire/release ordering of shared SQ +and CQ heads and tails, ring wrapping, feature-specific layouts, and upstream +kernel ABI evolution. Sanitizers do not prove that this concurrency protocol +is correct. + +liburing already centralizes those rules and is the same abstraction used by +the existing CFFI implementation. Its build and attribution costs are +concrete and bounded. The extra wheel size is small compared with the +correctness and maintenance risk removed from the Phase 1 request core. + +## Required follow-up + +Selecting liburing does not declare the current spike production-ready. The +following work blocks that transition: + +1. Include the pinned liburing source and required headers in the sdist, and + build the archive as part of a source/wheel build instead of assuming that + `libs/src/liburing.a` already exists. +1. Build liburing itself with ASAN/UBSAN flags in sanitizer jobs, then run the + native e2e suite against that instrumented archive. +1. Move the selected implementation behind the canonical `_uringcore` name + and remove the raw backend, duplicate stub, tests, and build configuration. +1. Implement and benchmark the native prepare/submit/reap batch boundary + before making claims about syscall or CPU improvements. +1. Continue with the refcounted multi-completion request state machine only + after the selected ring backend is packaged and sanitizer-clean. + +The pure-Python/CFFI implementation remains the behavioral oracle throughout +this work. diff --git a/docs/phase1-raw-syscall-spike.md b/docs/phase1-raw-syscall-spike.md index 0b86e5b..ecdba21 100644 --- a/docs/phase1-raw-syscall-spike.md +++ b/docs/phase1-raw-syscall-spike.md @@ -16,7 +16,8 @@ into the Python proactor. The CFFI implementation remains the behavioral oracle while the native API is developed. The extension calls the kernel ABI directly and does not link to liburing. -The follow-up static-liburing spike should implement the same lifecycle -boundary. A decision record can then compare measured wheel size, build and -sanitizer complexity, syscall overhead, license obligations, and unsafe -surface before either route becomes the production backend. +The follow-up static-liburing spike implemented the same lifecycle boundary. +The resulting [backend decision](phase1-native-ring-decision.md) selects the +statically linked, vendored liburing route. This raw implementation remains +temporary comparison code until the selected backend moves to the canonical +`_uringcore` module. diff --git a/docs/phase1-static-liburing-spike.md b/docs/phase1-static-liburing-spike.md index af0e71b..ff89ea7 100644 --- a/docs/phase1-static-liburing-spike.md +++ b/docs/phase1-static-liburing-spike.md @@ -12,12 +12,14 @@ MIT license, whose notice is included in the package. Like the raw-syscall spike, this module owns ring initialization and teardown but does not submit or reap operations and is not wired into the Python proactor. The source checkout must configure and build the pinned submodule -before building this experimental extension; packaging the vendored sources -for standalone wheel builds remains part of the route decision. +before building this experimental extension. Packaging the vendored sources +for standalone wheel builds remains required follow-up. -The decision record can now compare the two lifecycle implementations using -the same API and tests. Neither spike is the production backend until that -record selects a route. +The two lifecycle implementations use the same API and tests. The resulting +[backend decision](phase1-native-ring-decision.md) selects the statically +linked, vendored liburing route. This spike is not yet the production backend: +the follow-up work must make source builds self-contained, add full sanitizer +coverage, move this implementation to `_uringcore`, and remove the raw spike. On the initial CPython 3.12 x86-64 development build, including debug information, the module sizes are: @@ -27,7 +29,6 @@ information, the module sizes are: | Raw syscalls | 35,152 bytes | | Static liburing | 109,840 bytes | -The static module adds 74,688 bytes in this build. These are spike -measurements rather than release-wheel results; the decision record must -repeat them with the release build and record its compiler and strip -settings. +The static module adds 74,688 bytes in this build. See the backend decision +for compressed wheel-member sizes, stripped sizes, the measurement +environment, and the other selection criteria.