Testing & CI hardening phase 3: union × protocol conformance cells, coverage tooling (#48) - #55
Merged
Merged
Conversation
…overage tooling Continues issue #48 after #53 and #54: - Union x {rpcv2Cbor, jsonRpc2} conformance (protocol-tests/unions/): union round-tripping was only pinned for simpleRestJson; the other two cells relied on the seeded random integration tests, which flip a coin on whether the union appears at all and can only prove serde self-consistency — a symmetric encode/decode bug passes them. The new suites pin the wire subdocument for every SinkChoice variant deterministically, in all four directions per protocol (client encode, client decode, server decode, server echo), plus the reject cells (empty, multi-member, unknown-member, null-member, non-map unions), the exactly-one-member wire invariant, and the __type discriminator tolerance. Both suites were mutation-checked: one expectation flipped per suite fails the suite, so a first-run pass is not vacuous. - Code-coverage tooling (issue #48's "module gaps only findable by manual srcs<->tests correspondence"): a coverage CI job runs bazel coverage with a combined lcov report over the runtime, prints the per-module summary in the log, and uploads the rendered HTML as a build artifact; `make coverage` runs the same locally. Measurement only, no threshold gate yet — same posture as the benchmarks job. Verified locally: bazel test //... green (74 tests; Boost/benchmark targets excluded per docs/development.md's proxy note, CI covers them), and bazel coverage --combined_report=lcov produces a 21-file runtime report. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012VFUppN1idpmU1UrbHkknZ
5 tasks
aaylward
pushed a commit
that referenced
this pull request
Jul 8, 2026
… pinning, macOS sanitizers Continues issue #48 (after #53, #54, #55) and works through issue #56: - Union member-type gauntlet (protocol-tests/unions/): SinkChoice only covers string/int/struct members, so a generated-in-graph UnionGauntlet service (same Bazel path as codegen/compile-tests) extends the union x {rpcv2Cbor, jsonRpc2} cells to blob, timestamp, list, map, enum, intEnum, and recursive-struct members. Each member kind is pinned with a hand-built typed value on one side and a hand-built wire document on the other — never both ends produced by the code under test — through client encode, the full client<->server loopback loop, and the wire subdocument. Protocol-specific representations are pinned where they differ (blobs as base64 text and timestamps as epoch-seconds numbers in JSON; blob and tag-1 timestamp nodes in CBOR). - Byte-exact request vectors: the CBOR body pinned as hex derived by hand from RFC 8949's deterministic encoding rules, and the full JSON-RPC envelope pinned as its exact compact sorted-key text — closing the seam Document-level equality leaves between "codec is deterministic" and "serde feeds the codec what we think" (issue #56). - Error-shape union cell: a modeled error whose union member rides next to its __type discriminator, inspected on the wire (the payload keeps both) and round-tripped into the typed error detail on the client — the case the serde's exactly-one-member arithmetic exists to tolerate. That contract is now documented in docs/generated-types.md, giving the tolerance test an authority beyond the implementation (issue #56). - Reject-cell diagnosis pinning (issue #56): the phase-3 union reject cells now assert which rule was violated ("expected exactly one union member", "unknown or missing union member", "expected a map on the wire") on both client and server, not merely that a rejection happened. - Sanitizers x platforms (issue #48): the asan+ubsan CI job is now a linux/clang + macos/apple-clang matrix, putting the transport layer's Apple-specific paths (SO_NOSIGPIPE, libc++) under a sanitizer for the first time. MSVC ASan under Bazel remains future work. Also confirmed while scoping: the Gradle toolchain works in this environment (gradle build + the Java unit suite pass), so the generator-class unit-test item is unblocked for a next phase. Model note: recursion *through* a union (Node carrying a BigUnion) is rejected by the generator with a clear diagnostic (std::variant needs complete alternatives); the gauntlet documents that boundary and covers a self-recursive struct as a union member instead. Verified locally: bazel test //... green (76 tests; Boost/benchmark targets excluded per docs/development.md's proxy note, CI covers them). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012VFUppN1idpmU1UrbHkknZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Phase 3 of #48 (after #53 and #54):
Union × {rpcv2Cbor, jsonRpc2} conformance cells (
protocol-tests/unions/). Union round-tripping was only pinned for simpleRestJson; the other two cells relied on the seeded random integration tests, which flip a coin on whether the union appears at all and can only prove serde self-consistency — a symmetric encode/decode bug sails through them. The new suites pin the wire subdocument for everySinkChoicevariant deterministically, in all four directions per protocol:resultand answering the requestid).Plus the reject cells — empty, multi-member, unknown-member, null-member, and non-map unions fail on both client and server before any handler runs — and the
__typediscriminator tolerance next to a member.Following up on the review discussion about first-run-green banks: both suites were mutation-checked — flipping one expectation per suite (a union value, the echoed envelope id) makes that suite fail, so the first-run pass is not vacuous.
Code-coverage tooling (#48's "module gaps only findable by manual srcs↔tests correspondence"). A
coverageCI job runsbazel coverage --combined_report=lcovinstrumented over//runtime, prints the per-modulelcov --summary/--listin the log, and uploads the rendered HTML report as a build artifact.make coverageruns the same locally and prints the report path. Measurement only — no threshold gate yet, same posture as the benchmarks job.Remaining on #48: golden self-ratification, generator-class unit tests, HTTP-parser fuzzing for the Beast transport, benchmark gating, cross-platform sanitizers.
Testing
bazel test //...green locally — 74 tests (the two new union suites on top of phase 2's 72), Boost/Beast targets and//benchmarks/...excluded perdocs/development.md's proxy note; CI covers those.bazel coverage //runtime:core_test //runtime:cbor_test --combined_report=lcovverified locally to produce a combined report with 21 runtime source files at repo-relative paths.generated/trees).Checklist
bazel test //...passes locally (Boost/benchmark targets excluded per docs; CI covers them); codegen untouched, gradle job unaffected🤖 Generated with Claude Code
https://claude.ai/code/session_012VFUppN1idpmU1UrbHkknZ
Generated by Claude Code