Skip to content

refactor: dedupe the R2 witness adapters and hoist the jsonrpsee mock scaffolding - #192

Open
flyq wants to merge 3 commits into
mainfrom
liquan/refactor/r2-adapter-and-mock-hoist
Open

refactor: dedupe the R2 witness adapters and hoist the jsonrpsee mock scaffolding#192
flyq wants to merge 3 commits into
mainfrom
liquan/refactor/r2-adapter-and-mock-hoist

Conversation

@flyq

@flyq flyq commented Aug 27, 2026

Copy link
Copy Markdown
Member

Summary

Two queued deduplications, in one PR:

  • R2 witness adapters (follow-up from the feat(debug-trace-server): direct-from-R2 historical witnesses #175 review round — "Happy for this to be a follow-up"): the shared core of the two binaries' r2_witness.rs — the failure taxonomy (R2WitnessError, KINDS, kind()), the BackoffPolicy → RetryPacing conversion, transport construction, the target accessors, the blocking-decode error mapping, and the kinds_cover_all_fetch_kinds guard test (two copies protecting the same surface from the same drift) — now lives once in stateless_common::r2_witness. Transport construction happens at the two wiring sites that already own the flags and the metrics plumbing (main.rs / app.rs), which log the configured target straight off the transport; each adapter keeps a one-line new(transport) plus what genuinely differs: light-vs-full decode, deadline handling, retry budgets (3 vs 9), pacing pauses, and metrics hooks. New stateless-common → stateless-r2 dependency edge; acyclic, stateless-r2 stays a leaf crate.
  • jsonrpsee mock scaffolding: three crates hand-rolled the same mock-upstream scaffold (ephemeral server bootstrap, minimal RPC header shapes, hex block-number parse). stateless_test_utils::mock_rpc now provides serve / serve_with_config, header_stub / consistent_header, and parse_hex_u64 — the mock_r2 precedent — and the specialized mocks in stateless-common, debug-trace-server, and the validator's integration tests build on it instead of re-rolling it. It sits behind an opt-in mock-rpc feature so stateless-core and stateless-r2 — dev dependents with no other jsonrpsee edge — do not compile the server stack for tests that never mock an upstream.

Behavior is pinned by the existing tests

This is a refactor; the discriminating tests are the ones that already existed, unchanged and green: the validator's pacing-pause tests (deterministic throttle + max-backoff pause before surfacing), both adapters' MAX_ATTEMPTS tests, the construction-rejection tests on both targets, the custom-domain bare-key/unsigned-GET assertions, and the cap-wiring test from #191 (the_r2_cap_not_the_rpc_one_reaches_the_transport, still asserting on both target arms that --witness-max-concurrent-requests 16 --r2-max-concurrent-requests 48 yields a cap of 48, so reverting either arm's wiring fails by value). Three per-binary tests collapse into one shared copy each — the KINDS guard and the two construction-rejection tests — which is the workspace count going 478 → 477.

The one observable change

The shared error union carries DecodeTimeout for both consumers, so the validator now pre-registers an r2_witness_errors_total{kind="decode_timeout"} series it can never produce (its fetches pass no deadline) — the same structurally-unreachable status as its existing deadline kind, now documented on the shared type.

Two deliberate deviations from the original dedup notes

  • header_stub is not unified onto the self-consistent hash_slow() shape: the divergence-bisection tests must serve prescribed hashes (two chains differing only by hash), which a number-derived hash cannot express. Both shapes ship, each documenting which verify_hash paths accept it.
  • serve_with_config exists alongside serve because the validator's fixture-backed mock needs max_response_body_size above jsonrpsee's default for its multi-MB witness payloads.

Testing

cargo test --workspace: 477 passed, 0 failed. fmt / clippy --all-targets --all-features / cargo sort clean; cargo check -p stateless-test-utils standalone (the crate's own build constraint) clean, and cargo tree -p stateless-core --edges dev -i jsonrpsee now resolves to nothing.

🤖 Generated with Claude Code

flyq and others added 2 commits August 27, 2026 19:22
The two binaries' r2_witness.rs were near-verbatim copies on everything
below the decode: the failure taxonomy (enum, KINDS, kind()), the
BackoffPolicy->RetryPacing conversion, both constructors, the target
accessors, and the kinds_cover_all_fetch_kinds guard test - two guards
protecting the same surface from the same drift. That shared core now
lives in stateless_common::r2_witness (R2WitnessError + a
R2WitnessTransport wrapper; the custom-domain constructor takes each
binary's version-observer as a callback), and the adapters keep only
what genuinely differs: light-vs-full decode, deadline handling, retry
budgets, pacing pauses, and metrics hooks.

The error union carries DecodeTimeout for both consumers; a binary whose
fetches pass no deadline (the validator) never produces it, so its
pre-registered decode_timeout series stays at zero - the same status as
its existing structurally-unreachable deadline kind, now documented on
the shared type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tils

Three crates hand-rolled the same mock-upstream scaffold: the ephemeral
jsonrpsee server bootstrap, the minimal RPC header shapes, and the hex
block-number parse. They now live in stateless_test_utils::mock_rpc
(serve / serve_with_config, header_stub / consistent_header,
parse_hex_u64), following the mock_r2 precedent; the specialized mocks
(scripted witness sources, counting endpoints, the validator's
fixture-backed node) stay with the tests that script them, built on the
shared scaffold.

Both header shapes are kept deliberately: divergence-chain tests must
serve prescribed hashes (header_stub), while verify_hash = true paths
need the real hash_slow() (consistent_header) - collapsing them would
break one consumer or the other. serve_with_config carries the
response-size cap the validator's multi-MB witness fixtures need.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mega-maxwell

mega-maxwell Bot commented Aug 27, 2026

Copy link
Copy Markdown

Claude review status

Living comment — rewritten in place. The review workflow keeps this single comment up to date instead of posting a new one each round, so it always describes the latest reviewed commit and the earlier text is intentionally gone. No reply is needed here; reply to a finding in its own review thread, and answer an open question in a reply on this PR. The next review round reconciles your answer.

🛠️ Review did not finish

Attempted 313ed680..44b9ee68 · updated 2026-08-27T15:41:08+00:00

This round did not publish: MODEL_ACTION_FAILED in phase review_retry. Anything listed below is from the last round that did. Re-run the workflow or push a new commit to try again.

The first pass hoisted the shared core but left both binaries with
byte-identical delegation shells over it: four forwarding constructors
and seven accessor hops, re-creating one level up the duplication the
hoist set out to remove. Transport construction now happens at the two
wiring sites that already own the flags and the metrics plumbing
(main.rs / app.rs), which log the configured target straight off the
transport; each adapter keeps a one-line `new(transport)` and its own
fetch-decode-pace logic. app.rs's builder is renamed to
build_r2_transport for what it now returns.

The duplicated blocking-decode mapping goes the same way: a shared
decode_on_blocking_pool takes the decode fn and an optional deadline
(Some bounds it for the request path, None lets the pipeline's decode
finish), so the shared error type's Decode/DecodeTimeout/DecodePanicked
variants are constructed once, in the module that defines them.

Both construction-error tests move to the shared module, where one copy
covers both binaries and both targets — the rationale that already moved
kinds_cover_all_fetch_kinds. The cap-wiring test keeps its discriminating
assertion, now reading the cap off the transport the builder returns.

Also from the review: jsonrpsee becomes an optional test-utils dependency
behind a mock-rpc feature, so stateless-core and stateless-r2 — dev
dependents with no other jsonrpsee edge — no longer compile the server
stack for tests that never mock an upstream; is_missing's doc drops the
one-consumer framing; and the consistent_header re-export hop is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants