Wallet interoperability and reliability: bounded sync, proof recovery and BRC-118 payments

[ty-everett]

Wallet synchronization currently holds manager ownership across a whole transfer, so slow source I/O blocks foreground wallet work. Large HTTP payments also exceed practical header limits, and stale proof records can repeatedly poison otherwise spendable funding. This PR delivers a bounded resumable local sync path, canonical proof recovery and a complete prepared BRC-118 client/server payment path, with compatible PeerPay base64 reception.

Program and scope

• Fixes fix(wallet-toolbox): WERR_NETWORK_CHAIN is defined and deserialized but never thrown #541, fixes fix(wallet-toolbox): ProcessSyncChunkResult.error is declared but never set or read #542, fixes BRC-118 multipart body transport is unimplemented, and header-carried payments fail at 32KB behind Cloudflare #473. Refs proposal: bounded wallet sync scheduling, concurrent reads, and portable snapshot performance #544, wallet-toolbox: a stale merkle proof (orphaned block) in proven_txs makes every createAction that selects that coin fail permanently with WERR_INTERNAL: merged Beef failed validation. #545, fix: PeerPayClient rejects BRC-29's documented base64 transaction, and no size check precedes broadcast #548, [QA] Complete deferred coverage, fuzzing, conformance, and runtime validation #400 and [Maintenance] Validate Sonar administration and continue CI efficiency review #402.
• syncFromReaderResumable yields ownership around source/proof I/O, commits one page and checkpoint atomically, reports progress, supports safe cancellation/resume and fences destination changes. Its new default is 256 KiB rough pages; existing sync APIs/defaults remain. Remote/custom providers retain safe exclusive fallback.
• Both sync directions preserve public network/error contracts and stop before acknowledging returned provider errors or non-progressing pages.
• Selected-change, outgoing and monitor proof recovery validates canonical evidence, bounds parallel work and SQL lookup batches, and uses atomic compare-and-set so delayed reproof cannot overwrite newer records.
• SDK AuthFetch prepares the actual transaction/request before broadcast, negotiates header/multipart, authenticates exact multipart bytes, preserves application payloads and reuses payment state across retries. Opt-in raw auth and payment middleware provide the complete receiver path, with proxy/CORS/replay/cancellation tests. BRC-29 recipient derivation is corrected using independently rooted wallets.
• The now-published fix: CommonJS builds bind @bsv/sdk default imports to whole modules (#571) #577 templates import repair is preserved verbatim; real packed CJS/ESM consumers execute five synthetic spends and preserve matching script hashes on SDK 2.8.0 and 2.9.0. The published SDK fixture is pinned by size and SHA-512; a tampered tarball is rejected before installation.
• PeerPay accepts strict bounded base64 at the receive boundary alongside deployed byte representations. Sender encoding is unchanged.
• Coherent source snapshots, streaming BRC-38/39 archives, future wallet adoption, final encrypted MessageBox relay preflight and durable monetary outcome journals remain separate milestones. proposal: bounded wallet sync scheduling, concurrent reads, and portable snapshot performance #544/fix: PeerPayClient rejects BRC-29's documented base64 transaction, and no size check precedes broadcast #548 remain open. Contributor PRs Fix shared overlay lookup recovery and failure reporting #511/feat(group-messaging): add @bsv/group-messaging #546/feat(eqc): BRC-178 economic query client and overlay-express registerRouter hook #549/fix(wallet-toolbox): remove unused WERR_NETWORK_CHAIN and ProcessSyncChunkResult.error #551/chore(deps): Bump the "monthly-stack-maintenance" group with 6 updates across multiple ecosystems #564 are not merged or closed by this PR; fix(wallet-toolbox): remove unused WERR_NETWORK_CHAIN and ProcessSyncChunkResult.error #551's motivating contracts are preserved rather than deleting its public APIs.
• Current main integration: c8820a4f7 via merge commit f02333a3b (after the earlier 110aae46f integration in a4d625ade); the branch retains the published SDK 2.8.1/2.8.2, packaging fix: CommonJS builds bind @bsv/sdk default imports to whole modules (#571) #577 and Toolbox fix(wallet-toolbox): basket authorization, sendMax permissions, Monitor without push events, mobile exports #579 repairs. The SDK fix(sdk): preserve BRC100 discovery and signed action history #587 discovery/browser-fetch and signed-history fixes and MessageBox fix(messagebox): migrate durable HTTP authentication replay claims #588 replay-table migration plus fix(messagebox): consume SDK 2.8.3 and document proxy contracts #589 published SDK dependency/proxy guidance are retained exactly; review-update head f02333a3b is fully validated, with terminal successful hosted evidence below.

Impact

• Public package source or manifest changed
• Additive public API/types and browser/mobile behavior changed
• Security-sensitive authentication, payment and persistence boundaries changed
• Documentation/examples changed
• This PR adds no infrastructure source/deployment changes beyond refreshed main. Current wallet-release pins and published packages/images are not changed by this task.

Unpublished candidates: SDK 2.9.0; Toolbox core/client/mobile 2.15.0; auth middleware 2.3.0; payment middleware 2.2.0; MessageBox client 2.6.0; documentation-only 402-pay 0.3.3; templates 1.10.4. Packed dependency-range changes require patches for amountinator 2.1.7, did-client 1.3.4, fund-wallet 1.5.3, overlay-discovery-services 2.2.6, overlay-express 2.7.3, overlay-topics 1.8.5, simple 0.6.1 and wallet-helper 0.1.9. AuthSocket client 2.1.8 refreshes changed UMD bundle bytes after the SDK integration; its source API and event serialization remain unchanged. The release ledger preserves published baselines, including verified SDK 2.8.3 from protected release 35950850386.

Review update

• Addresses sirdeggen's empty-request-body finding: both dense byte arrays and Uint8Array/Buffer now use the BRC-104 -1 sentinel after express.raw, matching AuthFetch. Nineteen new cases cover byte helpers, actual POST/PUT authentication and signed responses, absent/nonempty bodies, in-transit tampering, and copied payload framing independent of shadowed byte-view length metadata. Both reproduced failures before the fixes.
• Empty binary clients/receivers that used length 0 require paired SDK 2.9.0/auth middleware 2.3.0 adoption. Nonempty preimages remain unchanged; README, guide and migration ledger document the boundary explicitly.
• Registry verification found SDK 2.8.3, Toolbox core/client/mobile 2.14.0, templates 1.10.3, DID client 1.3.3 and MessageBox 2.5.3 published. Candidate versions and baselines are reconciled without reusing a published version for new bytes.
• Storage forwarding now shares the reader/writer authorization paths while preserving their original queue-admission timing and rejection/ownership semantics. Chunk normalization uses the existing entity list and uses a Map of property descriptors, removing the intermediate mapping array while preserving in-place record identity and batch property updates. New tests pin transport failure identity, queue release, authorization timing, all entity arrays, prototype-sensitive keys and non-enumerable date fields.
• Mobile validation uses the official Node 24.18.0 runtime after identifying a Homebrew zlib difference. Identical Metro bytes produce identical Hermes bytes on Linux/x86_64 and macOS/ARM, with bytecode reproducible across build directories. The maintainer explicitly approved the documented Hermes Brotli cap update from 1,520,000 to 1,675,000 bytes; other five caps remain unchanged. The full gate passes Metro 2,377,408 / 609,909 / 463,868 and Hermes 4,619,927 / 1,955,135 / 1,523,206 raw/gzip/Brotli bytes. The mobile README records upstream composition comparison and 9.97% headroom. No production dependency, minifier, export, source-map or compression setting changed.

Verification

Integrated-graph validation uses official Node 24.18.0 / pnpm 10.33.2; earlier completed workspace evidence used Node 24.19.0 where recorded:

• pnpm build, pnpm typecheck, pnpm lint, pnpm format:check, pnpm health:check: pass; health has zero contract findings/control errors. pnpm audit:security: no known vulnerabilities. Frozen install passes.
• Full workspace tests pass on the integrated graph. Final Toolbox coverage passes 288 suites / 3,151 tests, with one unchanged governed skip. No new skip or relaxed threshold. AuthSocket adds 56 passing tests and fresh exact-tarball/browser acceptance.
• SDK coverage: 213 suites / 7,426 tests, 94.28% statements, 87.87% branches, 95.40% lines. Auth: 226 tests, 96.66% lines. Payment: 104 tests after final boundary regressions (the preceding 103-test coverage run has 97.33% lines). MessageBox client: 405 tests; integrated MessageBox service: 227 tests. Mobile: 48 tests, 100% of its governed coverage surface.
• Full TypeScript conformance: 6,494 vectors plus two runner checks pass, same 211 governed skips. Six new BRC-118 exact-body/Content-Type/preimage vectors also pass an independently written Python stdlib reader/writer pinned to BRCs 2b959b13f1f73040d13cc4eb14edbfc376f8010b.
• Relevant SDK/auth/payment/MessageBox/Toolbox property commands pass. Governed mutation: sdk-auth-http 93.08%, auth-express-bytes 90.16%, payment-replay 96.30%; zero uncovered/invalid mutants. The mutation run exposed cancellation-test gaps that were repaired without changing the gate.
• All 18 candidate packed-consumer checks pass. Templates additionally passes 150 tests; overlay-topics passes 774 tests with one unchanged governed todo. SDK exact-tarball browser, Toolbox Vite/esbuild/native IndexedDB, and portable payment bytes without Buffer/FormData/text-codec globals pass. Metro/Hermes acceptance passes the explicitly reviewed source budget.
• Real HTTP SDK/auth/payment composition covers enforced 6 KiB proxy headers, body refusal, exact binary/JSON/nested payloads, signature/payment tampering, replay and uncertain settlement. Native Chromium adds actual cross-origin preflights and signed-header exposure failures. The integrity-verified published SDK 2.8.0 client passes the nonempty legacy-payment probes against both new receiver modes; empty binary requests require the paired upgrade above.
• pnpm docs:examples: 8 examples / 21 exact tarballs pass. Docs-site build/link checks pass. Current API pages, migration/release notes and explicit next-stage design are included.
• Local createAction proof/funding, action-batch and existing storage-sync benchmark gates pass. Existing external PXC benchmark cohorts were not run by the default local invocation; their environment requirement remains unchanged.

Fresh review-update performance fixture on f02333a3b: 10,000 same-timestamp labels with tombstones, 64 x 32 KiB proofs and related transactions, 15 ms source latency, 256 KiB page target. Sequential measurements compare this branch's exclusive fallback with paged mode:

Backend	Full-copy ms, exclusive / paged	Foreground p95 ms, exclusive / paged	Paged event-loop p95 ms
Native Chromium IDB	4,018 / 3,945	4,013.30 / 22.90	1.00
SQLite	2,292 / 2,302	2,264.19 / 0.31	82.90
Authenticated HTTP → SQLite	23,270 / 21,009	23,177.20 / 0.42	197.86

Queue responsiveness does not eliminate synchronous HTTP crypto/encoding cost. Inclusive timestamp replay deliberately remains 45/47 pages for the all-tied fixture; settled unchanged copies use two. The guide records CPU, memory, wire/query counts, queue/commit measurements and reproducibility limits. These are fixture results, not device-wide latency guarantees.

• Self-reviewed correctness, security, compatibility, public API, artifacts, dependencies, docs and operations, with adversarial fault cases
• All final review-update hosted checks are terminal and successful
• Prior-head acceptance is historical evidence. The integrated head has complete CI, Codecov, CodeQL and strict Sonar success. Review replies link the pushed fixes and passing evidence; every conversation is resolved.

Security and dependencies

• Added only development references to already-resolved esbuild 0.28.1 and puppeteer-core 25.4.0 for native-browser acceptance, and express-rate-limit 8.6.1 to bound the real HTTP test fixture. The lock gains importer references; no new resolved transitive packages, production dependencies, overrides or lifecycle-script permissions.
• Reviewed upstream esbuild release and Puppeteer release; and express-rate-limit changelog; existing repository versions are reused. Browser selection uses installed Chrome/Chromium, with an explicit CHROME_BIN override.
• Runtime/peer compatibility, deduplicated frozen graph, audit and packed consumers reviewed
• New negative tests cover modified trust boundaries
• No new quality suppression, advisory dismissal or skipped test
• Exact-head CodeQL has no new alert
• Exact-head repository quality gate has zero new Sonar findings and unreviewed hotspots
• Workflow permissions and lifecycle-script behavior remain least privilege

Dependency evidence

• Release notes and necessity: The upstream release/changelog links above were reviewed. Existing esbuild/Puppeteer versions support native browser acceptance; existing express-rate-limit bounds the real HTTP fixture. The coverage aggregator now provisions its already-declared root compiler so type-only edits can be classified correctly.
• Runtime, build, and peer compatibility: Node 24 builds and clean ESM/CJS consumers pass. Auth/payment peers explicitly require SDK ^2.9.0 for the new helpers; the released SDK 2.8.0 browser client passes the nonempty legacy-payment probes against both receiver modes. Empty binary requests require the paired client/receiver correction. Templates retains its SDK ^2.1.6 peer range and uses category exports already present at that floor.
• Deduplicated lockfile: Frozen install passes. Only importer references are added for these development tools; no new resolved package, override, transitive cohort or production dependency is introduced.
• Audit and CodeQL: Dependency review and local high/critical audit pass with no known vulnerability; previous exact-head CodeQL alerts were fixed without dismissal. The final head passes CodeQL with zero open alerts.
• Package and consumer tests: The 18 candidate packed checks, all affected package/dependent tests, published-SDK consumer probes and 8 documentation examples against 21 exact tarballs pass locally; final-head hosted validation passes.
• Bundle and performance impact: SDK/browser/templates checks pass; Metro/Hermes acceptance passes the reviewed budget documented above. Templates' packed Vite bundle is 205,177 raw / 66,177 gzip / 54,910 Brotli bytes. Sync evidence and remaining event-loop cost are recorded above and in the guide.
• Affected public package versions: The 18 unpublished candidates are enumerated above and in the release ledger, with published baselines preserved. Current wallet pins and published artifacts are unchanged.

Release and operations

• No npm/image publication, release tags, PR merge or deployment performed
• Correct affected-package SemVer and release notes included
• Documentation and migration guidance current
• BRC-118 receiver enablement is opt-in; empty binary requests require the paired SDK/auth upgrade documented above. No new sync schema or archive format. Later protected publication, service adoption and wallet integration remain separately authorized work.

Completion evidence

• Final exact head reviewed with all applicable checks successful
• Review conversations resolved; no new Sonar/CodeQL findings
• No issue is closed early for an unimplemented milestone
• No failed/pending check is being represented as complete

Final head f02333a3bb1fe1ceb764f1b4a18a4793758902cb: CI / merge gate, CodeQL, and Conformance are terminal successful. All 48 applicable check runs pass, including Codecov patch and strict Sonar; the one scope-based dependent-test skip is accepted by the merge gate. Zero open CodeQL alerts, zero new Sonar issues/hotspots, and all review conversations resolved with evidence. Latest main c8820a4f7 is incorporated, GitHub reports no conflicts, and auto-merge is disabled.

Ready for maintainer re-review; sirdeggen's review has not been dismissed. No PR merge, publication or deployment is performed or authorized by this closeout.

[codecov]

Codecov Report

❌ Patch coverage is 91.06557% with 109 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
packages/sdk/src/auth/clients/AuthFetch.ts	81.75%	9 Missing and 18 partials ⚠️
...wallet-toolbox/src/storage/WalletStorageManager.ts	91.28%	9 Missing and 8 partials ⚠️
...middleware/payment-express-middleware/src/index.ts	84.72%	6 Missing and 5 partials ⚠️
...-toolbox/src/storage/remoting/StorageClientBase.ts	57.69%	11 Missing ⚠️
packages/sdk/src/auth/utils/paymentTransport.ts	91.59%	4 Missing and 6 partials ⚠️
...payment-express-middleware/src/multipartPayment.ts	92.00%	0 Missing and 6 partials ⚠️
...let/wallet-toolbox/src/storage/sync/syncSession.ts	92.68%	0 Missing and 6 partials ⚠️
...et-toolbox/src/storage/methods/repairBeefProofs.ts	92.85%	0 Missing and 4 partials ⚠️
...ware/auth-express-middleware/src/rawRequestBody.ts	96.20%	0 Missing and 3 partials ⚠️
.../wallet-toolbox/src/storage/sync/syncCheckpoint.ts	70.00%	1 Missing and 2 partials ⚠️
... and 8 more

📢 Thoughts on this report? Let us know!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.1% Duplication on New Code

See analysis details on SonarQube Cloud

[ty-everett]

[administrative coordination note] The intention is NOT to merge this PR until after the five wallet releases (BSV Browser, BSV Desktop, Metanet Explorer Mobile, Metanet Client Desktop, and Peacock Wallet) are fully rolled out and in production, validated with users. This is a subsequent, later, follow-on change, designed for after the dust settles around the SDK 2.8.x series.

Until then for this PR we have one mandate on the branch: keep CI green, keep it synced with main, address all review items, hold it open, continue adversarial review, and keep it healthy until AFTER all the wallet / 2.8.x work is done. Ensure its completeness. Likely merge timeline: perhaps Friday the 25th, with explicit human maintainer approval only.

Agents: keep the branch green and free of conflicts meanwhile, so it will not be stale, and also address all review findings, while inviting further review.