test(shared-fs): probe split-plane adaptive content custody

[peerbit-org]

Scope

Draft, test-only split metadata/content-plane design and opt-in experiment. No production schema, address, ACL, replication default, dependency, or published package code changes. Empty changeset: no release bump.

The intended boundary keeps all namespace/version metadata local while independently placing content chunks. This prototype uses immutable manifests, not the actual shared-fs namespace or concurrent-write semantics. The design documents migration, uncertain two-log outcomes, exact persisted-entry receipt ordering, source-retirement fencing, authorization/encryption, soft log-byte budgets, and fail-closed physical GC.

Experiment

Five independent disk-backed local processes: publisher plus 3→4→3 custodians, 24 files/42 chunks, experimental N=2, full-control vs unequal-budget adaptive placement. Intended final gates include a demonstrably missing-content read, renewed actual-entry receipts, same-identity offline reopen, local content verification, and natural process exit. Publisher copies never count toward remote custody. No retries or forced successful exits.

Results: both corrected live runs FAILED

Existing loaded library cohort: peerbit 5.3.35 / document 15.0.16 / shared-log 16.0.15, macOS 26.6.2 arm64, Node 24.13.1. This is NOT a test of the newer upstream cohort. No install/dedupe or held-upgrade-worktree mutation.

• Initial full-mode harness attempt failed before writes due to a string[] dial argument; corrected to supported string input. Its raw log is retained separately.
• Corrected full: 16 files acknowledged; all four custodians complete after joining. During subsequent peer loss, chunk put locally committed but persisted delivery timed out. Cleanup exited naturally.
• Adaptive (same corrected source, first attempt): observed 9/11/14 of 14 chunks across three custodians, with all 8 manifests everywhere and at least two observed copies per chunk. After 12 acknowledged files, persisted delivery timed out during the fourth custodian's join. That peer also exceeded the 20-second stop deadline; forced cleanup remains a failure.
• Neither run reached the final missing-content read, renewal barrier, source retirement or offline recovery. No safe-churn, fairness, hard-quota, network-traffic, speedup, or production-readiness claim.

The design document includes exact event timings, entry hashes, source/lockfile/log hashes, loaded package provenance, raw-log locations and reproduction commands. It distinguishes unpublished local commits from acknowledged data loss. Do not merge/enable production adaptive sharding based on these results; rebaseline on the coherent newer upstream cohort first.

Validation

• 12 snapshot-analysis tests passed; 2 live scenarios correctly skipped without opt-in.
• Standalone TypeScript checks for all new TypeScript files passed.
• Library build, explicit lint and formatting passed.
• npm package dry-run: 83 files; no test/experimental files included.
• Independent static audit of receipt ordering, coverage counting, identity checks, deadlines, offline checks and failure propagation.

This draft is authored and published as peerbit-org using the repository .envrc. Main checkout and unrelated worktrees remain unchanged. No release or upstream message sent.

[peerbit-org]

New released-cohort rebaseline: both live modes pass

Published as peerbit-org on experimental branch test/shared-fs-adaptive-rebaseline-20260905, exact head 9b234f988c1732eca6c544624653a7f723fc4556:
complete results, caveats, provenance and reproduction.

This keeps the dependency rebaseline separate from this PR's test-only head. It does not merge the held cohort upgrade or enable production adaptive replication. This PR's existing CI is for its old dependency head, not validation of this new branch.

One plain pnpm install after pinning peerbit 5.4.1 / document 15.0.32 / shared-log 16.0.29 / program 6.0.59 / trusted-network 6.0.130. No dedupe; only the shared-fs library lockfile importer changed. The five measured harness/helper source hashes are unchanged. Both full and adaptive ran once, sequentially, with the original deadlines and no retries.

• Both completed all 24 files/42 chunks, join/loss writes, exact-entry persisted receipt renewal, publisher stop, custodian SIGKILL, and verified same-identity offline reopen with zero connections and natural cleanup.
• Full offline custody: 42/42/42 chunks. Adaptive: 25/42/31; 28 chunks had two verified copies and 14 had three. All three custodians retained all 24 manifests.
• Adaptive proved a missing-local-content read: one actual remote return with full-file verification in 12.95 ms. No forced local replication.
• Median whole-file persisted receipt latency: 125.90 ms full / 139.52 ms adaptive (mean of middle observations, 24 writes); maxima 3.19 s / 3.15 s. Small-sample observations, not tail guarantees or a speedup estimate.
• Adaptive join metadata completed after 3.02 s of sampling, but inventories continued changing for 21.19 s; the stability gate ended at 23.20 s. Full-mode post-loss waiting instead included 10.26 s of metadata lag on an additional peer whose content was already complete. These are separate optimization targets, not a failed two-peer durability promise.

Independent audit confirmed raw report gates, offline identities/bytes, source hashes and phase timings. Additional checks: 12 analysis tests, 33 CLI tests, library/CLI builds, standalone TypeScript, lint/format and package exclusion all pass. No full new-cohort shared-fs suite or cross-OS/N=3/real-host campaign is claimed.

One integration concern remains: blocks-interface@2.2.0 evaluates an empty import of a nested stream-interface@6.0.16, which indirectly loads a separate crypto@3.1.6 class family alongside the library core. Both use the same Borsh instance. This is confirmed resolver/source evidence, not a demonstrated cause of failure; both campaigns pass. Do not claim a duplicate-free runtime or use dedupe as a workaround. An upstream message has been prepared for the user to relay; none was sent directly.

Raw files are retained locally under /private/tmp/peerbit-performance-20260905/; exact checksums are in the report. Old failed campaigns remain preserved. Main checkout and held upgrade worktree remain untouched. No merge or release.

[peerbit-org]

N=3 follow-up is published on experimental branch test/shared-fs-adaptive-n3-profile-20260905, exact head 15f40b98d0c054a11e768837c95bbc5f201d007c:

Complete N=3 results, profile interpretation, source/lock/log hashes and reproduction.

• Released 5.4.1 cohort; no additional install/dedupe or upstream patches.
• First full N=3 control passed24 writes, exact42chunk+24manifest persisted barrier and four-survivor same-identity offline crash recovery. One file receipt concurrent with planned peer loss took21.3s: durability passed, failure-time responsiveness remains a concern.
• First adaptive N=3 run failed its unchanged30s joined-inventory stability gate. All28 samples metN=3 coverage and metadata completed early, but content residency kept changing. It then failed custodian5's20s stop deadline. Budget reduction, planned loss, final barrier and offline reopen were not reached. No retry/timeout inflation/forced-success exit.
• Bounded separate metadata/chunk profiles show long emitted content join-plan spans (max20.285s) but short metadata join plans. These are overlapping elapsed spans, not CPU time or proven cause. Raw logs/profile identities were sent directly to the upstream Codex task with user authorization.
• 28 analysis/topology/telemetry tests and33CLI tests pass; library/CLI builds, strict standalone probe typecheck, lint, format and package exclusions pass. No full latest-cohort or cross-OS gate claimed. The frozen sources match both live reports.

Keep this PR draft and adaptive production integration disabled. The successful prior N=2 rebaseline is not a successful N=3 adaptive result. All git/GitHub actions used verified peerbit-org through .envrc.

[peerbit-org]

Bounded diagnostics follow-up published on the separate experimental branch, exact head 7b2ce8fd5106ab6b2cc8057a8c08f0b4404e27d9:

Diagnostic fields, stop phases, limits and validation.

• Keep eight slowest join-plan spans with only upstream's selected counts/booleans; preserve missing values as unknown.
• Separate shutdown command queueing, Peerbit.stop and its high-level cleanup calls, post-stop disk scan, IPC reply, and actual OS exit. The earlier20s command timeout is not yet localized to Peerbit.stop or native cleanup.
• Bounded5s/19s pending snapshots; original20s stop and15s natural-exit deadlines unchanged. Request-local traces preserve attribution if error cleanup queues another stop.
  -49 unit tests, strict types/lint/format/package checks pass. A single fresh offline-worker smoke emitted all18 expected markers and exited naturally. This is not an N=3 rebaseline or reproduction of the timeout.

Old run source remains at15f40b98; raw failures unchanged. No controller policy, production code, dependency install/dedupe, merge, or release. Full details and source have been sent directly upstream; waiting for its coherent published cohort. All git/GitHub actions verified peerbit-org.