Skip to content

Add opt-in Mongo admission commit on Engine submit - #525

Open
sirdeggen wants to merge 52 commits into
mainfrom
codex/overlay-admission-s03
Open

sirdeggen wants to merge 52 commits into
mainfrom
codex/overlay-admission-s03

Conversation

@sirdeggen

@sirdeggen sirdeggen commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Overlay reliability program

This change is one slice of Make Overlays Great Again.

Do not merge until the stacked TypeScript and Go overlay branches have been proven together locally. Isolated CI green is not integration acceptance. Keep this PR draft. No npm publication, live database migration, or Mongo/BASM default activation.

C05 (verified progressive API) is parked. C02 (ts-stack#517) and C04 (ts-stack#518) both edit LookupResolver.ts and must not merge independently.

Keep this pull request in draft until local validation is complete.

Program and scope

Impact

  • Public package source or manifest changed; affected packages are listed below
  • Security-sensitive boundary changed
  • Documentation or examples changed

Affected packages: @bsv/overlay

Verification

  • Local commands and results: overlay mongo 82 passed (lost ACK, invisible index, competing spend, SIGKILL); Engine+SQL 138 passed; Knex admission still undefined
  • Hosted CI run: not yet (draft)
  • All applicable checks are terminal and successful on the exact head

Security and dependencies

  • No new production default
  • Mongo remains opt-in

Release and operations

  • No npm publication was performed from a workstation or from this PR

Add replica-set schema bootstrap, content-addressed payload publication,
transactionally guarded references/GC, and majority-commit outcome helpers
behind an optional mongodb peer. This is not an AdmissionStorage adapter
and does not change Engine or Knex defaults.
Collection validators used lexicographic $lte on unpadded outputIndex
strings, which rejected legal indexes such as "9". Bound uint32 fields
with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP.
Wire Engine.submit to commitAdmission when overlay-admission-v1 is advertised.
MongoAdmissionStorage honors majority ACK, conditional spends, enlisted
indexes or a durable lookup/propagation outbox. Knex/SQL remains the default.
Keep failed-topic STEAK keys and merkle applied metadata on the Engine
admission plan. Version history-update pins by generation and hydrate
Mongo output scripts and BEEF without unsafe integer coercion.
Extract Engine submit and admission plan validators, use localeCompare
and structuredClone, and throw from unimplemented Mongo BEEF updates.
Brand StorageUint64, extract high-complexity helpers, and apply
Sonar-preferred optional chaining, regex, clone, and sort APIs
without changing payload identity or uint32 $toLong validators.
# Conflicts:
#	docs/reference/package-api-migrations.md
#	docs/reference/stack-facts.md
#	governance/repository-health/baselines.json
Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
…dex/overlay-admission-s03

# Conflicts:
#	docs/reference/package-api-migrations.md
@sirdeggen
sirdeggen changed the base branch from codex/overlay-mongo-s02 to main September 9, 2026 03:27
Health requires overrideRemovalReview.retainedCount to match the live registry.
Health requires overrideRemovalReview.retainedCount to match the live registry.
S6564 made StorageUint64 a unique branded string. Engine and Mongo
admission paths now validate decimal strings through asStorageUint64.
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
LookupResolver and evidence helpers pushed the UMD/Vite/Metro payloads
over the previous raw-size gates.
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
Message-box UMD, wallet Vite gzip, and Hermes bytecode now cover the
evidence/discovery helper payload growth.
Concurrent snapshot spends can surface WriteConflict without the
TransientTransactionError label. Retry those so the loser observes spend-conflict.
Concurrent snapshot spends can throw WriteConflict from findOneAndUpdate
outside the transaction body. Treat those as transient at the run loop.
Concurrent snapshot spends still escaped the transaction-runner retry.
commitAdmission now retries WriteConflict up to eight times.
S3776: commitAdmission was 21 after the retry loop. The loop now lives in
runAdmissionWithWriteConflictRetry.
…dex/overlay-admission-s03

# Conflicts:
#	docs/packages/overlays/overlay.md
#	packages/overlays/overlay/README.md
@sirdeggen sirdeggen added the overlay-reliability Part of org overlay reliability program (project 19) label Sep 9, 2026
@sirdeggen

Copy link
Copy Markdown
Contributor Author

Linked to org project Make Overlays Great Again and tracker https://github.com/bsv-blockchain/make-overlays-great-again/issues/1.

This PR stays draft until the overlay stack is proven locally across ts-stack, go-overlay-services, and go-sdk. Do not merge to main/master to make progress.

@sirdeggen
sirdeggen marked this pull request as ready for review September 10, 2026 21:49
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

overlay-reliability Part of org overlay reliability program (project 19)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant