WIP: napi-rs shared-core for Node (WASM stays elsewhere) - #781
Draft
slinkydeveloper wants to merge 1 commit into
Draft
WIP: napi-rs shared-core for Node (WASM stays elsewhere)#781slinkydeveloper wants to merge 1 commit into
slinkydeveloper wants to merge 1 commit into
Conversation
Run the Rust shared-core as a napi-rs native addon on Node to escape the wasm32 4GB memory ceiling and reduce buffer copies, while keeping the WASM build for every other runtime (Bun, Deno, Cloudflare, edge, browser) and as a universal fallback on Node. - New package @restatedev/restate-sdk-shared-core-native: napi port of sdk-shared-core-wasm-bindings, same restate-sdk-shared-core 7.0.2 + features for functional parity, published as per-platform prebuilt binaries. - restate-sdk: vm/index.ts selector loads native on Node (optionalDependencies + runtime detection) and falls back to WASM; the 5 vm consumers are repointed. Force with RESTATE_SHARED_CORE=native|wasm. - Cloudflare Workers unchanged (patch swaps in the workerd WASM entrypoint). - CI: native.yaml cross-compiles 5 triples (linux gnu+musl x64/arm64, darwin-arm64); test.yml gains a native-parity job; release.yml publishes the native packages via .tools/publish-native.sh with a completeness gate. Not yet validated: full conformance parity (runs in CI native-parity job; locally blocked by container networking) and the release publish flow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP / draft — opening early for visibility; conformance + release-publish still need a real CI run (see below).
What
Run the Rust shared-core as a napi-rs native addon on Node, to escape the wasm32 4 GB memory ceiling users are hitting and to cut buffer copies. Every other runtime (Bun, Deno, Cloudflare Workers, edge, browser) keeps the WASM build, which also stays bundled in
@restatedev/restate-sdkas a universal fallback on Node. Behavior is meant to be functionally identical (samerestate-sdk-shared-core 7.0.2+ features). For users it's transparent: bump the SDK version.How
@restatedev/restate-sdk-shared-core-native/— napi port ofsdk-shared-core-wasm-bindings, exposing the exact same JS surface. Published as per-platform prebuilt binaries (optionalDependencies).restate-sdkselector (src/endpoint/handlers/vm/index.ts) — loads native on Node (runtime detection) and falls back to WASM; the 5vm.*consumers are repointed. Force withRESTATE_SHARED_CORE=native|wasm.cloudflareWorkersBundlerPatch()hack).&[u8]/Uint8Arrayin, single-moveBufferout, no base64 decode/instantiate at startup.CI
native.yamlcross-compiles 5 triples (linux gnu+musl x64/arm64, darwin-arm64 — mirrors the Java SDK).test.ymlgains anative-parityjob (e2e withRESTATE_SHARED_CORE=native).release.ymlpublishes the native packages via.tools/publish-native.sh, with a Java-SDK-style completeness gate.Verified locally
pnpm build(all packages), full typecheck, ATTW + api-extractor, lint, prettier,--frozen-lockfile. Native crate compiles; VM executes shared-core with structured errors/headers/enums/buffers correct; selector picks native on Node / WASM elsewhere.Still to validate
native-parityCI job (locally blocked by container networking; native and WASM fail identically there)..tools/publish-native.shnapicreate-npm-dirs/artifacts/prepublishflags) — needs a dry-run with real npm creds.🤖 Generated with Claude Code