Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,11 @@ jobs:
run: pnpm test packages/cli/test/native-output.test.ts
- name: Helper diagnostics and object/link parity
if: matrix.shard == 1
run: pnpm test packages/compiler/test/native-codegen-integration.test.ts
run: >-
pnpm test
packages/compiler/test/native-codegen-integration.test.ts
packages/cli/test/native-link-info.test.ts
tests/harness/native-object-example.test.ts
- name: LLVM-tier helper object differential (${{ matrix.shard }}/3)
env:
SCRIPTC_LLVM_HELPER_ONLY: "1"
Expand All @@ -167,6 +171,12 @@ jobs:
"$PREFIX/node_modules/.bin/scriptc" build tests/corpus/001-hello.ts \
--emit=obj -o "$RUNNER_TEMP/installed.o"
file "$RUNNER_TEMP/installed.o" | grep 'Mach-O 64-bit object arm64'
"$PREFIX/node_modules/.bin/scriptc" build tests/corpus/001-hello.ts \
--print=native-link-info -o "$RUNNER_TEMP/installed-link.o" \
> "$RUNNER_TEMP/installed-link.json"
node examples/native-object/link.mjs cc \
"$RUNNER_TEMP/installed-link.json" "$RUNNER_TEMP/installed-program"
test "$("$RUNNER_TEMP/installed-program")" = 'hello world'

# Exercises the supported Windows GNU target and the built CLI end to end:
# TS7 must open its synthetic project, ambient files must resolve across
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,13 @@ assembly/object
emission is rejected until the helper's AddressSanitizer pipeline matches the
executable path.

External object consumption is experimental. Use
`--print=native-link-info` to emit the object and print a versioned JSON recipe
containing its target, `main` entry, exact `@scriptc/runtime` source pack,
required system libraries, FFI inputs, and ABI marker. The recipe never uses
hidden scriptc cache paths. See [`examples/native-object`](./examples/native-object)
for C-driver and direct Apple-linker builds.

## Use Node APIs

Supported Node APIs compile to the native runtime. For example, `server.ts`:
Expand Down
16 changes: 16 additions & 0 deletions docs/src/app/cli/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,13 @@ outputs use the matching native helper installed with scriptc and do not
invoke an external compiler, archiver, linker, or SDK. The object is a
relocatable program object with undefined <code>scr_*</code> runtime symbols
and a required <code>scr_runtime_abi_v1</code> marker, not a standalone library.
External consumption is experimental and requires the exact runtime version
reported by <code>--print=native-link-info</code>. That option still writes the
object, performs no link, and prints a versioned JSON recipe with the target,
<code>main</code> entry, installed source runtime pack, FFI inputs, and system
libraries. It never reports private scriptc cache paths. See
<a href="/native-objects">Native Program Objects</a> for complete C-driver and
direct-linker examples.
<code>--emit=exe</code> is the default and retains the existing executable
behavior.

Expand Down Expand Up @@ -83,6 +90,9 @@ Prebuilds the release runtime objects and native TLS/dynamic-engine archives aga
<dt><code>--emit &lt;ir|c|llvm|asm|obj|exe&gt;</code></dt>
<dd>Select the invocation's one primary artifact. <code>ir</code>, <code>c</code>, and <code>llvm</code> need only Node. <code>asm</code> and <code>obj</code> use the bundled LLVM helper on macOS 15+ arm64 and emit artifacts targeting macOS 14.0. <code>exe</code> is the default.</dd>

<dt><code>--print &lt;native-link-info&gt;</code></dt>
<dd>Build an object (equivalent to <code>--emit=obj</code>) and print its machine-readable external link recipe as JSON instead of printing the artifact path. The document names the exact installed source runtime pack and all link inputs, but does not invoke a linker.</dd>

<dt><code>--dynamic</code></dt>
<dd>Embed the dynamic engine (~620KB) so npm dependencies and <code>any</code>-typed code can run. Static stays the default — without this flag, dynamic-tier sites are per-site compile errors. See <a href="/dependencies">npm Dependencies</a>.</dd>

Expand Down Expand Up @@ -182,6 +192,12 @@ An explicit <code>--backend llvm</code> pins the LLVM backend and fails with dia
<td>Bundled scriptc LLVM helper</td>
<td>Not used</td>
</tr>
<tr>
<td>External link of <code>--emit=obj</code> with the reported source runtime pack</td>
<td>Not used by the artifact</td>
<td>C compiler required for runtime sources</td>
<td>macOS linker and SDK required</td>
</tr>
<tr>
<td><code>--emit=exe</code></td>
<td>Required to run scriptc</td>
Expand Down
7 changes: 7 additions & 0 deletions docs/src/app/how-it-works/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ TypeScript ──tsc: parse + typecheck──▶ lowering ──▶ typed IR ─
3. **Backends** — `--emit=c` writes readable C and stops; `--emit=llvm` writes textual LLVM IR and stops. Neither source-output command discovers or invokes a native toolchain. On macOS 15+ arm64, `--emit=asm|obj` sends LLVM IR to a version-matched out-of-process helper linked to LLVM 22; it needs no clang or linker and emits macOS 14-targeted artifacts. Executable builds default to LLVM and can fall back to C on a native program outside the LLVM tier (one stderr note; `--backend llvm` pins it and fails with a diagnostic instead). The production <code>wasm32-wasi</code> target never falls back.
4. **Link** — the runtime is a C library of link-gated feature units: binaries pay only for what they use. A hello-world links nothing but libSystem; a regex-using program links the regex engine; an `http` server links the net stack.

Program objects define <code>main</code> and leave their selected
<code>scr_*</code> runtime functions undefined. The
<code>scr_runtime_abi_v1</code> reference is a strong link-time compatibility
check. <code>--print=native-link-info</code> exposes the exact source runtime
pack and link ordering for external builds; that object ABI is currently
experimental and exact-runtime-version compatible, not semver-stable.

Inspect any stage yourself:

```console
Expand Down
7 changes: 7 additions & 0 deletions docs/src/app/native-objects/layout.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { pageMetadata } from "@/lib/page-metadata";

export const metadata = pageMetadata("native-objects");

export default function Layout({ children }: { children: React.ReactNode }) {
return children;
}
84 changes: 84 additions & 0 deletions docs/src/app/native-objects/page.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Native Program Objects

`scriptc build --emit=obj` produces one relocatable macOS arm64 program
object without invoking clang, a linker, or an SDK. The object defines
`main`; it is intended to become the program in an external native link. It
is not a host-callable library—use `scriptc build --lib --profile ...` for
that interface.

## ABI and runtime contract

The external object ABI is **experimental**. Its `scr_*` function and data
surface may change before 1.0, so consumers must use the exact
`@scriptc/runtime` version reported by the same compiler installation. This
is stricter than semver compatibility.

The object intentionally leaves its selected runtime symbols undefined. It
also holds a strong reference to `scr_runtime_abi_v1`, which the matching
runtime defines. Linking an object against a runtime with another ABI marker
fails at link time with the missing versioned symbol; it cannot become a
latent runtime incompatibility.

## Machine-readable link information

Add `--print=native-link-info` to emit the object and print a JSON document
instead of the ordinary path line:

```console
$ scriptc build main.ts --print=native-link-info -o app.o > link-info.json
```

The `scriptc.native-link-info.v1` document reports:

- target triple, object format, architecture, minimum OS, and relocation model;
- the `main` entry and versioned runtime ABI marker;
- the matching installed `@scriptc/runtime` source-pack root and exact source
sets, include paths, defines, and compile flags selected by the program;
- ordered program, FFI, runtime, and vendor inputs; and
- required system libraries and frameworks.

Paths inside each source set are relative to `runtime_pack.root`. FFI library
paths are the manifest-resolved absolute inputs. No path points into scriptc's
private build cache. The source pack requires a C compiler; the final link
requires the macOS SDK and linker. Precompiled runtime packs are not shipped
yet.

## C compiler as linker driver

The repository's `examples/native-object` directory is a runnable example
with a TypeScript program, a C FFI function, and a small consumer for the JSON
recipe:

```console
$ cd examples/native-object
$ clang -target arm64-apple-macosx14.0.0 -O2 -c native.c -o native.o
$ scriptc build main.ts --ffi ffi.json --print=native-link-info -o app.o > link-info.json
$ node link.mjs cc link-info.json app-cc
$ ./app-cc
42
```

The script compiles each reported source set and gives clang only the link
inputs and system libraries from the document. `--emit=obj` itself remains
clang-free; this compiler invocation belongs to the external runtime build.

## Native Apple linker

The same example can invoke Apple `ld` directly after compiling the reported
runtime source sets:

```console
$ node link.mjs ld link-info.json app-ld
$ ./app-ld
42
```

This lane asks `xcrun` for the selected macOS SDK and linker, then supplies
the target's minimum OS, every ordered object/archive input, and each reported
system library. It demonstrates the code-generation boundary precisely:
scriptc owns `app.o`; an external toolchain owns runtime compilation and the
platform link.

Outbound FFI declarations retain the same C ABI in clang-compiled LLVM and
helper-produced object paths. Scalar widths, string/byte pointer-plus-length
pairs, and callback signatures follow the [Native FFI](/ffi) manifest.
1 change: 1 addition & 0 deletions docs/src/app/quickstart/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -90,5 +90,6 @@ The package's JS is embedded into the binary at build time — the executable ne
## Next steps

- [CLI Reference](/cli) — every command and flag, including `--emit`, `--backend llvm`, and `--sanitize`.
- [Native Program Objects](/native-objects) — consume `app.o` from an external C or linker build.
- [Platform Support](/platforms) — cross-compiling to Linux and Windows with zig.
- [Limitations](/limitations) — what doesn't compile yet.
1 change: 1 addition & 0 deletions docs/src/lib/docs-navigation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export const navSections: NavSection[] = [
{ name: "Coverage Reports", href: "/coverage" },
{ name: "npm Dependencies", href: "/dependencies" },
{ name: "Native FFI", href: "/ffi" },
{ name: "Native Program Objects", href: "/native-objects" },
{ name: "Platform Support", href: "/platforms" },
],
},
Expand Down
1 change: 1 addition & 0 deletions docs/src/lib/page-titles.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ export const PAGE_TITLES: Record<string, string> = {
coverage: "Coverage Reports",
dependencies: "npm Dependencies",
ffi: "Native FFI",
"native-objects": "Native Program Objects",
platforms: "Platform Support",
"how-it-works": "How It Works",
limitations: "Limitations",
Expand Down
27 changes: 27 additions & 0 deletions examples/native-object/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# External program object

This macOS arm64 example links a scriptc program object, a small C FFI
implementation, and the exact installed source runtime pack. It uses no
scriptc cache path.

```console
$ clang -target arm64-apple-macosx14.0.0 -O2 -c native.c -o native.o
$ scriptc build main.ts --ffi ffi.json --print=native-link-info -o app.o > link-info.json
$ node link.mjs cc link-info.json app-cc
$ ./app-cc
42
$ node link.mjs ld link-info.json app-ld
$ ./app-ld
42
```

`cc` uses the C compiler as a linker driver. `ld` compiles the same reported
runtime sources and invokes the Apple linker directly with the selected SDK.
The object defines `main`; it is a complete program object, not a library to
load into another process. Use `scriptc build --lib --profile ...` for a
host-callable static library.

The external object ABI is experimental. Always consume the runtime pack at
the exact `runtime_pack.version` reported by the same scriptc installation.
The object requires `scr_runtime_abi_v1`, so a mismatched runtime fails during
the link instead of starting with an incompatible ABI.
13 changes: 13 additions & 0 deletions examples/native-object/ffi.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"ffi_format": 1,
"functions": [
{
"name": "nativeDouble",
"symbol": "native_double",
"params": ["f64"],
"returns": "f64"
}
],
"libraries": ["./native.o"],
"system_libraries": []
}
107 changes: 107 additions & 0 deletions examples/native-object/link.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
#!/usr/bin/env node
import { spawnSync } from "node:child_process";
import { mkdirSync, readFileSync, rmSync } from "node:fs";
import { basename, dirname, join, resolve } from "node:path";

const [mode, infoArg, outputArg] = process.argv.slice(2);
if ((mode !== "cc" && mode !== "ld") || !infoArg || !outputArg) {
console.error("usage: node link.mjs <cc|ld> <native-link-info.json> <output>");
process.exitCode = 2;
} else {
const info = JSON.parse(readFileSync(infoArg, "utf8"));
if (info.schema !== "scriptc.native-link-info.v1") {
throw new Error(`unsupported native link info schema: ${info.schema}`);
}
const runtimePackage = JSON.parse(
readFileSync(join(info.runtime_pack.root, "package.json"), "utf8"),
);
if (
runtimePackage.name !== info.runtime_pack.package ||
runtimePackage.version !== info.runtime_pack.version
) {
throw new Error(
`runtime pack identity mismatch: expected ${info.runtime_pack.package}@${info.runtime_pack.version} ` +
`at ${info.runtime_pack.root}, found ${runtimePackage.name ?? "<unnamed>"}@${runtimePackage.version ?? "<unversioned>"}`,
);
}
const run = (command, args, options = {}) => {
const result = spawnSync(command, args, { stdio: "inherit", ...options });
if (result.error) throw result.error;
if (result.status !== 0) throw new Error(`${command} exited ${result.status}`);
};
const capture = (command, args) => {
const result = spawnSync(command, args, { encoding: "utf8" });
if (result.error) throw result.error;
if (result.status !== 0) throw new Error(result.stderr || `${command} exited ${result.status}`);
return result.stdout.trim();
};

const infoPath = resolve(infoArg);
const output = resolve(outputArg);
const buildDir = join(dirname(infoPath), `.native-link-${mode}`);
rmSync(buildDir, { recursive: true, force: true });
mkdirSync(buildDir, { recursive: true });
const runtimeObjects = [];
const vendorArchives = [];
for (const set of info.runtime_pack.source_sets) {
const setDir = join(buildDir, set.name);
mkdirSync(setDir, { recursive: true });
const objects = [];
for (const source of set.sources) {
const object = join(setDir, `${source.replace(/[^A-Za-z0-9]+/g, "_")}.o`);
run("clang", [
...set.c_flags,
...set.defines.map((define) => `-D${define}`),
...set.include_directories.flatMap((path) => [
"-I", join(info.runtime_pack.root, path),
]),
"-c", join(info.runtime_pack.root, source), "-o", object,
]);
objects.push(object);
}
if (set.output === "objects") {
runtimeObjects.push(...objects);
} else {
const archive = join(buildDir, basename(set.suggested_output));
run("ar", ["rcs", archive, ...objects]);
vendorArchives.push(archive);
}
}

const inputs = [
info.program.object,
...info.ffi.libraries,
...runtimeObjects,
...vendorArchives,
];
const libraries = info.link.system_libraries.map((name) => `-l${name}`);
const frameworks = info.link.frameworks.flatMap((name) => ["-framework", name]);
if (mode === "cc") {
// Darwin compiler drivers add libSystem themselves. It remains explicit
// in the document because a direct ld invocation must name it.
const driverLibraries = info.link.system_libraries
.filter((name) => name !== "System")
.map((name) => `-l${name}`);
run("clang", [
...info.link.driver_flags,
...inputs,
...driverLibraries,
...frameworks,
"-o", output,
]);
} else {
const sdk = capture("xcrun", ["--sdk", "macosx", "--show-sdk-path"]);
const sdkVersion = capture("xcrun", ["--sdk", "macosx", "--show-sdk-version"]);
const linker = capture("xcrun", ["--sdk", "macosx", "--find", "ld"]);
run(linker, [
"-arch", info.target.architecture,
"-platform_version", "macos", info.target.minimum_os, sdkVersion,
"-syslibroot", sdk,
...(info.link.driver_flags.includes("-Wl,-dead_strip") ? ["-dead_strip"] : []),
...inputs,
...libraries,
...frameworks,
"-o", output,
]);
}
}
3 changes: 3 additions & 0 deletions examples/native-object/main.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
declare function nativeDouble(value: number): number;

console.log(nativeDouble(21));
3 changes: 3 additions & 0 deletions examples/native-object/native.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
double native_double(double value) {
return value * 2.0;
}
3 changes: 3 additions & 0 deletions packages/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ emission currently requires a macOS 15+ arm64 host and emits objects targeting
macOS 14.0 (`arm64-apple-macosx14.0.0`).
Objects retain undefined `scr_*` runtime references plus the
`scr_runtime_abi_v1` compatibility marker; they are not library archives.
External consumption is experimental and requires the exact matching runtime.
`scriptc build app.ts --print=native-link-info -o app.o` prints the versioned
JSON target/runtime/link recipe without performing a link.
`--emit=asm|obj --sanitize` is rejected until ASan pipeline parity is
available.

Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/bootstrap.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ async function tryFastPath(): Promise<number | null> {
if (
(command !== "build" && command !== "run") || inputArg === undefined ||
(values.emit !== undefined && values.emit !== "exe") ||
values.print !== undefined ||
values["emit-ir"] ||
values.lib || values["from-c"] || values["provenance-sources"] ||
(values["external-types"] ?? []).length > 0
Expand Down
Loading
Loading