diff --git a/docs/own-readiness.md b/docs/own-readiness.md index 02052ffbe..73809b082 100644 --- a/docs/own-readiness.md +++ b/docs/own-readiness.md @@ -14,8 +14,8 @@ preview for porting existing TypeScript.** - Everything its design set out to build is built, and the tracker has no open defect against it. - Nothing unsound was found in this round of measurement. -- It still rejects about one in five programs the other models accept, and it has no leak-free - way to write a back-reference. +- It still rejects about one in five programs the other models accept. A back-reference is a + `WeakRef` (spec 84), which a module that also spawns threads with closures cannot use. - Memory-safety bugs were still being found in real code in the three days before this was written. - None of the three tier 3 exit criteria in the plan is met. @@ -25,7 +25,7 @@ It stays in tier 3 ("experimental") in the plan's terms. | Area | Status | What it rests on | | --- | --- | --- | -| Design | Complete | Spec phases 0-8 and the later designs (parameter decorators, facts from object files, immortal keepers, generic containers) are merged. One item is deferred: `WeakRef` under own. | +| Design | Complete | Spec phases 0-8 and the later designs (parameter decorators, facts from object files, immortal keepers, generic containers, `WeakRef`) are merged. | | Tracker | Clean | No open GitHub issue. The own items of `TSLANG_ISSUES.md` (31-33, 39-42, 47-85) are fixed, with one remnant: whole-program facts (item 48). | | Soundness | No failure found, not proven | See [Soundness](#soundness). | | Coverage of TypeScript | Partial | 707 of 882 corpus files (80%) compile. See [Coverage](#coverage). | @@ -111,7 +111,7 @@ Probed one by one under the JIT: | `x ??= new B()` on a parameter | Rejected ("merged from several branches") | | A closure that assigns a parameter it captures | Rejected | | `m.forEach((v, k) => { total += v; })` on a `Map` or a `Set` | Compiles and runs | -| `m.forEach((v, k) => { names.push(k); })` | Rejected, with no source position in the error | +| `m.forEach((v, k) => { names.push(k); })` | Compiles and runs: a callback declaring fewer parameters than `forEach` passes is given a copy of a string it keeps | | try/catch/finally, unions, `any`, async functions (11 corpus files) | Compile; async was not run for this document | ## Real code @@ -139,7 +139,7 @@ From the plan's Phase 5 row for `-mm=own`: | Criterion | Met | Notes | | --- | --- | --- | -| Close the open items: optional accessor fields, capturing callbacks | No | An optional interface field filled by an accessor is a clean error under every model, not supported. Capturing callbacks over `Map` and `Set` work for numbers and fail when the callback pushes a key into a captured array. | +| Close the open items: optional accessor fields, capturing callbacks | Half | An optional interface field filled by an accessor is a clean error under every model, not supported. Capturing callbacks over `Map` and `Set` work, keys and values pushed into a captured array included; an error found in such a callback points at its source. | | A written specification of the ownership rules | No | The rules exist as a design log of about 5,000 lines (`tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md`) and as the user guide. Neither is a specification someone could implement or check against. | | Error messages reviewed by someone new to the model | No | Not done. Without `--di` most messages say `'this value'` in place of the variable's name. | @@ -157,12 +157,11 @@ assertions-enabled CI leg, and a soak run. The first holds today; the other two module, still cannot take a field this way, and the popped node cannot be used after the assignment that frees it. 3. **The "not yet" messages.** Second references the compiler cannot classify, values merged from branches, object literals that hold owned values. -4. **Capturing callbacks over `Map` and `Set`**, and a source position for the error they give. -5. **Whole-program facts**, so a class nobody extends does not need `@final` to keep its callers' +4. **Whole-program facts**, so a class nobody extends does not need `@final` to keep its callers' borrows. -6. **The exit criteria**: a rules specification, an outside review of the messages, variable names +5. **The exit criteria**: a rules specification, an outside review of the messages, variable names in messages without `--di`. -7. **Soundness evidence at scale**: fuzzing on Linux with a perturbing allocator, an AddressSanitizer +6. **Soundness evidence at scale**: fuzzing on Linux with a perturbing allocator, an AddressSanitizer run of compiled own programs, a second and third outside codebase. ## What it is ready for @@ -172,7 +171,7 @@ assertions-enabled CI leg, and a soak run. The first holds today; the other two | New code written for own: trees of owners, values, arrays, strings, `Shared` where shared | Yes, as a preview | | Numeric and allocation-heavy kernels where a collector or counting costs too much | Yes, as a preview | | Porting an existing TypeScript program unchanged | No | -| Programs built on graphs with back-references | No | +| Programs built on graphs with back-references | As a preview, with `WeakRef`; not with threads in the same module | | A release that promises memory safety to outside users | No | ## How this was measured diff --git a/docs/production-readiness-plan.md b/docs/production-readiness-plan.md index f0aaca644..cc825a73c 100644 --- a/docs/production-readiness-plan.md +++ b/docs/production-readiness-plan.md @@ -61,7 +61,7 @@ These four calls shape everything below. Each one is a proposal and can be overr - The Debug-built compiler had 38 known suite failures on 2026-10-01, including a JIT `--opt` abort on any program. Not re-run since. - Known open items that are not in the tracker: - - `-mm=own`: optional accessor fields, capturing callbacks in Map/Set + - `-mm=own`: optional accessor fields (capturing callbacks in Map/Set work since 2026-11-16) - memory-model fuzzer findings R1-R10, O4, G1 - run-time `Symbol` values and `obj[sym]` - a `(() => T) | null` field cast diff --git a/tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md b/tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md index 1b27f9272..029202cab 100644 --- a/tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md +++ b/tslang/docs/superpowers/specs/2026-09-24-own-memory-model-design.md @@ -5959,3 +5959,29 @@ own compiler frees without it. The default library is rebuilt with the compiler, Tests: `00weakref_basic`, `00weakref`, `00weakref_frame` (own, gc, none; `-noopt` under own), `own_weakref_borrow`, `own_weakref_share_unrelated`; errors `own_err_weakref_rc`, `_not_class`, `_stored`, `_returned`, `_captured`, `_after_call`, `_after_release`, `_local_reassigned`, `_thread`. + +## 85. A callback declaring fewer parameters; a WeakRef of a nullable target, 2026-11-16 + +**The adapter.** A function value with fewer parameters than the type it is given as (`(v) => ...` for +`(value, key, map) => void`) is called through an adapter MLIRGen makes (`fewerParametersThunk`, +`.args`), which passes on its first parameters. It passed its block arguments straight +to the call, so the signature pass never saw it keep a parameter its callee keeps, and the inference +reported "given to ..., which keeps it, but -mm=own cannot move it here", at `:0` (the adapter +had no location). That was readiness item 4: `m.forEach((v, k) => { names.push(k); })` over a Map or a +Set (their `forEach` type has three parameters). A callback declaring every parameter compiled. + +The adapter now takes its parameters into slots (`ts.Param`, as any function), so it keeps what its +callee keeps and the callback family decides it as any member (spec 33). When the members disagree +(one callback keeps the value, another the key) they are demoted, and a string the adapter then only +borrows, given to a callee that keeps it, gets a copy (`callCopies`, spec 22), as a string read out of +a place does: `giveKeptArguments` takes a load of a parameter slot that `copiesInto` the call as such a +read. Without `--di` the adapter carries the location it is made at, so an error found in it points +at the callback's use; with `--di` it keeps its own subprogram, as before. + +**WeakRef of a nullable declared target.** `let n: Node | null = new Node(); new WeakRef(n)`: this +compiler keeps n's declared type where TypeScript narrows it by the assignment, so the WeakRef's `T` +was `Node | null`: refused under own and none ("needs a class object"), and under gc a WeakRef that a +later cast failed to verify. With no type argument `T` is now the target's type without null; a null +target makes a WeakRef whose `deref()` gives undefined (copy, release and deref test for null). + +Tests: `own_callback_fewer_params` (own, rc, gc, none), `00weakref_narrowed` (own, gc, none). diff --git a/tslang/lib/TypeScript/MLIRGenCast.cpp b/tslang/lib/TypeScript/MLIRGenCast.cpp index 6875c9b95..349dbc2a6 100644 --- a/tslang/lib/TypeScript/MLIRGenCast.cpp +++ b/tslang/lib/TypeScript/MLIRGenCast.cpp @@ -2436,8 +2436,11 @@ namespace mlirgen { mlir::OpBuilder::InsertionGuard guard(builder); - // compiler glue with no source to step through, as INSTANCEOF_NONE_NAME's - mlir::Location noLocation = mlir::UnknownLoc::get(builder.getContext()); + // compiler glue with no source to step through, as INSTANCEOF_NONE_NAME's; without --di it + // carries the place it is made for, so an error found in it (an own rule) points there + mlir::Location noLocation = compileOptions.generateDebugInfo + ? mlir::Location(mlir::UnknownLoc::get(builder.getContext())) + : stripMetadata(location); auto funcOp = mlir_ts::FuncOp::create(noLocation, thunkName, thunkType); funcOp.setPrivate(); funcOp->setAttr("internal_linkage", builder.getUnitAttr()); @@ -2466,8 +2469,18 @@ namespace mlirgen auto callee = builder.create(noLocation, inFuncType, mlir::FlatSymbolRefAttr::get(builder.getContext(), calleeName)); - auto callOp = builder.create(noLocation, callee, - entryBlock.getArguments().take_front(inInputs.size())); + // what it passes on goes through its parameters, as any function's does: -mm=own reads a + // parameter given to a callee that keeps it as one this function keeps (its callers then + // give it, or a copy of a string they lend), where a bare argument is given without being owned + SmallVector passed; + for (auto [index, argument] : llvm::enumerate(entryBlock.getArguments().take_front(inInputs.size()))) + { + auto paramOp = builder.create(noLocation, mlir_ts::RefType::get(argument.getType()), argument, + builder.getBoolAttr(false), builder.getIndexAttr(index + 1)); + passed.push_back(builder.create(noLocation, argument.getType(), paramOp)); + } + + auto callOp = builder.create(noLocation, callee, passed); if (hasResult) { // the callee's result is the thunk's, handed on as `return f(x)` hands it: owned, and diff --git a/tslang/lib/TypeScript/MLIRGenExpressions.cpp b/tslang/lib/TypeScript/MLIRGenExpressions.cpp index c2068ab6e..d94bec195 100644 --- a/tslang/lib/TypeScript/MLIRGenExpressions.cpp +++ b/tslang/lib/TypeScript/MLIRGenExpressions.cpp @@ -1956,9 +1956,13 @@ namespace mlirgen EXIT_IF_FAILED_OR_NO_VALUE(result) auto value = V(result); + // T is the target's type without null: TypeScript reads `n` as `Node` after + // `let n: Node | null = new Node()`, where this compiler keeps the declared type. A null + // target makes a WeakRef whose deref() gives undefined (the copy, the release and deref + // all test for null). if (!elementType) { - elementType = mth.wideStorageType(value.getType()); + elementType = mth.wideStorageType(mth.stripOptionalType(NonNullableTypes(value.getType()))); } // a class object, or one a test narrowed out of `C | null` (`if (m) new WeakRef(m)`) diff --git a/tslang/lib/TypeScript/OwnershipInferencePass.cpp b/tslang/lib/TypeScript/OwnershipInferencePass.cpp index 3c9e55a38..0d7cb7a5b 100644 --- a/tslang/lib/TypeScript/OwnershipInferencePass.cpp +++ b/tslang/lib/TypeScript/OwnershipInferencePass.cpp @@ -859,6 +859,20 @@ namespace own_inference } else if (!isFresh(root) && !placeReadOf(root)) { + // a string read out of a parameter this function only borrows: the callee takes a + // copy (spec 22), as it does of one read out of a place (callCopies). The adapter + // that calls a callback declaring fewer parameters (`(v) => names.push(v)`) passes + // its parameters on so, once the callbacks passed in its place disagree on them. + // The dry run still fails here: its failure is what makes the copies, and the real + // run then finds a fresh copy given + auto loadOp = arg.getDefiningOp(); + auto varOp = loadOp ? loadOp.getReference().getDefiningOp() : mlir_ts::VariableOp(); + auto position = -1; + if (dryRun && varOp && isParameterSlot(varOp, position) && copiesInto(call, arg)) + { + callCopies.push_back({loadOp, call}); + } + reportGivenNotOwned(call, arg); } } diff --git a/tslang/test/tester/CMakeLists.txt b/tslang/test/tester/CMakeLists.txt index 1c9ac4712..d8c1b6282 100644 --- a/tslang/test/tester/CMakeLists.txt +++ b/tslang/test/tester/CMakeLists.txt @@ -4662,7 +4662,7 @@ foreach(own_test own_fresh_string own_fresh_array own_return_new own_try_local o own_generic_specializations own_callback_cast own_callback_interface_slot own_equal_strings_into_any own_any_copies own_array_copies own_string_pushed_twice own_rest_takes own_shares_elements own_rest_keeps_declared own_interface_getter_let own_callback_assigns_capture - own_param_capture_copies own_swap_local_field own_closure_keeps_param own_chain_returns_this) + own_param_capture_copies own_swap_local_field own_closure_keeps_param own_chain_returns_this own_callback_fewer_params) tslang_add_test(NAME test-jit-own-${own_test} COMMAND test-runner -jit -mm=own "${PROJECT_SOURCE_DIR}/test/tester/own/${own_test}.ts") tslang_add_test(NAME test-compile-own-${own_test} COMMAND test-runner -mm=own "${PROJECT_SOURCE_DIR}/test/tester/own/${own_test}.ts") # The promise: no counting at all in what own emits. @@ -4702,7 +4702,7 @@ foreach(keeps_mm rc gc none) own_callback_shapes own_callback_extended own_callback_interface own_generic_specializations own_callback_cast own_callback_interface_slot own_equal_strings_into_any own_rest_keeps_declared own_interface_getter_let own_callback_assigns_capture - own_param_capture_copies own_swap_local_field own_closure_keeps_param own_chain_returns_this) + own_param_capture_copies own_swap_local_field own_closure_keeps_param own_chain_returns_this own_callback_fewer_params) tslang_add_test(NAME test-jit-${keeps_mm}-${keeps_test} COMMAND test-runner -jit -mm=${keeps_mm} "${PROJECT_SOURCE_DIR}/test/tester/own/${keeps_test}.ts") tslang_add_test(NAME test-compile-${keeps_mm}-${keeps_test} COMMAND test-runner -mm=${keeps_mm} "${PROJECT_SOURCE_DIR}/test/tester/own/${keeps_test}.ts") endforeach() @@ -5089,7 +5089,7 @@ foreach(shared_mm ${shared_models}) endforeach() # WeakRef (own spec 84): built into the compiler under gc, none and own; rc has no weak count foreach(weakref_mm gc none own) - foreach(weakref_test 00weakref_basic 00weakref 00weakref_frame 00weakref_literal) + foreach(weakref_test 00weakref_basic 00weakref 00weakref_frame 00weakref_literal 00weakref_narrowed) tslang_add_test(NAME test-jit-${weakref_test}-${weakref_mm} COMMAND test-runner -jit -mm=${weakref_mm} "${PROJECT_SOURCE_DIR}/test/tester/tests/${weakref_test}.ts") tslang_add_test(NAME test-compile-${weakref_test}-${weakref_mm} COMMAND test-runner -mm=${weakref_mm} "${PROJECT_SOURCE_DIR}/test/tester/tests/${weakref_test}.ts") endforeach() @@ -6718,7 +6718,7 @@ tslang_add_test(NAME test-compile-debug-shared-export-import-class-implements-in tslang_add_test(NAME test-jit-debug-shared-export-import-class-implements-interface-abstract COMMAND test-runner -jit -shared -noopt "${PROJECT_SOURCE_DIR}/test/tester/tests/import_class_implements_interface_abstract.ts" "${PROJECT_SOURCE_DIR}/test/tester/tests/export_class_implements_interface_abstract.ts") foreach(debug_own_test own/own_interface_getter_read own/own_immortal_keepers tests/00nullable_interface_to_interface stack-promotion/to_any tests/00linked_list tests/00linked_list_kept tests/00linked_list_build - tests/00weakref_basic tests/00weakref tests/00weakref_frame tests/00weakref_literal) + tests/00weakref_basic tests/00weakref tests/00weakref_frame tests/00weakref_literal tests/00weakref_narrowed) get_filename_component(debug_own_name ${debug_own_test} NAME) tslang_add_test(NAME test-jit-debug-own-${debug_own_name} COMMAND test-runner -jit -mm=own -noopt "${PROJECT_SOURCE_DIR}/test/tester/${debug_own_test}.ts") tslang_add_test(NAME test-compile-debug-own-${debug_own_name} COMMAND test-runner -mm=own -noopt "${PROJECT_SOURCE_DIR}/test/tester/${debug_own_test}.ts") diff --git a/tslang/test/tester/own/own_callback_fewer_params.ts b/tslang/test/tester/own/own_callback_fewer_params.ts new file mode 100644 index 000000000..1d5502929 --- /dev/null +++ b/tslang/test/tester/own/own_callback_fewer_params.ts @@ -0,0 +1,59 @@ +// A callback that declares fewer parameters than its type (`(v) => ...` for `(value, key, map) => void`) +// is called through an adapter that passes on what it is given. A callback that keeps a string it is +// lent (`names.push(k)`) gets a copy, as one that declares every parameter does: Map and Set's forEach +// lend each entry's key and value out of their entries. +class Entry { + constructor(public key: string, public value: string) {} +} + +class Table { + entries: Entry[] = []; + tuples: [string, number][] = []; + + add(key: string, value: string, tupleKey: string) { + this.entries.push(new Entry(key, value)); + this.tuples.push([tupleKey, 1]); + } + + forEach(f: (value: string, key: string, table: Table) => void) { + for (let i = 0; i < this.entries.length; i++) { + f(this.entries[i].value, this.entries[i].key, this); + } + } + + forEachTuple(f: (key: string, count: number, table: Table) => void) { + for (let i = 0; i < this.tuples.length; i++) { + f(this.tuples[i][0], this.tuples[i][1], this); + } + } +} + +function run(round: number) { + // strings made at run time, so that the table's are freed when it drops them + const t = new Table(); + t.add("a" + round, "x" + round, "ta" + round); + t.add("b" + round, "y" + round, "tb" + round); + + const values: string[] = []; + t.forEach((v) => { values.push(v); }); + const keys: string[] = []; + t.forEach((v, k) => { keys.push(k); }); + const fromTuples: string[] = []; + t.forEachTuple((k) => { fromTuples.push(k); }); + + // the table's entries go: what the callbacks kept is their own + t.entries = []; + t.tuples = []; + t.add("c" + round, "z" + round, "tc" + round); + assert(values.length == 2 && values[0] == "x" + round && values[1] == "y" + round, "values"); + assert(keys.length == 2 && keys[0] == "a" + round && keys[1] == "b" + round, "keys"); + assert(fromTuples.length == 2 && fromTuples[1] == "tb" + round, "tuple keys"); +} + +function main() { + for (let i = 0; i < 200; i++) { + run(i); + } + + print("done."); +} diff --git a/tslang/test/tester/tests/00weakref_narrowed.ts b/tslang/test/tester/tests/00weakref_narrowed.ts new file mode 100644 index 000000000..e24ae4c4a --- /dev/null +++ b/tslang/test/tester/tests/00weakref_narrowed.ts @@ -0,0 +1,19 @@ +// A target whose declared type allows null, narrowed by its assignment (`let n: Node | null = new +// Node()`): `new WeakRef(n)` makes a WeakRef, as TypeScript reads n there. A null target makes one +// whose deref() gives undefined. +class Node { + constructor(public v: number) {} +} + +function main() { + let n: Node | null = new Node(42); + const w = new WeakRef(n); + const u: WeakRef | string = w; + assert(w.deref()?.v == 42, "deref"); + assert(typeof u != "string", "a WeakRef"); + // a null one refers to nothing + let m: Node | null = null; + const none = new WeakRef(m); + assert(none.deref() === undefined, "null target"); + print("done."); +} diff --git a/tslang/test/tester/verify-ownership.cmake b/tslang/test/tester/verify-ownership.cmake index 18bdf5ec1..5c78d56d6 100644 --- a/tslang/test/tester/verify-ownership.cmake +++ b/tslang/test/tester/verify-ownership.cmake @@ -40,6 +40,7 @@ set(not_compilable_alone 00weakref_basic.ts 00weakref_frame.ts 00weakref_literal.ts + 00weakref_narrowed.ts # a @cpp class is an error under rc and own (cppvtable/memory_model.ts) 00cpp_slot.ts 00cpp_overloads.ts