Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions TSLANG_ISSUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,127 @@ a local `values` broke the default library's `Array<T>`): a generic declared out
now sees no local of another function; one declared in a function still reads that function's locals,
and lambdas in a generic still capture the generic's own. Regression test
`tslang/test/tester/tests/00tslang_issue_30.ts`.
Items 34-36, found in the default library on 2026-11-08, are fixed or answered too: 34 (a string
literal starting with U+FEFF lost it, so its `.length` was 0): LLVM's UTF-16 to UTF-8
conversion took a leading U+FEFF for a byte order mark and skipped it, and would have byte-swapped a
literal starting with U+FFFE. Literals are now converted as they are. 35 (`"\\n"` read as a line feed)
did not reproduce: on the same build the repro prints `2 6 4 2 2 3` and `92 2`, as expected. The repro
file was most likely written through a shell that turns `\\` into `\`, which gives exactly the output
reported. Both are covered by `tslang/test/tester/tests/00string_literal_escapes.ts`. 36 (an older
runtime on `TSLANG_LIB_PATH` failed every `--emit=dll` with a bare LNK2001 for `__tslang_heap_shared`):
tslang now checks the runtime's `TypeScriptAsyncRuntime.lib` before linking a library and says it is
older than the compiler. `-v` is an alias for `--verbose`, which prints the linker's command line, so
the linker's "use -v" hint works.

Items 31-33 are `-mm=own` limits found in the default library (TypeScriptCompilerDefaultLib), not in
native_ts_graphics. There the tests `thread_spawn.ts` and `atomics_wait.ts` are marked `// not own`
because of them. Each repro was compiled with `--emit=exe` (the JIT's default-library DLL didn't
link against this build) and runs under `-mm=gc` with the output shown.

### 31. `-mm=own`: two closures can't capture the same array

tslang (commit 94b8bdb2, built with uncommitted local changes). Found 2026-11-08.

A closure passed to `Thread.spawn` moves what it captures, so a second closure capturing the same
`Int32Array` is a use after move. Threads can't share memory under `-mm=own`, which is what
`Atomics` is for. A single capture is fine: in the third repro the main thread still reads the
array after the thread captured it. The error names `'this value'`, not `counter`, and points at the
second arrow function rather than at `counter` inside it.

Repro (`share_two.ts`):
```ts
// -mm=own: two threads that capture the same Int32Array
function main() {
const counter = new Int32Array([0]);
const a = Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; });
const b = Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; });
a.join();
b.join();
print(Atomics.load(counter, 0));
}
```

`-mm=gc` prints `2`. `-mm=own`:
```
share_two.ts:5:28: error: 'this value' is used here after its value was moved
const b = Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; });
^
share_two.ts:4:28: note: value moved here
const a = Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; });
^
```

The same in a loop (`share_loop.ts`), the usual way to start workers, gives a different error:
```ts
// -mm=own: threads spawned in a loop, each capturing the same Int32Array
function main() {
const counter = new Int32Array([0]);
const workers: ThreadHandle<int>[] = [];
for (let t = 0; t < 4; t++) {
workers.push(Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; }));
}

for (const w of workers) w.join();
print(Atomics.load(counter, 0));
}
```

`-mm=gc` prints `4`. `-mm=own`:
```
share_loop.ts:6:35: error: 'this value' is moved inside a loop but was made outside it; only a borrow could do that, and -mm=own cannot prove one yet
workers.push(Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; }));
^
```

One capture compiles and runs under both models, printing `1` (`share_after.ts`):
```ts
// -mm=own: the main thread uses the Int32Array after a thread captured it
function main() {
const counter = new Int32Array([0]);
const a = Thread.spawn((): int => { Atomics.add(counter, 0, 1); return 0; });
a.join();
print(Atomics.load(counter, 0));
}
```

Expected: a way to share one array between threads under `-mm=own`, such as `Shared<T>` captured by
each closure, or a capture that borrows when the closure can't outlive the array.

### 32. `-mm=own`: the error for a move in a loop doesn't say what was moved

tslang (commit 94b8bdb2, built with uncommitted local changes). Found 2026-11-08.

Both errors in 31 call the captured variable `'this value'`. In a program with several captures,
nothing says which one is moved. The error should name `counter` and point at its use inside the
arrow function.

### 33. `-mm=own`: a thread can't return an object, and the error points into the default library

tslang (commit 94b8bdb2, built with uncommitted local changes). Found 2026-11-08.

`Thread.spawn` keeps the function's result in a field of `ThreadStart<R>` for `join()` to return.
When `R` is a class, storing it is rejected. The error is reported in the default library's
`lib.generics.ts` (`this.result = result;` in `ThreadStart.run`) with no line in the user's program,
so nothing points at the `Thread.spawn` call that caused it. A string or a number works.

Repro (`return_object.ts`):
```ts
// -mm=own: a thread whose function returns an object
class Box {
constructor(public value: int) {}
}

function main() {
const t = Thread.spawn((): Box => new Box(42));
print(t.join().value);
}
```

`-mm=gc` prints `42`. `-mm=own`:
```
.\__build\defaultlib\generics/lib.generics.ts:857:13: error: 'this value' takes a second reference; -mm=own cannot prove a move or a borrow here yet
```

`const result = this.fn(); this.result = result;` moves a value the function just returned, which
the field should be able to take. Expected: the move is accepted. Failing that, the error should
also name the instantiating call in the user's file.
58 changes: 36 additions & 22 deletions tslang/include/TypeScript/MLIRLogic/MLIRHelper.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,36 +24,50 @@ namespace typescript
// codecvt_utf8_utf16, as TypeScript counts a string in UTF-16 units - so a character outside the BMP
// is a surrogate pair even where wchar_t has 32 bits (Linux). llvm::convertWideToUTF8 takes a 32-bit
// wchar_t string for UTF-32, where a surrogate is invalid, and gave "" for a literal such as "😀".
//
// The units are converted as they are: llvm::convertUTF16ToUTF8String reads a leading U+FEFF as a byte
// order mark and skips it (and a leading U+FFFE byte-swaps the whole string), so a string literal
// starting with \uFEFF lost it (TSLANG_ISSUES 34).
static std::string convertWideToUTF8(const std::wstring &ws)
{
std::string s;
if constexpr (sizeof(wchar_t) == sizeof(llvm::UTF16))
{
llvm::convertWideToUTF8(ws, s);
}
else
llvm::SmallVector<llvm::UTF16> units;
units.reserve(ws.size());
for (auto c : ws)
{
llvm::SmallVector<llvm::UTF16> units;
units.reserve(ws.size());
for (auto c : ws)
auto code = static_cast<uint32_t>(c);
if (code > 0xFFFF && code <= 0x10FFFF)
{
auto code = static_cast<uint32_t>(c);
if (code > 0xFFFF && code <= 0x10FFFF)
{
// a code point made elsewhere (not by stows) - its surrogate pair
code -= 0x10000;
units.push_back(static_cast<llvm::UTF16>(0xD800 + (code >> 10)));
units.push_back(static_cast<llvm::UTF16>(0xDC00 + (code & 0x3FF)));
}
else
{
units.push_back(static_cast<llvm::UTF16>(code));
}
// a code point made elsewhere (not by stows) - its surrogate pair
code -= 0x10000;
units.push_back(static_cast<llvm::UTF16>(0xD800 + (code >> 10)));
units.push_back(static_cast<llvm::UTF16>(0xDC00 + (code & 0x3FF)));
}
else
{
units.push_back(static_cast<llvm::UTF16>(code));
}
}

llvm::convertUTF16ToUTF8String(units, s);
std::string s;
if (units.empty())
{
return s;
}

// room for every unit, as convertUTF16ToUTF8String allocates
s.resize(units.size() * UNI_MAX_UTF8_BYTES_PER_CODE_POINT + 1);
const llvm::UTF16 *src = units.begin();
auto *dstStart = reinterpret_cast<llvm::UTF8 *>(&s[0]);
auto *dst = dstStart;
auto result = llvm::ConvertUTF16toUTF8(&src, units.end(), &dst, dstStart + s.size(), llvm::strictConversion);
if (result != llvm::conversionOK)
{
// as convertUTF16ToUTF8String: an invalid sequence (a lone surrogate) gives ""
s.clear();
return s;
}

s.resize(dst - dstStart);
return s;
}

Expand Down
6 changes: 6 additions & 0 deletions tslang/test/tester/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1814,6 +1814,12 @@ foreach(issues_mm gc rc none own)
tslang_add_test(NAME test-jit-00-tslang-issue-29-thread-${issues_mm} COMMAND test-runner -jit -mm=${issues_mm} "${PROJECT_SOURCE_DIR}/test/tester/tests/${issue_29_thread_test}")
tslang_add_test(NAME test-compile-00-tslang-issue-29-thread-${issues_mm} COMMAND test-runner -mm=${issues_mm} "${PROJECT_SOURCE_DIR}/test/tester/tests/${issue_29_thread_test}")
endforeach()
# TSLANG 34/35: a string literal holds what its escapes say: a leading U+FEFF is kept, and "\\n" is a
# backslash and n
foreach(issues_mm gc rc none own)
tslang_add_test(NAME test-jit-00-string-literal-escapes-${issues_mm} COMMAND test-runner -jit -mm=${issues_mm} "${PROJECT_SOURCE_DIR}/test/tester/tests/00string_literal_escapes.ts")
tslang_add_test(NAME test-compile-00-string-literal-escapes-${issues_mm} COMMAND test-runner -mm=${issues_mm} "${PROJECT_SOURCE_DIR}/test/tester/tests/00string_literal_escapes.ts")
endforeach()
# an Opaque out of an `any` is as wide as the value in the box: `<Opaque><any>0` is null (the
# padding after a 4-byte value was read as the pointer's high bytes)
foreach(issues_mm gc rc none own)
Expand Down
28 changes: 28 additions & 0 deletions tslang/test/tester/tests/00string_literal_escapes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// A string literal holds what its escapes say, and .length counts its UTF-8 bytes.
// TSLANG_ISSUES 34: a literal starting with U+FEFF lost it - the conversion to UTF-8 took it for a
// byte order mark. TSLANG_ISSUES 35: "\\" is one backslash, so "\\n" is the two characters \ and n.

function main() {
// 34: U+FEFF is three bytes wherever it is, written as an escape or as itself
assert("\uFEFF".length == 3, "a literal that is U+FEFF");
assert("\uFEFFx".length == 4, "a literal starting with U+FEFF");
assert("a\uFEFFb".length == 5, "U+FEFF in the middle");
assert("x".length == 4, "a literal starting with U+FEFF typed as is");
assert("\uFEFFx" == "\u{FEFF}x", "the same as the code point escape");
// U+FFFE is no byte-swapped order mark either: the rest is not swapped
assert("\uFFFEab".length == 5, "a literal starting with U+FFFE");
assert("\uFFFEab" != "\uFFFE\u6100\u6200", "keeps its other characters as they are");
assert("\u00E9".length == 2 && "\u200B".length == 3 && "\uFFFD".length == 3, "other escapes");

// 35: an escaped backslash is one backslash, and what follows it is itself
assert("\\n".length == 2, "\\n");
assert("\\u0041".length == 6, "\\u0041");
assert("\\x41".length == 4, "\\x41");
assert("\\d".length == 2, "\\d");
assert("\\\\".length == 2, "\\\\");
assert("a\\b".length == 3, "a\\b");
const s = "\\n";
assert(s == "\x5Cn" && s == "\u005Cn" && s != "\n", "a backslash, then n");

print("done.");
}
39 changes: 39 additions & 0 deletions tslang/tslang/exe.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,40 @@ static bool resolveWindowsLibPath(const llvm::Triple &triple, std::string &path,
return checkWindowsBinaryMachine(triple, lib);
}

// A library tslang builds on Windows links with `-include:__tslang_heap_shared`, which the runtime
// (TypeScriptAsyncRuntime.lib) defines since #784. A runtime older than that failed every
// --emit=dll with a bare "unresolved external symbol __tslang_heap_shared" (TSLANG_ISSUES 36): say
// which runtime it is and that it is older than the compiler. `path` is the directory the link
// takes the runtime from (resolveWindowsLibPath has made it the x86 one for x86); a runtime not
// found there is the linker's to report.
static bool checkRuntimeSharesHeap(llvm::StringRef path)
{
if (path.empty())
{
return true;
}

llvm::SmallString<256> lib(path);
llvm::sys::path::append(lib, "TypeScriptAsyncRuntime.lib");
auto buffer = llvm::MemoryBuffer::getFile(lib, /*IsText=*/false, /*RequiresNullTerminator=*/false);
if (!buffer)
{
return true;
}

// the archive's symbol table names it (x86 decorates it with one more underscore)
if ((*buffer)->getBuffer().contains("__tslang_heap_shared"))
{
return true;
}

llvm::WithColor::error(llvm::errs(), "tslang")
<< "the TypeScript runtime " << lib << " is older than this tslang: it has no __tslang_heap_shared, "
<< "which a library (--emit=dll) links with. Point --tslang-lib-path (or TSLANG_LIB_PATH) at the "
<< "runtime that came with this tslang.\n";
return false;
}

void addCommandArgs(clang::driver::Compilation *c, llvm::ArrayRef<const char*> cmdParts)
{
for (auto &job : c->getJobs())
Expand Down Expand Up @@ -975,6 +1009,11 @@ int linkBinary(const char *argv0, const LinkRequest &request, CompileOptions &co
return 1;
}

if (shared && winX86OrX64 && !checkRuntimeSharesHeap(tslangLibPath))
{
return 1;
}

tslangLibPathOpt = getLibsPathOpt(tslangLibPath);
if (!tslangLibPathOpt.empty())
{
Expand Down
4 changes: 3 additions & 1 deletion tslang/tslang/tslang.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,9 @@ cl::opt<bool> jitCache{"jit-cache", cl::desc("--emit=jit: compile the program an
cl::opt<bool> jitNative{"jit-native", cl::desc("--emit=jit on Windows: link the JIT cache's objects into a DLL with a PDB and load it with the system loader, so that WinDbg and Visual Studio can debug the program with its symbols and source (implies --di unless --opt is given; needs a linker and the JIT cache)"), cl::cat(TypeScriptCompilerCategory)};
cl::opt<std::string> jitCacheDir{"jit-cache-dir", cl::desc("--emit=jit: the folder the JIT cache keeps its objects in (default: '__jit' beside each source file)"), cl::value_desc("folder"), cl::cat(TypeScriptCompilerCategory)};

cl::opt<bool> verbose{"verbose", cl::Hidden, cl::desc("Verbose output"), cl::init(false), cl::cat(TypeScriptCompilerDebugCategory)};
cl::opt<bool> verbose{"verbose", cl::desc("Verbose output: print the linker's command line and what else is run"), cl::init(false), cl::cat(TypeScriptCompilerDebugCategory)};
// the linker's own failure message says "use -v to see invocation" (TSLANG_ISSUES 36)
cl::alias verboseShort{"v", cl::desc("Alias for --verbose"), cl::aliasopt(verbose)};
cl::opt<bool> printOp{"print-op", cl::Hidden, cl::desc("Print Op on Diagnostic"), cl::init(false), cl::cat(TypeScriptCompilerDebugCategory)};
cl::opt<bool> printStackTrace{"print-stack-trace", cl::Hidden, cl::desc("Print stack trace on Diagnostic"), cl::init(false), cl::cat(TypeScriptCompilerDebugCategory)};

Expand Down
Loading