From efa3b4933fd906bd11a0ba34377fb1e2e59fd327 Mon Sep 17 00:00:00 2001 From: ASDAlexander77 Date: Sun, 8 Nov 2026 01:20:15 +0000 Subject: [PATCH 1/2] A library tslang built allocates from the program's heap A block made by one module is routinely freed by another, so a process has one heap. Until now a program that loaded a library tslang built - a dynamic import, the default library's DLL under the JIT, a --shared-libs library - stayed on the process heap so its blocks and the library's matched. Now the heap is the program's, mimalloc included, and the libraries borrow it: - Every heap entry point is exported: TypeScriptMimalloc's in an executable, the async runtime's process-heap ones otherwise, and tslang.exe's own under the JIT. - lib/ProcessHeapExports.inc, in every library tslang builds and in TypeScriptRuntime.dll, looks the five up in the process's executable at its first allocation (lock-free, all five or none) and calls them; in the program's own module, or under a host that exports none, it is the process heap as before. It exports `__tslang_heap_shared` to say so, and a DLL link forces it in even when the library never allocates. - A program on mimalloc (TypeScriptMimalloc exports `__tslang_heap_mimalloc`) refuses, before any of its code runs, a library without the marker - one an older tslang built, which keeps a heap of its own - and says to rebuild it (DynamicRuntime, and the JIT's loader). The JIT chooses its heap before loading any library, and a --shared-libs library without the marker keeps it on the process heap (ProcessHeap.h, librarySharesHeap). - exe.cpp and test-runner give an executable importing libraries mimalloc too; every -shared rc/none/own test now runs its library on the program's mimalloc. Checked with a host that exports mimalloc's entry points: the library's own allocation is a mimalloc block; without the exports it is not. An ordinary JIT run, with the default library's DLL, now gets mimalloc. The default library has to be rebuilt to carry the marker; an older build keeps the JIT on the process heap. Co-Authored-By: Claude Opus 5.5 --- docs/memory-models.md | 18 ++- tslang/include/TypeScript/ProcessHeap.h | 33 +++- tslang/lib/ProcessHeapExports.inc | 129 +++++++++++++++- .../TypeScriptAsyncRuntime/DynamicRuntime.cpp | 17 +++ tslang/lib/TypeScriptMimalloc/MimallocHeap.c | 24 +-- tslang/lib/TypeScriptRuntime/MemRuntime.cpp | 9 +- .../TypeScriptRuntime/TypeScriptRuntime.def | 7 +- tslang/test/tester/CMakeLists.txt | 4 +- tslang/test/tester/mimalloc-heap.cmake | 23 ++- tslang/test/tester/test-runner.cpp | 18 ++- tslang/tslang/exe.cpp | 19 ++- tslang/tslang/jit.cpp | 142 ++++++++++-------- 12 files changed, 327 insertions(+), 116 deletions(-) diff --git a/docs/memory-models.md b/docs/memory-models.md index 5b52bd1b7..da1f6b149 100644 --- a/docs/memory-models.md +++ b/docs/memory-models.md @@ -198,12 +198,18 @@ Under `-mm=rc`, `-mm=own` and `-mm=none`, a Windows program allocates from `TypeScriptAsyncRuntime.lib`); - a program run with `--emit=jit` gets the same. -There is one exception, because a block allocated by one module is often freed by another. A -program that loads a library tslang built keeps the process heap, and so does that library. That -covers importing a `--emit=dll` library, the default library's DLL under the JIT, and a library -given with `--shared-libs`. A program you link yourself, against `TypeScriptAsyncRuntime.lib`, also -keeps the process heap unless you link `TypeScriptMimalloc.lib` ahead of it. Do that only for an -executable that loads no library tslang built. +A block allocated by one module is often freed by another, so a process has one heap, and it is +the program's. A library tslang built - a `--emit=dll` library the program imports, the default +library's DLL under the JIT, a library given with `--shared-libs` - allocates from the program's +heap, which it finds through the entry points the program exports. Under a program that exports +none (one tslang did not build), a library allocates from the process heap. + +A library built by an older tslang keeps a heap of its own. A program on mimalloc refuses to load it +and says to rebuild it; under the JIT, such a library given with `--shared-libs` keeps the program +on the process heap instead. + +A program you link yourself, against `TypeScriptAsyncRuntime.lib`, keeps the process heap unless you +link `TypeScriptMimalloc.lib` ahead of it. `--process-heap` keeps the process heap anyway, for heap debugging tools such as page heap or Application Verifier. `-mm=gc` allocates from the collector, and other platforms from the C diff --git a/tslang/include/TypeScript/ProcessHeap.h b/tslang/include/TypeScript/ProcessHeap.h index f7ae4c159..66278075f 100644 --- a/tslang/include/TypeScript/ProcessHeap.h +++ b/tslang/include/TypeScript/ProcessHeap.h @@ -11,9 +11,11 @@ // list of its blocks per copy, while tslang.exe's own `malloc` is rpmalloc when LLVM is the // prebuilt package. The process heap is the same heap in every module and every build. // -// An executable that loads no library tslang built is one module, so it allocates from mimalloc -// instead: tslang links TypeScriptMimalloc.lib, whose definitions of the same entry points win over -// these (lib/TypeScriptMimalloc), and the JIT does the same for the program it runs (jit.cpp). +// The process's one heap is the program's, and an executable tslang links has mimalloc for it +// (TypeScriptMimalloc.lib, whose definitions of the entry points win over these), as does a program +// the JIT runs (jit.cpp). A library tslang built allocates from the program's heap through the +// entry points the program exports, and from the process heap only when the program exports none +// (lib/ProcessHeapExports.inc). // // Header-only, so each place that has to provide them - the async runtime library a program links, // TypeScriptRuntime.dll and the JIT - compiles the same code. @@ -91,6 +93,31 @@ inline void *allocateAligned(size_t align, size_t size) return allocate(size); } +// Whether the program allocates from mimalloc (an executable that links TypeScriptMimalloc exports +// this marker; the JIT knows for itself). +inline bool programUsesMimalloc() +{ + return GetProcAddress(GetModuleHandleW(nullptr), "__tslang_heap_mimalloc") != nullptr; +} + +// Whether a library allocates from the program's heap (lib/ProcessHeapExports.inc exports the +// marker) rather than keeping one of its own, as a library a tslang before it built does. Read from +// the library mapped without running any of its code or loading what it imports, found where +// LoadLibrary would find it - so it can be asked before the library is loaded. A library that is not +// there says yes: loading it fails on its own. +inline bool librarySharesHeap(const wchar_t *fileName) +{ + auto library = LoadLibraryExW(fileName, nullptr, DONT_RESOLVE_DLL_REFERENCES); + if (library == nullptr) + { + return true; + } + + auto shares = GetProcAddress(library, "__tslang_heap_shared") != nullptr; + FreeLibrary(library); + return shares; +} + } // namespace process_heap } // namespace typescript diff --git a/tslang/lib/ProcessHeapExports.inc b/tslang/lib/ProcessHeapExports.inc index a7acadeda..b00f43cdc 100644 --- a/tslang/lib/ProcessHeapExports.inc +++ b/tslang/lib/ProcessHeapExports.inc @@ -1,34 +1,149 @@ -// The process-heap allocator under the names ProcessHeapPass calls (PROCESS_HEAP_* in +// The heap entry points under the names ProcessHeapPass calls (PROCESS_HEAP_* in // TypeScript/Defines.h). Included by the async runtime library, which a program and a library link, -// and by TypeScriptRuntime.dll, which the JIT loads; see TypeScript/ProcessHeap.h for why. +// and by TypeScriptRuntime.dll, which the JIT loads; see TypeScript/ProcessHeap.h for why there is +// one heap per process. +// +// The program owns the heap. In the program's own module these are the process heap; an executable +// that links TypeScriptMimalloc gets that library's definitions instead (mimalloc), and tslang.exe +// gives a program it runs its own (jit.cpp). Every one of them is exported, so a library - a +// module other than the process's executable - looks them up there at its first allocation and +// calls the program's: the library's blocks and the program's come from one heap, whichever it is. +// A program that exports none (a host tslang did not build, or one built before this) leaves a +// library on the process heap, which is what such a program allocates from too. +// +// `__tslang_heap_shared` says a module does this, so that a program on mimalloc can refuse a library +// that would keep a heap of its own (ProcessHeap.h, librarySharesHeap). #ifdef _WIN32 +#include + +#include "TypeScript/Defines.h" #include "TypeScript/ProcessHeap.h" -extern "C" void *__tslang_heap_malloc(size_t size) +extern "C" __declspec(dllexport) const int __tslang_heap_shared = 1; + +namespace +{ + +using HeapMallocFn = void *(*)(size_t); +using HeapCallocFn = void *(*)(size_t, size_t); +using HeapReallocFn = void *(*)(void *, size_t); +using HeapFreeFn = void (*)(void *); +using HeapAlignedAllocFn = void *(*)(size_t, size_t); + +void *processHeapMalloc(size_t size) { return typescript::process_heap::allocate(size); } -extern "C" void *__tslang_heap_calloc(size_t count, size_t size) +void *processHeapCalloc(size_t count, size_t size) { return typescript::process_heap::allocateZeroed(count, size); } -extern "C" void *__tslang_heap_realloc(void *ptr, size_t size) +void *processHeapRealloc(void *ptr, size_t size) { return typescript::process_heap::reallocate(ptr, size); } -extern "C" void __tslang_heap_free(void *ptr) +void processHeapFree(void *ptr) { typescript::process_heap::release(ptr); } -extern "C" void *__tslang_heap_aligned_alloc(size_t alignment, size_t size) +void *processHeapAlignedAlloc(size_t alignment, size_t size) { return typescript::process_heap::allocateAligned(alignment, size); } +// Filled once, by whichever thread allocates first; every thread that races to it computes the +// same five functions. The library's own global constructors may allocate before anything else +// in it runs, so this is not left to a constructor of this file. +std::atomic heapMalloc{nullptr}; +std::atomic heapCalloc{nullptr}; +std::atomic heapRealloc{nullptr}; +std::atomic heapFree{nullptr}; +std::atomic heapAlignedAlloc{nullptr}; + +void resolveHeap() +{ + HeapMallocFn mallocFn = processHeapMalloc; + HeapCallocFn callocFn = processHeapCalloc; + HeapReallocFn reallocFn = processHeapRealloc; + HeapFreeFn freeFn = processHeapFree; + HeapAlignedAllocFn alignedAllocFn = processHeapAlignedAlloc; + + // the program's heap, unless this is the program + HMODULE self = nullptr; + auto program = GetModuleHandleW(nullptr); + if (GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS | GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT, + reinterpret_cast(&resolveHeap), &self) + && program != nullptr && program != self) + { + auto programMalloc = reinterpret_cast(GetProcAddress(program, PROCESS_HEAP_MALLOC)); + auto programCalloc = reinterpret_cast(GetProcAddress(program, PROCESS_HEAP_CALLOC)); + auto programRealloc = reinterpret_cast(GetProcAddress(program, PROCESS_HEAP_REALLOC)); + auto programFree = reinterpret_cast(GetProcAddress(program, PROCESS_HEAP_FREE)); + auto programAlignedAlloc = + reinterpret_cast(GetProcAddress(program, PROCESS_HEAP_ALIGNED_ALLOC)); + // all five or none: half of one heap and half of another frees blocks on the wrong one + if (programMalloc && programCalloc && programRealloc && programFree && programAlignedAlloc) + { + mallocFn = programMalloc; + callocFn = programCalloc; + reallocFn = programRealloc; + freeFn = programFree; + alignedAllocFn = programAlignedAlloc; + } + } + + heapCalloc.store(callocFn, std::memory_order_relaxed); + heapRealloc.store(reallocFn, std::memory_order_relaxed); + heapFree.store(freeFn, std::memory_order_relaxed); + heapAlignedAlloc.store(alignedAllocFn, std::memory_order_relaxed); + // last, and released: a thread that sees malloc sees the other four + heapMalloc.store(mallocFn, std::memory_order_release); +} + +void ensureHeap() +{ + if (!heapMalloc.load(std::memory_order_acquire)) + { + resolveHeap(); + } +} + +} // namespace + +extern "C" __declspec(dllexport) void *__tslang_heap_malloc(size_t size) +{ + ensureHeap(); + return heapMalloc.load(std::memory_order_relaxed)(size); +} + +extern "C" __declspec(dllexport) void *__tslang_heap_calloc(size_t count, size_t size) +{ + ensureHeap(); + return heapCalloc.load(std::memory_order_relaxed)(count, size); +} + +extern "C" __declspec(dllexport) void *__tslang_heap_realloc(void *ptr, size_t size) +{ + ensureHeap(); + return heapRealloc.load(std::memory_order_relaxed)(ptr, size); +} + +extern "C" __declspec(dllexport) void __tslang_heap_free(void *ptr) +{ + ensureHeap(); + heapFree.load(std::memory_order_relaxed)(ptr); +} + +extern "C" __declspec(dllexport) void *__tslang_heap_aligned_alloc(size_t alignment, size_t size) +{ + ensureHeap(); + return heapAlignedAlloc.load(std::memory_order_relaxed)(alignment, size); +} + #endif // _WIN32 diff --git a/tslang/lib/TypeScriptAsyncRuntime/DynamicRuntime.cpp b/tslang/lib/TypeScriptAsyncRuntime/DynamicRuntime.cpp index c54129574..b016e17d4 100644 --- a/tslang/lib/TypeScriptAsyncRuntime/DynamicRuntime.cpp +++ b/tslang/lib/TypeScriptAsyncRuntime/DynamicRuntime.cpp @@ -20,6 +20,10 @@ #define WIN32_LEAN_AND_MEAN #define NOMINMAX #include + +#include + +#include "TypeScript/ProcessHeap.h" #else #include #endif @@ -56,6 +60,19 @@ void *openLibrary(const char *fileName) return nullptr; } + // A program on mimalloc lends it to every library it loads (ProcessHeapExports.inc). A library + // that keeps a heap of its own - one a tslang before that built - would free the program's + // blocks on it, and the program its: refused here, before any of its code runs. + if (typescript::process_heap::programUsesMimalloc() + && !typescript::process_heap::librarySharesHeap(wideFileName.c_str())) + { + fprintf(stderr, + "tslang runtime: %s does not allocate from the program's heap (a tslang before this one built it); " + "rebuild it, or build the program with --process-heap\n", + fileName); + return nullptr; + } + return reinterpret_cast(LoadLibraryW(wideFileName.c_str())); #else return dlopen(fileName, RTLD_LAZY | RTLD_GLOBAL); diff --git a/tslang/lib/TypeScriptMimalloc/MimallocHeap.c b/tslang/lib/TypeScriptMimalloc/MimallocHeap.c index 3aad21947..c290d83d4 100644 --- a/tslang/lib/TypeScriptMimalloc/MimallocHeap.c +++ b/tslang/lib/TypeScriptMimalloc/MimallocHeap.c @@ -4,10 +4,10 @@ // TypeScriptAsyncRuntime's ProcessHeap.cpp is never pulled in. // // One allocator per process is still the rule (TypeScript/ProcessHeap.h): a block made by one module -// is freed by another. A program linked whole - its own code, the static default library and -// everything else it links statically - is one module, so it can own a mimalloc of its own. A -// program that loads a library tslang built (--emit=dll, a dynamic import) does not get this -// library: that library allocates from the process heap, and so does the program (exe.cpp). +// is freed by another. The program owns this mimalloc - its own code, the static default library and +// everything else it links statically - and a library tslang built that it loads allocates from it +// as well, through the exports below (ProcessHeapExports.inc). Only an executable links this: a +// library never owns the heap. // // The same rules as ProcessHeap.h where the C functions leave room: an allocation of nothing is a // block of its own, and a block resized to nothing is freed and nothing is returned. @@ -25,12 +25,18 @@ // neither tslang nor test-runner links advapi32 into a program otherwise #pragma comment(lib, "advapi32.lib") -void *__tslang_heap_malloc(size_t size) +// Exported, as the process-heap entry points are (ProcessHeapExports.inc): a library tslang built +// that this program loads looks them up here and allocates from this mimalloc too. The marker says +// the program is on mimalloc, so a library that would keep a heap of its own is refused +// (TypeScript/ProcessHeap.h, librarySharesHeap). +__declspec(dllexport) const int __tslang_heap_mimalloc = 1; + +__declspec(dllexport) void *__tslang_heap_malloc(size_t size) { return mi_malloc(size ? size : 1); } -void *__tslang_heap_calloc(size_t count, size_t size) +__declspec(dllexport) void *__tslang_heap_calloc(size_t count, size_t size) { if (count == 0 || size == 0) { @@ -41,7 +47,7 @@ void *__tslang_heap_calloc(size_t count, size_t size) return mi_calloc(count, size); } -void *__tslang_heap_realloc(void *ptr, size_t size) +__declspec(dllexport) void *__tslang_heap_realloc(void *ptr, size_t size) { if (ptr && size == 0) { @@ -52,12 +58,12 @@ void *__tslang_heap_realloc(void *ptr, size_t size) return mi_realloc(ptr, size); } -void __tslang_heap_free(void *ptr) +__declspec(dllexport) void __tslang_heap_free(void *ptr) { mi_free(ptr); } -void *__tslang_heap_aligned_alloc(size_t alignment, size_t size) +__declspec(dllexport) void *__tslang_heap_aligned_alloc(size_t alignment, size_t size) { return mi_malloc_aligned(size ? size : 1, alignment); } diff --git a/tslang/lib/TypeScriptRuntime/MemRuntime.cpp b/tslang/lib/TypeScriptRuntime/MemRuntime.cpp index 915b26620..95a35a5bc 100644 --- a/tslang/lib/TypeScriptRuntime/MemRuntime.cpp +++ b/tslang/lib/TypeScriptRuntime/MemRuntime.cpp @@ -35,14 +35,15 @@ namespace mlir namespace runtime { -// The heap these hand out is the process heap, which is what every module tslang builds allocates -// from and frees to (TypeScript/ProcessHeap.h): a coroutine frame this allocates, the program frees. +// The heap these hand out is the program's, which is what every module tslang builds allocates from +// and frees to (TypeScript/ProcessHeap.h): a coroutine frame this allocates, the program frees. Under +// the JIT that is tslang.exe's, found through the entry points it exports (ProcessHeapExports.inc). // This DLL's own `malloc` need not be that heap - a debug CRT keeps its blocks per copy, and against // the prebuilt LLVM, LLVMSupport makes it rpmalloc. #ifdef _WIN32 -static void *heapMalloc(size_t size) { return typescript::process_heap::allocate(size); } +static void *heapMalloc(size_t size) { return __tslang_heap_malloc(size); } -static void heapFree(void *ptr) { typescript::process_heap::release(ptr); } +static void heapFree(void *ptr) { __tslang_heap_free(ptr); } #else static void *heapMalloc(size_t size) { return malloc(size); } diff --git a/tslang/lib/TypeScriptRuntime/TypeScriptRuntime.def b/tslang/lib/TypeScriptRuntime/TypeScriptRuntime.def index 9fc69620e..92c0b50bd 100644 --- a/tslang/lib/TypeScriptRuntime/TypeScriptRuntime.def +++ b/tslang/lib/TypeScriptRuntime/TypeScriptRuntime.def @@ -36,12 +36,7 @@ EXPORTS aligned_alloc=AlignedAlloc aligned_free=AlignedFree - ; --- the process-heap allocator (ProcessHeapExports.inc) a JIT'd program calls on Windows --- - __tslang_heap_malloc - __tslang_heap_calloc - __tslang_heap_realloc - __tslang_heap_free - __tslang_heap_aligned_alloc + ; --- the heap entry points (ProcessHeapExports.inc) export themselves (__declspec(dllexport)) --- ; --- dynamic-library runtime (DynamicRuntime.cpp, extern "C", names already match) --- tslang_load_library_permanently diff --git a/tslang/test/tester/CMakeLists.txt b/tslang/test/tester/CMakeLists.txt index 3217f9e26..3cdb687e9 100644 --- a/tslang/test/tester/CMakeLists.txt +++ b/tslang/test/tester/CMakeLists.txt @@ -3067,8 +3067,8 @@ if (WIN32) "-DOPT=${gc_shared_auto_opt}" -P "${CMAKE_CURRENT_SOURCE_DIR}/shared-collector-auto.cmake") - # ...and the heap under the other models: mimalloc in an executable and under the JIT, the - # process heap beside a library tslang built - see mimalloc-heap.cmake. 00mimalloc_heap is the + # ...and the heap under the other models: mimalloc in an executable and under the JIT, lent to + # the libraries tslang built that the program loads - see mimalloc-heap.cmake. 00mimalloc_heap is the # program it builds; it runs under every model as an ordinary test too. add_test(NAME test-compile-mimalloc-heap COMMAND ${CMAKE_COMMAND} diff --git a/tslang/test/tester/mimalloc-heap.cmake b/tslang/test/tester/mimalloc-heap.cmake index fae0f9e7e..fc7d1da93 100644 --- a/tslang/test/tester/mimalloc-heap.cmake +++ b/tslang/test/tester/mimalloc-heap.cmake @@ -1,7 +1,8 @@ # The heap a Windows program allocates from (every memory model but gc): mimalloc -# (TypeScriptMimalloc.lib) in an executable tslang links, and under the JIT - unless the program -# loads a library tslang built, which allocates from the process heap and frees what the program -# allocated, or --process-heap asks for the process heap. +# (TypeScriptMimalloc.lib) in an executable tslang links, and under the JIT, unless --process-heap +# asks for the process heap. A library tslang built allocates from the program's heap, whichever it +# is (lib/ProcessHeapExports.inc); one that would keep a heap of its own keeps the JIT on the process +# heap. # # An executable that has mimalloc prints its statistics at exit when MIMALLOC_SHOW_STATS is set; # one that allocates from the process heap prints nothing. The JIT says which heap it gave the @@ -87,11 +88,17 @@ jit("mimalloc" -mm=own "${program}") jit("mimalloc" -mm=rc --jit-cache=false "${program}") jit("process heap" -mm=own --process-heap "${program}") -# 3. A program importing a library tslang built shares that library's process heap: the importer -# makes objects of the library's class, which the library's code releases. +# 3. A program importing a library tslang built lends it its heap: the importer makes objects of the +# library's class, which the library's code releases - as an executable, under the JIT, and as an +# executable on the process heap. run("--emit=dll" "${TSLANG}" --emit=dll ${common} -mm=rc "${TESTS_DIR}/export_class_interface.ts" -o export_class_interface.dll) # `import './export_class_interface'` finds the DLL in the working directory before the source -build(import.exe FALSE -mm=rc "${TESTS_DIR}/import_class_interface.ts") -jit("process heap" -mm=rc "${TESTS_DIR}/import_class_interface.ts") -jit("process heap" -mm=rc --jit-cache=false "${TESTS_DIR}/import_class_interface.ts") +build(import.exe TRUE -mm=rc "${TESTS_DIR}/import_class_interface.ts") +build(import_process.exe FALSE -mm=rc --process-heap "${TESTS_DIR}/import_class_interface.ts") +jit("mimalloc" -mm=rc "${TESTS_DIR}/import_class_interface.ts") +jit("mimalloc" -mm=rc --jit-cache=false "${TESTS_DIR}/import_class_interface.ts") + +# 4. A library given to the JIT that does not say it allocates from the program's heap - here one +# tslang did not build at all - keeps the program on the process heap. +jit("process heap" -mm=own "--shared-libs=$ENV{SystemRoot}/System32/version.dll" "${program}") diff --git a/tslang/test/tester/test-runner.cpp b/tslang/test/tester/test-runner.cpp index 855a5a89c..7b93ce6b4 100644 --- a/tslang/test/tester/test-runner.cpp +++ b/tslang/test/tester/test-runner.cpp @@ -161,13 +161,21 @@ std::string optVariantSuffix() } #ifdef WIN32 -// What tslang links into an executable it builds whole (exe.cpp): mimalloc, ahead of the async -// runtime, under every memory model but gc. Not for a program that loads a library of its test - -// that library allocates from the process heap, and so must the program. +// What tslang links into an executable (exe.cpp): mimalloc, ahead of the async runtime, under every +// memory model but gc. A library of a -shared test allocates from it too, through the entry points +// the program exports (lib/ProcessHeapExports.inc). std::string mimallocLib() { return memoryModel.empty() || memoryModel == "gc" ? std::string() : std::string("TypeScriptMimalloc.lib "); } + +// ...and what tslang links into a library: the marker saying it allocates from the program's heap, +// which a program on mimalloc checks before loading it - kept even in a library that never +// allocates. x86 decorates C names with an underscore. +std::string sharedHeapMarkerOpt() +{ + return x86 ? " /include:___tslang_heap_shared" : " /include:__tslang_heap_shared"; +} #endif std::string jitBatName() @@ -590,7 +598,7 @@ void createSharedMultiBatchFile(std::string tempOutputFileNameNoExt, std::vector << LIBS << TYPESCRIPT_LIB << GC_LIB << CMAKE_C_STANDARD_LIBRARIES << " /libpath:%GC_LIB_PATH% /libpath:%TSLANG_LIB_PATH%" << " /libpath:%LIBPATH% /libpath:%SDKPATH% /libpath:%UCRTPATH%" - << machineOpt() + << machineOpt() << sharedHeapMarkerOpt() << std::endl; shared_libs << shared_filenameNoExt << ".lib "; @@ -616,7 +624,7 @@ void createSharedMultiBatchFile(std::string tempOutputFileNameNoExt, std::vector batFile << shared_filenameNoExt << ".lib "; } - batFile << LIBS << TYPESCRIPT_LIB << GC_LIB << CMAKE_C_STANDARD_LIBRARIES + batFile << LIBS << mimallocLib() << TYPESCRIPT_LIB << GC_LIB << CMAKE_C_STANDARD_LIBRARIES << " /libpath:%GC_LIB_PATH% /libpath:%TSLANG_LIB_PATH%" << " /libpath:%LIBPATH% /libpath:%SDKPATH% /libpath:%UCRTPATH%" << machineOpt() << exeManifestOpt() diff --git a/tslang/tslang/exe.cpp b/tslang/tslang/exe.cpp index cc0808d4e..f30342356 100644 --- a/tslang/tslang/exe.cpp +++ b/tslang/tslang/exe.cpp @@ -982,11 +982,10 @@ int linkBinary(const char *argv0, const LinkRequest &request, CompileOptions &co } // An executable allocates from mimalloc (TypeScriptMimalloc.lib) rather than the process - // heap, as long as it is one module: one that loads a library tslang built shares that - // library's process heap, since each frees what the other allocates. gc allocates from the - // collector either way. --process-heap keeps the process heap, for heap debugging tools. - useMimalloc = winX86OrX64 && !shared && !compileOptions.importsSharedLibrary - && !compileOptions.needsGCRuntime() && !processHeap; + // heap, and lends it to every library tslang built that it loads (ProcessHeapExports.inc): + // a library never owns the heap. gc allocates from the collector either way. + // --process-heap keeps the process heap, for heap debugging tools. + useMimalloc = winX86OrX64 && !shared && !compileOptions.needsGCRuntime() && !processHeap; if (useMimalloc && !tslangLibPath.empty()) { llvm::SmallString<256> mimallocLib(tslangLibPath); @@ -1082,6 +1081,16 @@ int linkBinary(const char *argv0, const LinkRequest &request, CompileOptions &co } args.push_back("-lTypeScriptAsyncRuntime"); + + // A library says it allocates from the program's heap (`__tslang_heap_shared`, with the + // entry points that find it), or a program on mimalloc refuses to load it - so every one + // carries the marker, even one that never allocates and so would not pull it in + // (TypeScript/ProcessHeap.h, librarySharesHeap). x86 decorates C names with an underscore. + if (shared && winX86OrX64) + { + args.push_back(arch == llvm::Triple::x86 ? "-Wl,-include:___tslang_heap_shared" + : "-Wl,-include:__tslang_heap_shared"); + } } if (!win && !wasm) diff --git a/tslang/tslang/jit.cpp b/tslang/tslang/jit.cpp index d2eb8c5f4..f4d345abb 100644 --- a/tslang/tslang/jit.cpp +++ b/tslang/tslang/jit.cpp @@ -47,6 +47,8 @@ #ifdef _WIN32 #include + +#include "TypeScript/ProcessHeap.h" #endif // From TypeScript/ObjDumper.h, which cannot be included here: its llvm/BinaryFormat/COFF.h @@ -407,8 +409,28 @@ static void jitEnableGCThreads() // them from TypeScriptAsyncRuntime; under the JIT they are defined here, for every memory model, // because TypeScriptRuntime is only loaded for `gc`. They go through this process's own // DynamicLibrary, so a lookup also sees the libraries passed with `--shared-libs`. +// Whether the program the JIT runs allocates from mimalloc rather than the process heap (see the heap +// entry points below, and jitChooseHeap). +static bool jitMimalloc = false; + +#ifdef _WIN32 +static std::wstring jitWidePath(llvm::StringRef path); +#endif + static int jitLoadLibraryPermanently(const char *fileName) { +#ifdef _WIN32 + // a library that keeps a heap of its own (a tslang before this one built it) would free the + // program's mimalloc blocks on it: refused, before any of its code runs + if (jitMimalloc && !typescript::process_heap::librarySharesHeap(jitWidePath(fileName).c_str())) + { + llvm::WithColor::error(llvm::errs(), "tslang") + << fileName << " does not allocate from the program's heap (a tslang before this one built it); " + << "rebuild it, or run with --process-heap\n"; + return 1; + } +#endif + return llvm::sys::DynamicLibrary::LoadLibraryPermanently(fileName); } @@ -441,43 +463,40 @@ static void jitAssertFailed(const char *message, const char *file, unsigned line } #ifdef _WIN32 -// The allocator JIT'd code gets. A block it allocates can be freed by another module and the other -// way round - an object of a library's class under -mm=rc or -mm=own is made in the library and -// destroyed by the program, a coroutine frame comes from TypeScriptRuntime.dll - so it is the one -// every module tslang builds uses: the process heap (TypeScript/ProcessHeap.h). The program calls -// it by the names ProcessHeapPass gives it; `malloc`, `calloc`, `realloc` and `free` are bound to -// it as well, for whatever still calls those. tslang.exe's own would not do: with the prebuilt -// LLVM its `malloc` is rpmalloc, and a debug CRT keeps its blocks per copy of the CRT. -#include "../lib/ProcessHeapExports.inc" - -// ...or mimalloc, as an executable tslang links allocates from (TypeScriptMimalloc), when the -// program is the only module allocating: no library tslang built is loaded - the default library's -// DLL, one given with --shared-libs, one the program imports dynamically - since such a library -// allocates from the process heap and frees what the program allocated. Decided once, before the -// program's first instruction runs (jitChooseHeap), and never changed after: a block goes back to -// the heap it came from. -static bool jitMimalloc = false; - -static void *jitHeapMalloc(size_t size) +// The heap JIT'd code gets. A block it allocates can be freed by another module and the other way +// round - an object of a library's class under -mm=rc or -mm=own is made in the library and +// destroyed by the program, a coroutine frame comes from TypeScriptRuntime.dll - so there is one +// heap in the process (TypeScript/ProcessHeap.h), and it is the program's: tslang.exe's here. +// Exported, so the libraries the program loads - the default library's DLL, --shared-libs, a +// dynamic import, TypeScriptRuntime.dll - find it and allocate from it too +// (lib/ProcessHeapExports.inc). The program calls it by the names ProcessHeapPass gives it; +// `malloc`, `calloc`, `realloc` and `free` are bound to it as well, for whatever still calls those. +// tslang.exe's own `malloc` would not do: with the prebuilt LLVM it is rpmalloc, and a debug CRT +// keeps its blocks per copy of the CRT. +// +// It is mimalloc, as an executable tslang links allocates from (TypeScriptMimalloc), or the process +// heap: decided once, before the first library is loaded or the program's first instruction runs +// (jitChooseHeap), and never changed after - a block goes back to the heap it came from. +extern "C" __declspec(dllexport) void *__tslang_heap_malloc(size_t size) { - return jitMimalloc ? mi_malloc(size ? size : 1) : __tslang_heap_malloc(size); + return jitMimalloc ? mi_malloc(size ? size : 1) : typescript::process_heap::allocate(size); } -static void *jitHeapCalloc(size_t count, size_t size) +extern "C" __declspec(dllexport) void *__tslang_heap_calloc(size_t count, size_t size) { if (!jitMimalloc) { - return __tslang_heap_calloc(count, size); + return typescript::process_heap::allocateZeroed(count, size); } return count == 0 || size == 0 ? mi_calloc(1, 1) : mi_calloc(count, size); } -static void *jitHeapRealloc(void *ptr, size_t size) +extern "C" __declspec(dllexport) void *__tslang_heap_realloc(void *ptr, size_t size) { if (!jitMimalloc) { - return __tslang_heap_realloc(ptr, size); + return typescript::process_heap::reallocate(ptr, size); } // as the process heap's: a block resized to nothing is freed, and there is none @@ -490,14 +509,15 @@ static void *jitHeapRealloc(void *ptr, size_t size) return mi_realloc(ptr, size); } -static void jitHeapFree(void *ptr) +extern "C" __declspec(dllexport) void __tslang_heap_free(void *ptr) { - jitMimalloc ? mi_free(ptr) : __tslang_heap_free(ptr); + jitMimalloc ? mi_free(ptr) : typescript::process_heap::release(ptr); } -static void *jitHeapAlignedAlloc(size_t alignment, size_t size) +extern "C" __declspec(dllexport) void *__tslang_heap_aligned_alloc(size_t alignment, size_t size) { - return jitMimalloc ? mi_malloc_aligned(size ? size : 1, alignment) : __tslang_heap_aligned_alloc(alignment, size); + return jitMimalloc ? mi_malloc_aligned(size ? size : 1, alignment) + : typescript::process_heap::allocateAligned(alignment, size); } struct JitHeapFunction @@ -507,33 +527,43 @@ struct JitHeapFunction }; static const JitHeapFunction jitHeapFunctions[] = { - {"malloc", (void *)&jitHeapMalloc}, - {"calloc", (void *)&jitHeapCalloc}, - {"realloc", (void *)&jitHeapRealloc}, - {"free", (void *)&jitHeapFree}, - {PROCESS_HEAP_MALLOC, (void *)&jitHeapMalloc}, - {PROCESS_HEAP_CALLOC, (void *)&jitHeapCalloc}, - {PROCESS_HEAP_REALLOC, (void *)&jitHeapRealloc}, - {PROCESS_HEAP_FREE, (void *)&jitHeapFree}, - {PROCESS_HEAP_ALIGNED_ALLOC, (void *)&jitHeapAlignedAlloc}, + {"malloc", (void *)&__tslang_heap_malloc}, + {"calloc", (void *)&__tslang_heap_calloc}, + {"realloc", (void *)&__tslang_heap_realloc}, + {"free", (void *)&__tslang_heap_free}, + {PROCESS_HEAP_MALLOC, (void *)&__tslang_heap_malloc}, + {PROCESS_HEAP_CALLOC, (void *)&__tslang_heap_calloc}, + {PROCESS_HEAP_REALLOC, (void *)&__tslang_heap_realloc}, + {PROCESS_HEAP_FREE, (void *)&__tslang_heap_free}, + {PROCESS_HEAP_ALIGNED_ALLOC, (void *)&__tslang_heap_aligned_alloc}, }; + +static std::wstring jitWidePath(llvm::StringRef path) +{ + llvm::SmallVector wide; + if (!llvm::convertUTF8ToUTF16String(path, wide)) + { + return std::wstring(); + } + + // null-terminated, and the terminator is not part of the path + return std::wstring(wide.begin(), std::find(wide.begin(), wide.end(), 0)); +} #endif -// Whether the program the JIT runs allocates from mimalloc (see jitMimalloc). `importsLibrary`: -// the program, or a module of it, imports a library dynamically. Called once the libraries the -// process loads (clSharedLibs) are known and before the program runs. -static void jitChooseHeap(const CompileOptions &compileOptions, bool nativeImage, bool importsLibrary) +// Whether the program the JIT runs allocates from mimalloc (see jitMimalloc). Called once the +// libraries the process loads (clSharedLibs) are known, before any of them is loaded. +static void jitChooseHeap(const CompileOptions &compileOptions) { #ifdef _WIN32 - // TypeScriptRuntime.dll and gc.dll hand the program no block of their heap to free - auto loadsTslangLibrary = llvm::any_of(clSharedLibs, [](const std::string &libPath) { - auto stem = llvm::sys::path::stem(libPath); - return !stem.contains_insensitive("TypeScriptRuntime") && !stem.equals_insensitive("gc"); + // Each library shares the heap unless a tslang before this one built it: then it keeps the + // process heap, and so does the program. gc.dll allocates nothing the program frees. + auto librariesShare = llvm::all_of(clSharedLibs, [](const std::string &libPath) { + return llvm::sys::path::stem(libPath).equals_insensitive("gc") + || typescript::process_heap::librarySharesHeap(jitWidePath(libPath).c_str()); }); - // a --jit-native image links the async runtime, process heap and all, into itself - jitMimalloc = !nativeImage && !compileOptions.needsGCRuntime() && !processHeap && !loadsTslangLibrary - && !importsLibrary; + jitMimalloc = !compileOptions.needsGCRuntime() && !processHeap && librariesShare; if (verbose && !compileOptions.needsGCRuntime()) { @@ -542,14 +572,6 @@ static void jitChooseHeap(const CompileOptions &compileOptions, bool nativeImage #endif } -// Whether an object the JIT cache made calls tslang_load_library_permanently, which a dynamic import -// lowers to. Its name in the object's string table is enough: a string literal spelling the same -// name only costs the program mimalloc. -static bool objectImportsLibrary(const llvm::MemoryBuffer &object) -{ - return object.getBuffer().contains("tslang_load_library_permanently"); -} - #ifndef _WIN32 // glibc's `__assert_fail`, which the lowering calls off Windows, ends in abort(). tslang's own // crash handler takes that SIGABRT for a compiler crash and prints a "Stack dump" after the @@ -1422,7 +1444,7 @@ int runJit(int argc, char **argv, mlir::ModuleOp module, CompileOptions &compile return result; } - jitChooseHeap(compileOptions, /*nativeImage=*/false, compileOptions.importsSharedLibrary); + jitChooseHeap(compileOptions); if (dumpObjectFile) { @@ -1567,6 +1589,9 @@ int runJitCached(int argc, char **argv, CompileOptions &compileOptions) return result; } + // before a library is loaded: its global constructors already allocate from the program's heap + jitChooseHeap(compileOptions); + auto needsGCEnableThreadsStandIn = false; if (auto result = loadJitSharedLibraries(needsGCEnableThreadsStandIn)) { @@ -1590,8 +1615,6 @@ int runJitCached(int argc, char **argv, CompileOptions &compileOptions) std::vector initFunctions; auto hasGCtorsMethod = false; - // a unit loaded from the cache has no module to ask, only its object - auto importsLibrary = false; for (auto &unit : units) { if (!unit.initFunction.empty()) @@ -1600,11 +1623,8 @@ int runJitCached(int argc, char **argv, CompileOptions &compileOptions) } hasGCtorsMethod |= unit.isMain && unit.hasGCtorsMethod; - importsLibrary |= unit.object && objectImportsLibrary(*unit.object); } - jitChooseHeap(compileOptions, /*nativeImage=*/jitNative, importsLibrary); - #ifdef _WIN32 if (jitNative) { From 46f07ae172b03772046ba844b3af25f0266d4717 Mon Sep 17 00:00:00 2001 From: ASDAlexander77 Date: Sun, 8 Nov 2026 11:31:16 +0000 Subject: [PATCH 2/2] A Linux executable and the JIT allocate from mimalloc On Linux, an executable that defines malloc provides it to every shared object in the process, so TypeScriptMimalloc is built with MI_MALLOC_OVERRIDE and linked whole into an executable tslang builds under rc, own and none. The default library's .so, a library tslang built, and libc all allocate from it; no marker is needed and older libraries need no rebuild. --process-heap keeps libc's malloc. tslang links the same override (except in sanitizer builds), so a program run with --emit=jit and every library it loads share tslang's mimalloc. The test runner's Linux link lines and the Linux release package include the library; docs/memory-models.md covers both systems and gives Linux timings (arrays 1.9x, heap-object ray tracer 1.1x, strings and @value code unchanged). Linux (WSL, prebuilt LLVM 22.1.8): ctest 6659/6659. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/create-release.yml | 1 + docs/memory-models.md | 62 ++++++++++++++------ tslang/lib/TypeScriptMimalloc/CMakeLists.txt | 56 ++++++++++++------ tslang/test/tester/test-runner.cpp | 28 ++++++--- tslang/tslang/CMakeLists.txt | 11 ++++ tslang/tslang/exe.cpp | 31 ++++++---- 6 files changed, 134 insertions(+), 55 deletions(-) diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml index 2a1d7af94..7109faf54 100644 --- a/.github/workflows/create-release.yml +++ b/.github/workflows/create-release.yml @@ -463,6 +463,7 @@ jobs: cp ./tslang/ninja/release/lib/libTypeScriptRuntime.so ./__stage/ cp ./gc/ninja/release/libgc.a ./__stage/ cp ./tslang/ninja/release/lib/libTypeScriptAsyncRuntime.a ./__stage/ + cp ./tslang/ninja/release/lib/libTypeScriptMimalloc.a ./__stage/ cp ../3rdParty/llvm/release/bin/wasm-ld ./__stage/ # Add the default library as a whole tree so its per-target, per-build, per-memory-model # subfolders are preserved: diff --git a/docs/memory-models.md b/docs/memory-models.md index da1f6b149..1ae0ec651 100644 --- a/docs/memory-models.md +++ b/docs/memory-models.md @@ -188,11 +188,15 @@ two throwaway heap allocations of the same size back at the same point and the p at the unpromoted speed (157 ms against 158). Expect the same kind of swing, in either direction, from any change to what a program allocates early. -## The heap on Windows +## The heap -Under `-mm=rc`, `-mm=own` and `-mm=none`, a Windows program allocates from -[mimalloc](https://github.com/microsoft/mimalloc) rather than the Windows process heap -(`HeapAlloc`): +Under `-mm=rc`, `-mm=own` and `-mm=none`, a program on Windows or Linux allocates from +[mimalloc](https://github.com/microsoft/mimalloc) rather than the system's heap. Linux works +differently from Windows; see [On Linux](#on-linux) below. + +### On Windows + +A Windows program allocates from mimalloc rather than the Windows process heap (`HeapAlloc`): - an executable `tslang --emit=exe` builds links `TypeScriptMimalloc.lib` (it ships beside `TypeScriptAsyncRuntime.lib`); @@ -212,22 +216,44 @@ A program you link yourself, against `TypeScriptAsyncRuntime.lib`, keeps the pro link `TypeScriptMimalloc.lib` ahead of it. `--process-heap` keeps the process heap anyway, for heap debugging tools such as page heap or -Application Verifier. `-mm=gc` allocates from the collector, and other platforms from the C -library's `malloc`; neither changes. +Application Verifier. -Measured, median of eleven runs, ahead-of-time build with `--opt --opt_level=3`, in milliseconds. -`--process-heap` → default: +### On Linux -| program | model | `--process-heap` → default | -| --- | --- | --- | -| ray tracer, every vector and colour an object on the heap | `-mm=own` | 529 → 180 (2.9×) | -| ray tracer, vectors and colours `@value` types | `-mm=own` | 78 → 74 | -| ray tracer, vectors and colours `@value` types | `-mm=rc` | 111 → 106 | -| 20,000 arrays of 200 new objects | `-mm=rc` / `-mm=own` | 131 → 48 (2.7×) | -| 5 million strings built by `+` | `-mm=own` | 910 → 696 (1.3×) | - -The gain is in programs that make and drop many small objects, arrays and strings. Where `@value` -types or stack promotion already keep a program's data off the heap, the heap barely matters. +An executable `tslang --emit=exe` builds links `libTypeScriptMimalloc.a`, which defines `malloc`, +`free` and the rest. On Linux, the `malloc` an executable defines is the one every shared object in +the process uses: the default library's `.so`, a library tslang built, and the C library itself. So +the whole process has one heap without any further step, and a library built by an older tslang +needs no rebuild. + +tslang itself is linked the same way, so a program run with `--emit=jit`, and every library it +loads, allocates from tslang's mimalloc. Under the JIT, `--process-heap` has no effect on Linux. +A tslang built with a sanitizer keeps the C library's `malloc`, because the sanitizer needs to own +it. + +A program you link yourself keeps the C library's `malloc` unless you link +`-Wl,--whole-archive -lTypeScriptMimalloc -Wl,--no-whole-archive`. `--process-heap` keeps the C +library's `malloc` in an executable tslang links. + +### Everywhere + +`-mm=gc` allocates from the collector, and other platforms (wasm, Android) from the C library's +`malloc`; neither changes. + +Measured, median of eleven runs, ahead-of-time build with `--opt --opt_level=3`, in milliseconds, +`--process-heap` → default. Windows 11 and Linux (Ubuntu under WSL 2, glibc) on the same machine: + +| program | model | Windows | Linux | +| --- | --- | --- | --- | +| ray tracer, every vector and colour an object on the heap | `-mm=own` | 529 → 180 (2.9×) | 131 → 118 (1.1×) | +| ray tracer, vectors and colours `@value` types | `-mm=own` | 78 → 74 | 60 → 60 | +| ray tracer, vectors and colours `@value` types | `-mm=rc` | 111 → 106 | 101 → 102 | +| 20,000 arrays of 200 new objects | `-mm=rc` / `-mm=own` | 131 → 48 (2.7×) | 68 → 35 (1.9×) | +| 5 million strings built by `+` | `-mm=own` | 910 → 696 (1.3×) | 431 → 440 | + +The gain is in programs that make and drop many small objects, arrays and strings, and it is larger +on Windows, whose process heap is slower than glibc's `malloc`. Where `@value` types or stack +promotion already keep a program's data off the heap, the heap barely matters. ## Other limits of `-mm=rc` diff --git a/tslang/lib/TypeScriptMimalloc/CMakeLists.txt b/tslang/lib/TypeScriptMimalloc/CMakeLists.txt index 702d74337..cfba7883b 100644 --- a/tslang/lib/TypeScriptMimalloc/CMakeLists.txt +++ b/tslang/lib/TypeScriptMimalloc/CMakeLists.txt @@ -1,28 +1,46 @@ -# The heap a Windows executable built by tslang allocates from: mimalloc (lib/mimalloc, one -# translation unit) behind the process-heap entry points (MimallocHeap.c). exe.cpp links it ahead -# of TypeScriptAsyncRuntime. tslang.exe links the `TypeScriptMimallocCore` half alone, for the JIT. +# The heap a program built by tslang allocates from: mimalloc (lib/mimalloc, one translation unit). +# exe.cpp links it into an executable; tslang links it into itself, for the program the JIT runs. # The same sources are built for x86 by tslang/runtime/CMakeLists.txt. -if (NOT WIN32) - return() -endif() +# +# Windows: behind the process-heap entry points (MimallocHeap.c), ahead of TypeScriptAsyncRuntime. +# tslang.exe links the `TypeScriptMimallocCore` half alone and serves the JIT from it (jit.cpp). +# +# Linux: as the process's `malloc` (MI_MALLOC_OVERRIDE). An executable that defines malloc, free and +# the rest is where every shared object of the process finds them - the default library's, one +# tslang built, libc's own - so one copy in the executable is the heap of the whole process. +if (WIN32) + set(MIMALLOC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../mimalloc") -set(MIMALLOC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../mimalloc") + # mimalloc itself, its symbols prefixed mi_ - nothing else in the process is replaced + add_library(TypeScriptMimallocCore OBJECT "${MIMALLOC_DIR}/src/static.c") + target_include_directories(TypeScriptMimallocCore PUBLIC "${MIMALLOC_DIR}/include") + # not hardened: the guard pages and encoded free lists of MI_SECURE cost what this is for + target_compile_definitions(TypeScriptMimallocCore PRIVATE MI_SECURE=0) + if (MSVC) + # upstream's own code; its warnings are not ours to fix + target_compile_options(TypeScriptMimallocCore PRIVATE /W0) + endif() -# mimalloc itself, its symbols prefixed mi_ - nothing else in the process is replaced -add_library(TypeScriptMimallocCore OBJECT "${MIMALLOC_DIR}/src/static.c") -target_include_directories(TypeScriptMimallocCore PUBLIC "${MIMALLOC_DIR}/include") -# not hardened: the guard pages and encoded free lists of MI_SECURE cost what this is for -target_compile_definitions(TypeScriptMimallocCore PRIVATE MI_SECURE=0) -if (MSVC) + add_library(TypeScriptMimalloc STATIC + MimallocHeap.c + $) + target_include_directories(TypeScriptMimalloc PRIVATE "${MIMALLOC_DIR}/include") +elseif (CMAKE_SYSTEM_NAME STREQUAL "Linux") + set(MIMALLOC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../mimalloc") + + add_library(TypeScriptMimalloc STATIC "${MIMALLOC_DIR}/src/static.c") + target_include_directories(TypeScriptMimalloc PUBLIC "${MIMALLOC_DIR}/include") + # malloc, free, calloc, realloc, posix_memalign, aligned_alloc, malloc_usable_size... defined by it + target_compile_definitions(TypeScriptMimalloc PRIVATE MI_SECURE=0 MI_MALLOC_OVERRIDE) + # an executable tslang links may be position independent (-pie) + set_target_properties(TypeScriptMimalloc PROPERTIES POSITION_INDEPENDENT_CODE ON) # upstream's own code; its warnings are not ours to fix - target_compile_options(TypeScriptMimallocCore PRIVATE /W0) + target_compile_options(TypeScriptMimalloc PRIVATE -w) +else() + return() endif() -add_library(TypeScriptMimalloc STATIC - MimallocHeap.c - $) -target_include_directories(TypeScriptMimalloc PRIVATE "${MIMALLOC_DIR}/include") -# beside TypeScriptAsyncRuntime.lib, where --tslang-lib-path / TSLANG_LIB_PATH point (the generator +# beside TypeScriptAsyncRuntime, where --tslang-lib-path / TSLANG_LIB_PATH point (the generator # expression keeps a multi-config generator from adding a per-config subfolder) set_target_properties(TypeScriptMimalloc PROPERTIES ARCHIVE_OUTPUT_DIRECTORY "$") diff --git a/tslang/test/tester/test-runner.cpp b/tslang/test/tester/test-runner.cpp index 7b93ce6b4..0a535511e 100644 --- a/tslang/test/tester/test-runner.cpp +++ b/tslang/test/tester/test-runner.cpp @@ -160,15 +160,27 @@ std::string optVariantSuffix() return std::string(fastMath ? "fm" : "") + memoryModel + (x86 ? "x86" : "") + (noStackPromotion ? "nosp" : ""); } -#ifdef WIN32 -// What tslang links into an executable (exe.cpp): mimalloc, ahead of the async runtime, under every -// memory model but gc. A library of a -shared test allocates from it too, through the entry points -// the program exports (lib/ProcessHeapExports.inc). +// What tslang links into an executable (exe.cpp): mimalloc, under every memory model but gc. A +// library of a -shared test allocates from it too: on Windows through the entry points the program +// exports (lib/ProcessHeapExports.inc), on Linux because the program's malloc is the process's. std::string mimallocLib() { - return memoryModel.empty() || memoryModel == "gc" ? std::string() : std::string("TypeScriptMimalloc.lib "); + if (memoryModel.empty() || memoryModel == "gc") + { + return std::string(); + } + +#ifdef WIN32 + // ahead of the async runtime, whose process-heap definitions it replaces + return std::string("TypeScriptMimalloc.lib "); +#else + // whole: the archive defines malloc and the rest, which the linker would otherwise take from libc + return std::string("-Wl,--whole-archive -lTypeScriptMimalloc -Wl,--no-whole-archive "); +#endif } +#ifdef WIN32 + // ...and what tslang links into a library: the marker saying it allocates from the program's heap, // which a program on mimalloc checks before loading it - kept even in a library that never // allocates. x86 decorates C names with an underscore. @@ -268,7 +280,7 @@ void createCompileBatchFile() batFile << "GC_LIB_PATH=" << TEST_GCPATH << std::endl; batFile << "$TSLANGEXEPATH/tslang --emit=obj --entry-point " << tslang_opt << " " << tslang_opt_ext << " $FILEPATH -relocation-model=pic -o=$FILENAME.o" << std::endl; batFile << TEST_COMPILER << " -o $FILENAME $LINKER_OPTS -L$GC_LIB_PATH -L$TSLANG_LIB_PATH $FILENAME.o " - << TYPESCRIPT_LIB << GC_LIB << LIBS << std::endl; + << mimallocLib() << TYPESCRIPT_LIB << GC_LIB << LIBS << std::endl; batFile << "./$FILENAME 1> $FILENAME.txt 2> $FILENAME.err" << std::endl; batFile << "echo $? > $FILENAME.code" << std::endl; batFile << "rm -f $FILENAME.o" << std::endl; @@ -518,7 +530,7 @@ void createMultiCompileBatchFile(std::string tempOutputFileNameNoExt, std::vecto batFile << TEST_COMPILER << " -o $FILENAME " << objs.str() << "-L$GC_LIB_PATH -L$TSLANG_LIB_PATH " - << TYPESCRIPT_LIB << GC_LIB << LIBS << std::endl; + << mimallocLib() << TYPESCRIPT_LIB << GC_LIB << LIBS << std::endl; batFile << "./$FILENAME 1> $FILENAME.txt 2> $FILENAME.err" << std::endl; batFile << "echo $? > $FILENAME.code" << std::endl; @@ -722,7 +734,7 @@ void createSharedMultiBatchFile(std::string tempOutputFileNameNoExt, std::vector batFile << "-L`pwd` -Wl,-rpath=`pwd` -l" << shared_filenameNoExt << " "; } - batFile << TYPESCRIPT_LIB << GC_LIB << LIBS << std::endl; + batFile << mimallocLib() << TYPESCRIPT_LIB << GC_LIB << LIBS << std::endl; batFile << "rm -f " << exec_objs.str() << std::endl; diff --git a/tslang/tslang/CMakeLists.txt b/tslang/tslang/CMakeLists.txt index cec389abf..cd35f0fea 100644 --- a/tslang/tslang/CMakeLists.txt +++ b/tslang/tslang/CMakeLists.txt @@ -79,6 +79,17 @@ if (WIN32) target_link_libraries(tslang PRIVATE TypeScriptMimallocCore) # ...and the library an executable it links needs, built with it add_dependencies(tslang TypeScriptMimalloc) +elseif (CMAKE_SYSTEM_NAME STREQUAL "Linux") + # mimalloc as tslang's own malloc: the program the JIT runs and every shared object it loads + # (the default library's, one tslang built) resolve malloc to it, as an executable tslang links + # gives them its own (TypeScriptMimalloc overrides malloc). Not under a sanitizer, which has to + # own malloc itself. + if ("${CMAKE_CXX_FLAGS} ${CMAKE_EXE_LINKER_FLAGS}" MATCHES "-fsanitize") + add_dependencies(tslang TypeScriptMimalloc) + else() + # whole: the archive defines malloc, which the linker would otherwise take from libc first + target_link_libraries(tslang PRIVATE -Wl,--whole-archive TypeScriptMimalloc -Wl,--no-whole-archive) + endif() endif() target_compile_definitions(tslang PUBLIC TSLANG_PACKAGE_VERSION="${TSLANG_PACKAGE_VERSION}") diff --git a/tslang/tslang/exe.cpp b/tslang/tslang/exe.cpp index f30342356..3cbc28f50 100644 --- a/tslang/tslang/exe.cpp +++ b/tslang/tslang/exe.cpp @@ -981,21 +981,24 @@ int linkBinary(const char *argv0, const LinkRequest &request, CompileOptions &co args.push_back(tslangLibPathOpt.c_str()); } - // An executable allocates from mimalloc (TypeScriptMimalloc.lib) rather than the process - // heap, and lends it to every library tslang built that it loads (ProcessHeapExports.inc): - // a library never owns the heap. gc allocates from the collector either way. - // --process-heap keeps the process heap, for heap debugging tools. - useMimalloc = winX86OrX64 && !shared && !compileOptions.needsGCRuntime() && !processHeap; + // An executable allocates from mimalloc (TypeScriptMimalloc) rather than the system's heap, + // and every library tslang built that it loads allocates from it too: on Windows through the + // entry points the executable exports (ProcessHeapExports.inc), on Linux because the + // executable's malloc is the process's. A library never owns the heap. gc allocates from the + // collector either way. --process-heap keeps the system's heap, for heap debugging tools. + auto linuxTarget = os == llvm::Triple::Linux && !android; + useMimalloc = (winX86OrX64 || linuxTarget) && !shared && !compileOptions.needsGCRuntime() && !processHeap; if (useMimalloc && !tslangLibPath.empty()) { + auto mimallocName = getTargetStaticLibName("TypeScriptMimalloc"); llvm::SmallString<256> mimallocLib(tslangLibPath); - llvm::sys::path::append(mimallocLib, "TypeScriptMimalloc.lib"); + llvm::sys::path::append(mimallocLib, mimallocName); if (!llvm::sys::fs::exists(mimallocLib)) { llvm::WithColor::error(llvm::errs(), "tslang") - << "TypeScriptMimalloc.lib, the heap an executable allocates from, is not in " << tslangLibPath - << " next to TypeScriptAsyncRuntime.lib. Use the runtime libraries of the same tslang build, " - << "or link with --process-heap to allocate from the Windows process heap instead.\n"; + << mimallocName << ", the heap an executable allocates from, is not in " << tslangLibPath + << " next to the async runtime library. Use the runtime libraries of the same tslang build, " + << "or link with --process-heap to allocate from the system's heap instead.\n"; return 1; } } @@ -1074,11 +1077,19 @@ int linkBinary(const char *argv0, const LinkRequest &request, CompileOptions &co if (isTslangLibNeeded) { - if (useMimalloc) + if (useMimalloc && win) { // ahead of TypeScriptAsyncRuntime: its process-heap entry points are the ones taken args.push_back("-lTypeScriptMimalloc"); } + else if (useMimalloc) + { + // whole: the archive defines malloc, free and the rest, which the linker would otherwise + // take from libc + args.push_back("-Wl,--whole-archive"); + args.push_back("-lTypeScriptMimalloc"); + args.push_back("-Wl,--no-whole-archive"); + } args.push_back("-lTypeScriptAsyncRuntime");