Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -791,6 +791,27 @@ calling methods nothing overrides directly, and testing a union for null without
still pays for retains and releases in the inner loops. The benchmark's README has every step and
the earlier results.

Loops over arrays are measured by [tslang/benchmarks/numeric](tslang/benchmarks/numeric/README.md):
six kernels over flat arrays of 4096 elements, with Rust and C++ ports (release build, Ubuntu 24.04
x64 on a 4-core cloud VM, median of 9 runs, ms, measured 2026-11-10).

| build | saxpy | triad | stencil | dot | matmul | hash |
| --- | --- | --- | --- | --- | --- | --- |
| `-mm=gc` | 85.5 | 93.3 | 97.5 | 93.2 | 23.2 | 44.6 |
| `-mm=rc` | 97.9 | 95.0 | 98.8 | 98.1 | 28.7 | 44.7 |
| `-mm=none` | 88.7 | 93.8 | 97.3 | 93.6 | 23.2 | 43.8 |
| `-mm=own` | 87.1 | 94.7 | 97.3 | 96.5 | 26.7 | 48.0 |
| Rust (`Vec<f64>`, iterators) | 68.2 | 76.0 | 103.4 | 94.9 | 20.6 | 44.4 |
| C++ (`std::vector`, clang -O3) | 69.0 | 75.2 | 107.4 | 93.5 | 30.1 | 42.8 |

Here the memory model does not matter: the loops allocate nothing, and all four models vectorize
them. tslang is level with Rust and C++ on four of the kernels and takes about 1.25x as long on
saxpy and triad. Under `--emit=jit` the loops are vectorized for the host's CPU (AVX2 on that
machine) rather than the baseline x86-64 an executable targets, which made saxpy, triad, matmul
and hash 15-25% faster than the JIT of the release before it. The same ray tracer on that Linux
machine takes 128 ms under `-mm=gc`, 123 under `-mm=none`, 130 under `-mm=own` and 238 under
`-mm=rc`, against 103 ms for the C++ port; the benchmarks' READMEs have the full tables.

### `-mm=own`

`-mm=own` works the way Rust's borrow checker does. Every heap block has exactly one owner: a
Expand Down
3 changes: 3 additions & 0 deletions tslang/benchmarks/numeric/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
out/
rust/target/
rust/Cargo.lock
102 changes: 102 additions & 0 deletions tslang/benchmarks/numeric/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Numeric loops over arrays: memory models vs Rust and C++

`nbench.ts` is six tight loops over flat arrays, the code a vectorizer is for. The ray tracer in
`../raytrace` is scalar code with calls through interfaces, so it cannot show whether loops are
vectorized; this one can. `rust/` and `cpp/` hold ports of the same kernels, written the way each
language's programmer would (`Vec<f64>` with slices and iterators, `std::vector<double>`).

| Kernel | Loop | Notes |
|---|---|---|
| saxpy | `y[i] += a * x[i]` | element-wise, two arrays |
| triad | `z[i] = x[i] * y[i] + c` | element-wise, three arrays |
| stencil | `dst[i] = (src[i-1] + 2 src[i] + src[i+1]) / 4` | neighbours of one array |
| dot | `s += x[i] * y[i]` | a floating-point reduction: strict IEEE order keeps it scalar in every language without fast-math |
| matmul | `c[i][j] += a[i][k] * b[k][j]` | 64x64, flat arrays, i-k-j order |
| hash | `h[i] = (h[i] * 31 + g[i]) & 0xffff`, summed | `int[]`, with an integer reduction |

Arrays hold 4096 elements (32 KB of doubles), so the loops measure the generated code and not the
memory bus. Every kernel ends in a check value, and every build prints the same six. No kernel needs
floating-point operations reordered; the C++ port is built with `-ffp-contract=off` so that clang
does not fuse `a * b + c`, which changes the low digits.

Run (needs a Release tslang, and cargo and clang++ for the ports):

```
TSLANG=<path to tslang> bash tslang/benchmarks/numeric/bench.sh
```

- tslang: `--emit=exe --opt --opt_level=3 --no-default-lib --mm=<model>` (`--no-default-lib`
because the file declares its own `floor`/`fmod`)
- `JIT_LIB=<libTypeScriptRuntime.so>` adds a row per model under `--emit=jit`, the JIT cache warm
- Rust: `opt-level=3`, `lto="fat"`, `codegen-units=1`, `panic="abort"`
- C++: `clang++ -std=c++20 -O3 -ffp-contract=off`

With a kernel's name as its argument a program runs only that kernel, which is how `bench.sh` times
them one by one. A JIT figure includes the compiler's start-up, about 13 ms here.

## Results 2026-11-10

Release, Ubuntu 24.04 x64, a 4-core cloud VM (Xeon 2.1 GHz, AVX2); main d78c735 with the collector's
allocations declared `noalias` (below). Median of 9 runs after a warm-up, ms; every build prints
the same check values. The machine is shared, so a few percent either way is noise.

| Build | saxpy | triad | stencil | dot | matmul | hash |
|---|---|---|---|---|---|---|
| tslang gc | 85.5 | 93.3 | 97.5 | 93.2 | 23.2 | 44.6 |
| tslang rc | 97.9 | 95.0 | 98.8 | 98.1 | 28.7 | 44.7 |
| tslang none | 88.7 | 93.8 | 97.3 | 93.6 | 23.2 | 43.8 |
| tslang own | 87.1 | 94.7 | 97.3 | 96.5 | 26.7 | 48.0 |
| tslang gc (jit) | 101.2 | 93.4 | 118.6 | 106.9 | 32.6 | 57.2 |
| tslang rc (jit) | 91.4 | 86.9 | 96.4 | 108.2 | 33.3 | 42.0 |
| tslang none (jit) | 84.0 | 88.7 | 104.0 | 104.3 | 22.4 | 46.1 |
| tslang own (jit) | 90.9 | 91.0 | 108.0 | 103.3 | 25.6 | 48.5 |
| Rust | 68.2 | 76.0 | 103.4 | 94.9 | 20.6 | 44.4 |
| C++ (clang) | 69.0 | 75.2 | 107.4 | 93.5 | 30.1 | 42.8 |

- The four memory models are level with each other, and level with Rust and C++ on stencil, dot,
matmul and hash. On saxpy and triad tslang takes about 1.25x as long as the two ports. All of
them vectorize these loops for SSE2: an executable targets the baseline x86-64, as the Rust and
C++ builds do.
- dot is level across all ten rows: a strict floating-point reduction runs one addition at a time
everywhere. `--fast-math` is what would change it.
- The JIT rows generate the same loops for the host's CPU (below) and carry the compiler's
start-up. The gc and none objects the JIT caches hold the same number of vector instructions;
what makes the gc row slower than the none row here was not looked into.

### gc before its allocations were `noalias`

An array is a header holding the pointer to its elements. Under rc, none and own the header and
the elements come from `malloc` and `realloc`, whose results LLVM knows by name to be blocks no
other pointer reaches. Under gc they come from `GC_malloc` and `GC_realloc`, which were declared
without that, so a store to an element might have written the header: every loop loaded the
elements' pointer again on each iteration, and none of them was vectorized. GCPass now declares
the results of the collector's allocation functions `noalias`.

| Build | saxpy | triad | stencil | dot | matmul | hash |
|---|---|---|---|---|---|---|
| tslang gc, d78c735 | 190.5 | 225.3 | 206.9 | 94.1 | 91.2 | 220.4 |
| tslang gc, with `noalias` | 85.5 | 93.3 | 97.5 | 93.2 | 23.2 | 44.6 |

Before the change the gc executable had no packed floating-point instruction; after it, its
optimized LLVM IR has as many vector operations as the none build's. The ray tracer in
`../raytrace` is unchanged by it (gc 123 ms before, 128 ms after, within that run's noise).

### The JIT before and after the optimizer was given the host's CPU

The JIT generates code for the host CPU. Before d78c735 the optimizer's cost model was for a
generic x86-64, so it vectorized for SSE2 and the code generator had nothing wider to emit. The
same benchmark under the release before that change (v0.0-pre-alpha122), same machine and run:

| Build (jit) | saxpy | triad | stencil | dot | matmul | hash |
|---|---|---|---|---|---|---|
| none, alpha122 | 111.5 | 106.8 | 106.6 | 106.4 | 29.9 | 58.6 |
| none, d78c735 | 82.4 | 85.3 | 111.7 | 105.6 | 23.0 | 45.2 |
| rc, alpha122 | 103.9 | 104.0 | 107.5 | 105.1 | 38.4 | 52.6 |
| rc, d78c735 | 86.7 | 84.0 | 97.0 | 104.5 | 32.8 | 50.4 |
| own, alpha122 | 98.4 | 105.0 | 112.1 | 102.3 | 31.7 | 54.3 |
| own, d78c735 | 86.2 | 92.8 | 103.0 | 102.9 | 22.4 | 44.2 |

In the object the JIT caches for `-mm=none`, alpha122 has no instruction on a 256-bit register and
d78c735 has 720. saxpy, triad, matmul and hash are 15-25% faster; stencil and dot do not move
(dot is not vectorized, and stencil's figures are within the noise of each other). Executables are
unchanged by it: they target the baseline CPU unless `-mcpu` says otherwise.
65 changes: 65 additions & 0 deletions tslang/benchmarks/numeric/bench.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Builds nbench.ts under every memory model and the Rust and C++ ports, checks that they all print
# the same six check values, then times each kernel by itself: a warm-up run and RUNS timed runs,
# reporting the median wall time in ms.
#
# TSLANG=<path to tslang> [RUNS=9] [JIT_LIB=<libTypeScriptRuntime.so>] bash bench.sh
#
# tslang finds its libraries through GC_LIB_PATH and TSLANG_LIB_PATH (or beside itself, as a
# release archive has them). With JIT_LIB the program is also timed under --emit=jit, the JIT cache
# warm, which adds the compiler's start-up to every figure.
set -u
here=$(cd "$(dirname "$0")" && pwd)
tslang=${TSLANG:-$here/../../../__build/tslang/ninja/release/bin/tslang}
runs=${RUNS:-9}
out=$here/out
mkdir -p "$out"
kernels="saxpy triad stencil dot matmul hash"
expected="saxpy: 346503 triad: 527762 stencil: 218331 dot: 369289 matmul: 615082 hash: 102784"

names=()
cmds=()
add() { names+=("$1"); cmds+=("$2"); }

flags="--opt --opt_level=3 --no-default-lib"
for mm in gc rc none own; do
if "$tslang" --emit=exe $flags --mm=$mm "$here/nbench.ts" -o "$out/nb_$mm" >"$out/nb_$mm.log" 2>&1; then
add "tslang $mm" "$out/nb_$mm"
else
echo "tslang $mm does not compile, see $out/nb_$mm.log"
fi
if [ -n "${JIT_LIB:-}" ]; then
add "tslang $mm (jit)" "$tslang --emit=jit $flags --mm=$mm --shared-libs=$JIT_LIB --jit-cache-dir=$out/jit_$mm $here/nbench.ts"
fi
done

if command -v cargo >/dev/null; then
(cd "$here/rust" && cargo build --release --quiet) && add "rust" "$here/rust/target/release/nbench"
fi
if command -v clang++ >/dev/null; then
clang++ -std=c++20 -O3 -ffp-contract=off -o "$out/cpp_clang" "$here/cpp/nbench.cpp" && add "c++ (clang)" "$out/cpp_clang"
fi

now_ms() { echo $(($(date +%s%N) / 1000)); }

printf '%-20s' build
for k in $kernels; do printf '%9s' "$k"; done
printf ' %s\n' checksums

for i in "${!names[@]}"; do
cmd=${cmds[$i]}
# the whole program once: the check values, and under the JIT the object for the cache
got=$($cmd | grep -v -e '^start' -e '^done' | tr '\n' ' ' | sed 's/ *$//')
printf '%-20s' "${names[$i]}"
for k in $kernels; do
$cmd "$k" >/dev/null
times=()
for _ in $(seq "$runs"); do
t0=$(now_ms); $cmd "$k" >/dev/null; t1=$(now_ms)
times+=($((t1 - t0)))
done
median=$(printf '%s\n' "${times[@]}" | sort -n | sed -n "$(((runs + 1) / 2))p")
printf '%9s' "$(awk "BEGIN { printf \"%.1f\", $median / 1000 }")"
done
if [ "$got" = "$expected" ]; then echo " ok"; else echo " MISMATCH: $got"; fi
done
135 changes: 135 additions & 0 deletions tslang/benchmarks/numeric/cpp/nbench.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
// C++ port of tslang/benchmarks/numeric/nbench.ts: the same six kernels over std::vector<double> /
// std::vector<int>, printing the same six check values. Build with -ffp-contract=off: clang
// otherwise fuses a*b+c into one instruction where the CPU has it, which changes the low digits.
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <string>
#include <vector>

static const int N = 4096;
static const int M = 64;

static double check(double v) { return std::fmod(std::floor(v * 1000.0), 999983.0); }

static std::vector<double> makeArray(int n, double mul, int mod, double add) {
std::vector<double> a(n);
for (int i = 0; i < n; i++) a[i] = (i % mod) * mul + add;
return a;
}

static double sum(const std::vector<double> &a) {
double s = 0;
for (double v : a) s += v;
return s;
}

__attribute__((noinline)) static void saxpy(double a, const std::vector<double> &x, std::vector<double> &y, int n) {
for (int i = 0; i < n; i++) y[i] += a * x[i];
}

__attribute__((noinline)) static void triad(const std::vector<double> &x, const std::vector<double> &y,
std::vector<double> &z, double c, int n) {
for (int i = 0; i < n; i++) z[i] = x[i] * y[i] + c;
}

__attribute__((noinline)) static void stencil(const std::vector<double> &src, std::vector<double> &dst, int n) {
for (int i = 1; i < n - 1; i++) dst[i] = (src[i - 1] + 2.0 * src[i] + src[i + 1]) * 0.25;
}

__attribute__((noinline)) static double dot(const std::vector<double> &x, const std::vector<double> &y, int n) {
double s = 0;
for (int i = 0; i < n; i++) s += x[i] * y[i];
return s;
}

__attribute__((noinline)) static void matmul(const std::vector<double> &a, const std::vector<double> &b,
std::vector<double> &c, int m) {
for (int i = 0; i < m; i++)
for (int k = 0; k < m; k++) {
const double aik = a[i * m + k];
for (int j = 0; j < m; j++) c[i * m + j] += aik * b[k * m + j];
}
}

__attribute__((noinline)) static int hash(std::vector<int> &h, const std::vector<int> &g, int n) {
uint32_t s = 0;
for (int i = 0; i < n; i++) {
h[i] = (int)(((uint32_t)h[i] * 31u + (uint32_t)g[i]) & 0xffffu);
s += (uint32_t)h[i];
}
return (int)s;
}

static double runSaxpy() {
auto x = makeArray(N, 0.5, 97, 1.0);
auto y = makeArray(N, 0.25, 89, 2.0);
for (int r = 0; r < 60000; r++) saxpy(0.000001, x, y, N);
return check(sum(y));
}

static double runTriad() {
auto x = makeArray(N, 0.5, 97, 1.0);
auto y = makeArray(N, 0.25, 89, 2.0);
auto z = makeArray(N, 0.0, 1, 0.0);
double s = 0;
for (int r = 0; r < 60000; r++) {
triad(x, y, z, r * 0.001, N);
s += z[r % N];
}
return check(s + sum(z));
}

static double runStencil() {
auto a = makeArray(N, 0.5, 97, 1.0);
auto b = makeArray(N, 0.5, 97, 1.0);
for (int r = 0; r < 30000; r++) {
stencil(a, b, N);
stencil(b, a, N);
}
return check(sum(a));
}

static double runDot() {
auto x = makeArray(N, 0.5, 97, 1.0);
auto y = makeArray(N, 0.25, 89, 2.0);
double s = 0;
for (int r = 0; r < 30000; r++) {
x[r % N] += 0.5;
s += dot(x, y, N) * 0.000001;
}
return check(s);
}

static double runMatmul() {
auto a = makeArray(M * M, 0.001, 13, 0.01);
auto b = makeArray(M * M, 0.002, 7, 0.02);
auto c = makeArray(M * M, 0.0, 1, 0.0);
for (int r = 0; r < 400; r++) matmul(a, b, c, M);
return check(sum(c));
}

static int runHash() {
std::vector<int> h(N), g(N);
for (int i = 0; i < N; i++) {
h[i] = i % 251;
g[i] = i % 241 + 7;
}
uint32_t s = 0;
for (int r = 0; r < 60000; r++) s = (s + (uint32_t)hash(h, g, N)) & 0xfffffu;
return (int)s;
}

// With no argument every kernel runs; with a kernel's name only that one does.
int main(int argc, char **argv) {
const std::string only = argc > 1 ? argv[1] : "";
auto run = [&](const char *name) { return only.empty() || only == name; };
std::printf("start...\n");
if (run("saxpy")) std::printf("saxpy: %.0f\n", runSaxpy());
if (run("triad")) std::printf("triad: %.0f\n", runTriad());
if (run("stencil")) std::printf("stencil: %.0f\n", runStencil());
if (run("dot")) std::printf("dot: %.0f\n", runDot());
if (run("matmul")) std::printf("matmul: %.0f\n", runMatmul());
if (run("hash")) std::printf("hash: %d\n", runHash());
std::printf("done.\n");
}
Loading
Loading