From 79eb22c145f3aa3bb15581a8cbccf175bb21eceb Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 12:51:59 +0000 Subject: [PATCH 1/3] A benchmark of numeric loops over arrays, with Rust and C++ ports Six kernels over flat arrays (saxpy, triad, stencil, dot, matmul and an integer hash), each ending in a check value every build prints. bench.sh builds nbench.ts under every memory model, optionally under the JIT, and the two ports, and times each kernel by itself. The README has the first results, the JIT before and after the optimizer was given the host's CPU, and the finding that the gc build's loops are not vectorized. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J8KRfVk4FT2FALW47Ux2Fp --- tslang/benchmarks/numeric/.gitignore | 3 + tslang/benchmarks/numeric/README.md | 84 ++++++++++ tslang/benchmarks/numeric/bench.sh | 65 ++++++++ tslang/benchmarks/numeric/cpp/nbench.cpp | 135 +++++++++++++++ tslang/benchmarks/numeric/nbench.ts | 182 +++++++++++++++++++++ tslang/benchmarks/numeric/rust/Cargo.toml | 10 ++ tslang/benchmarks/numeric/rust/src/main.rs | 134 +++++++++++++++ 7 files changed, 613 insertions(+) create mode 100644 tslang/benchmarks/numeric/.gitignore create mode 100644 tslang/benchmarks/numeric/README.md create mode 100755 tslang/benchmarks/numeric/bench.sh create mode 100644 tslang/benchmarks/numeric/cpp/nbench.cpp create mode 100644 tslang/benchmarks/numeric/nbench.ts create mode 100644 tslang/benchmarks/numeric/rust/Cargo.toml create mode 100644 tslang/benchmarks/numeric/rust/src/main.rs diff --git a/tslang/benchmarks/numeric/.gitignore b/tslang/benchmarks/numeric/.gitignore new file mode 100644 index 000000000..89d60790b --- /dev/null +++ b/tslang/benchmarks/numeric/.gitignore @@ -0,0 +1,3 @@ +out/ +rust/target/ +rust/Cargo.lock diff --git a/tslang/benchmarks/numeric/README.md b/tslang/benchmarks/numeric/README.md new file mode 100644 index 000000000..907ea4d5e --- /dev/null +++ b/tslang/benchmarks/numeric/README.md @@ -0,0 +1,84 @@ +# Numeric loops over arrays: memory models vs Rust and C++ + +`nbench.ts` is six tight loops over flat arrays, the code a vectorizer is for. The ray tracer in +`../raytrace` is scalar code with calls through interfaces, so it cannot show whether loops are +vectorized; this one can. `rust/` and `cpp/` hold ports of the same kernels, written the way each +language's programmer would (`Vec` with slices and iterators, `std::vector`). + +| Kernel | Loop | Notes | +|---|---|---| +| saxpy | `y[i] += a * x[i]` | element-wise, two arrays | +| triad | `z[i] = x[i] * y[i] + c` | element-wise, three arrays | +| stencil | `dst[i] = (src[i-1] + 2 src[i] + src[i+1]) / 4` | neighbours of one array | +| dot | `s += x[i] * y[i]` | a floating-point reduction: strict IEEE order keeps it scalar in every language without fast-math | +| matmul | `c[i][j] += a[i][k] * b[k][j]` | 64x64, flat arrays, i-k-j order | +| hash | `h[i] = (h[i] * 31 + g[i]) & 0xffff`, summed | `int[]`, with an integer reduction | + +Arrays hold 4096 elements (32 KB of doubles), so the loops measure the generated code and not the +memory bus. Every kernel ends in a check value, and every build prints the same six. No kernel needs +floating-point operations reordered; the C++ port is built with `-ffp-contract=off` so that clang +does not fuse `a * b + c`, which changes the low digits. + +Run (needs a Release tslang, and cargo and clang++ for the ports): + +``` +TSLANG= bash tslang/benchmarks/numeric/bench.sh +``` + +- tslang: `--emit=exe --opt --opt_level=3 --no-default-lib --mm=` (`--no-default-lib` + because the file declares its own `floor`/`fmod`) +- `JIT_LIB=` adds a row per model under `--emit=jit`, the JIT cache warm +- Rust: `opt-level=3`, `lto="fat"`, `codegen-units=1`, `panic="abort"` +- C++: `clang++ -std=c++20 -O3 -ffp-contract=off` + +With a kernel's name as its argument a program runs only that kernel, which is how `bench.sh` times +them one by one. A JIT figure includes the compiler's start-up, about 13 ms here. + +## Results 2026-11-10 + +Release, Ubuntu 24.04 x64, a 4-core cloud VM (Xeon 2.1 GHz, AVX2); main d78c735. Median of 9 runs +after a warm-up, ms; every build prints the same check values. The machine is shared, so a few +percent either way is noise. + +| Build | saxpy | triad | stencil | dot | matmul | hash | +|---|---|---|---|---|---|---| +| tslang gc | 190.5 | 225.3 | 206.9 | 94.1 | 91.2 | 220.4 | +| tslang rc | 84.0 | 94.2 | 96.6 | 92.5 | 28.0 | 43.4 | +| tslang none | 92.3 | 100.7 | 101.5 | 94.0 | 23.8 | 44.8 | +| tslang own | 89.0 | 97.3 | 94.6 | 96.3 | 23.3 | 44.1 | +| tslang gc (jit) | 172.9 | 189.7 | 223.7 | 105.5 | 102.7 | 220.2 | +| tslang rc (jit) | 86.7 | 84.0 | 97.0 | 104.5 | 32.8 | 50.4 | +| tslang none (jit) | 82.4 | 85.3 | 111.7 | 105.6 | 23.0 | 45.2 | +| tslang own (jit) | 86.2 | 92.8 | 103.0 | 102.9 | 22.4 | 44.2 | +| Rust | 73.0 | 82.6 | 103.6 | 96.5 | 21.2 | 46.6 | +| C++ (clang) | 66.4 | 72.4 | 107.2 | 92.9 | 31.5 | 43.4 | + +- rc, none and own are within about 1.2x of Rust and C++ on every kernel, and level with them on + stencil, dot, matmul and hash. Their loops are vectorized (SSE2 in an executable, which targets + the baseline x86-64 as the Rust and C++ builds do). +- **gc is 2 to 5 times slower than the other models on every kernel but dot.** Its executable has + no packed floating-point instruction at all: none of the loops is vectorized, where the same + source under none has them. dot is the same in every build because nothing vectorizes it. Why + the gc build's loops stay scalar has not been looked into. +- dot is level across all ten rows: a strict floating-point reduction runs one addition at a time + everywhere. `--fast-math` is what would change it. + +### The JIT before and after the optimizer was given the host's CPU + +The JIT generates code for the host CPU. Before d78c735 the optimizer's cost model was for a +generic x86-64, so it vectorized for SSE2 and the code generator had nothing wider to emit. The +same benchmark under the release before that change (v0.0-pre-alpha122), same machine and run: + +| Build (jit) | saxpy | triad | stencil | dot | matmul | hash | +|---|---|---|---|---|---|---| +| none, alpha122 | 111.5 | 106.8 | 106.6 | 106.4 | 29.9 | 58.6 | +| none, d78c735 | 82.4 | 85.3 | 111.7 | 105.6 | 23.0 | 45.2 | +| rc, alpha122 | 103.9 | 104.0 | 107.5 | 105.1 | 38.4 | 52.6 | +| rc, d78c735 | 86.7 | 84.0 | 97.0 | 104.5 | 32.8 | 50.4 | +| own, alpha122 | 98.4 | 105.0 | 112.1 | 102.3 | 31.7 | 54.3 | +| own, d78c735 | 86.2 | 92.8 | 103.0 | 102.9 | 22.4 | 44.2 | + +In the object the JIT caches for `-mm=none`, alpha122 has no instruction on a 256-bit register and +d78c735 has 720. saxpy, triad, matmul and hash are 15-25% faster; stencil and dot do not move +(dot is not vectorized, and stencil's figures are within the noise of each other). Executables are +unchanged by it: they target the baseline CPU unless `-mcpu` says otherwise. diff --git a/tslang/benchmarks/numeric/bench.sh b/tslang/benchmarks/numeric/bench.sh new file mode 100755 index 000000000..21c6f7961 --- /dev/null +++ b/tslang/benchmarks/numeric/bench.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# Builds nbench.ts under every memory model and the Rust and C++ ports, checks that they all print +# the same six check values, then times each kernel by itself: a warm-up run and RUNS timed runs, +# reporting the median wall time in ms. +# +# TSLANG= [RUNS=9] [JIT_LIB=] bash bench.sh +# +# tslang finds its libraries through GC_LIB_PATH and TSLANG_LIB_PATH (or beside itself, as a +# release archive has them). With JIT_LIB the program is also timed under --emit=jit, the JIT cache +# warm, which adds the compiler's start-up to every figure. +set -u +here=$(cd "$(dirname "$0")" && pwd) +tslang=${TSLANG:-$here/../../../__build/tslang/ninja/release/bin/tslang} +runs=${RUNS:-9} +out=$here/out +mkdir -p "$out" +kernels="saxpy triad stencil dot matmul hash" +expected="saxpy: 346503 triad: 527762 stencil: 218331 dot: 369289 matmul: 615082 hash: 102784" + +names=() +cmds=() +add() { names+=("$1"); cmds+=("$2"); } + +flags="--opt --opt_level=3 --no-default-lib" +for mm in gc rc none own; do + if "$tslang" --emit=exe $flags --mm=$mm "$here/nbench.ts" -o "$out/nb_$mm" >"$out/nb_$mm.log" 2>&1; then + add "tslang $mm" "$out/nb_$mm" + else + echo "tslang $mm does not compile, see $out/nb_$mm.log" + fi + if [ -n "${JIT_LIB:-}" ]; then + add "tslang $mm (jit)" "$tslang --emit=jit $flags --mm=$mm --shared-libs=$JIT_LIB --jit-cache-dir=$out/jit_$mm $here/nbench.ts" + fi +done + +if command -v cargo >/dev/null; then + (cd "$here/rust" && cargo build --release --quiet) && add "rust" "$here/rust/target/release/nbench" +fi +if command -v clang++ >/dev/null; then + clang++ -std=c++20 -O3 -ffp-contract=off -o "$out/cpp_clang" "$here/cpp/nbench.cpp" && add "c++ (clang)" "$out/cpp_clang" +fi + +now_ms() { echo $(($(date +%s%N) / 1000)); } + +printf '%-20s' build +for k in $kernels; do printf '%9s' "$k"; done +printf ' %s\n' checksums + +for i in "${!names[@]}"; do + cmd=${cmds[$i]} + # the whole program once: the check values, and under the JIT the object for the cache + got=$($cmd | grep -v -e '^start' -e '^done' | tr '\n' ' ' | sed 's/ *$//') + printf '%-20s' "${names[$i]}" + for k in $kernels; do + $cmd "$k" >/dev/null + times=() + for _ in $(seq "$runs"); do + t0=$(now_ms); $cmd "$k" >/dev/null; t1=$(now_ms) + times+=($((t1 - t0))) + done + median=$(printf '%s\n' "${times[@]}" | sort -n | sed -n "$(((runs + 1) / 2))p") + printf '%9s' "$(awk "BEGIN { printf \"%.1f\", $median / 1000 }")" + done + if [ "$got" = "$expected" ]; then echo " ok"; else echo " MISMATCH: $got"; fi +done diff --git a/tslang/benchmarks/numeric/cpp/nbench.cpp b/tslang/benchmarks/numeric/cpp/nbench.cpp new file mode 100644 index 000000000..1ba6464ee --- /dev/null +++ b/tslang/benchmarks/numeric/cpp/nbench.cpp @@ -0,0 +1,135 @@ +// C++ port of tslang/benchmarks/numeric/nbench.ts: the same six kernels over std::vector / +// std::vector, printing the same six check values. Build with -ffp-contract=off: clang +// otherwise fuses a*b+c into one instruction where the CPU has it, which changes the low digits. +#include +#include +#include +#include +#include + +static const int N = 4096; +static const int M = 64; + +static double check(double v) { return std::fmod(std::floor(v * 1000.0), 999983.0); } + +static std::vector makeArray(int n, double mul, int mod, double add) { + std::vector a(n); + for (int i = 0; i < n; i++) a[i] = (i % mod) * mul + add; + return a; +} + +static double sum(const std::vector &a) { + double s = 0; + for (double v : a) s += v; + return s; +} + +__attribute__((noinline)) static void saxpy(double a, const std::vector &x, std::vector &y, int n) { + for (int i = 0; i < n; i++) y[i] += a * x[i]; +} + +__attribute__((noinline)) static void triad(const std::vector &x, const std::vector &y, + std::vector &z, double c, int n) { + for (int i = 0; i < n; i++) z[i] = x[i] * y[i] + c; +} + +__attribute__((noinline)) static void stencil(const std::vector &src, std::vector &dst, int n) { + for (int i = 1; i < n - 1; i++) dst[i] = (src[i - 1] + 2.0 * src[i] + src[i + 1]) * 0.25; +} + +__attribute__((noinline)) static double dot(const std::vector &x, const std::vector &y, int n) { + double s = 0; + for (int i = 0; i < n; i++) s += x[i] * y[i]; + return s; +} + +__attribute__((noinline)) static void matmul(const std::vector &a, const std::vector &b, + std::vector &c, int m) { + for (int i = 0; i < m; i++) + for (int k = 0; k < m; k++) { + const double aik = a[i * m + k]; + for (int j = 0; j < m; j++) c[i * m + j] += aik * b[k * m + j]; + } +} + +__attribute__((noinline)) static int hash(std::vector &h, const std::vector &g, int n) { + uint32_t s = 0; + for (int i = 0; i < n; i++) { + h[i] = (int)(((uint32_t)h[i] * 31u + (uint32_t)g[i]) & 0xffffu); + s += (uint32_t)h[i]; + } + return (int)s; +} + +static double runSaxpy() { + auto x = makeArray(N, 0.5, 97, 1.0); + auto y = makeArray(N, 0.25, 89, 2.0); + for (int r = 0; r < 60000; r++) saxpy(0.000001, x, y, N); + return check(sum(y)); +} + +static double runTriad() { + auto x = makeArray(N, 0.5, 97, 1.0); + auto y = makeArray(N, 0.25, 89, 2.0); + auto z = makeArray(N, 0.0, 1, 0.0); + double s = 0; + for (int r = 0; r < 60000; r++) { + triad(x, y, z, r * 0.001, N); + s += z[r % N]; + } + return check(s + sum(z)); +} + +static double runStencil() { + auto a = makeArray(N, 0.5, 97, 1.0); + auto b = makeArray(N, 0.5, 97, 1.0); + for (int r = 0; r < 30000; r++) { + stencil(a, b, N); + stencil(b, a, N); + } + return check(sum(a)); +} + +static double runDot() { + auto x = makeArray(N, 0.5, 97, 1.0); + auto y = makeArray(N, 0.25, 89, 2.0); + double s = 0; + for (int r = 0; r < 30000; r++) { + x[r % N] += 0.5; + s += dot(x, y, N) * 0.000001; + } + return check(s); +} + +static double runMatmul() { + auto a = makeArray(M * M, 0.001, 13, 0.01); + auto b = makeArray(M * M, 0.002, 7, 0.02); + auto c = makeArray(M * M, 0.0, 1, 0.0); + for (int r = 0; r < 400; r++) matmul(a, b, c, M); + return check(sum(c)); +} + +static int runHash() { + std::vector h(N), g(N); + for (int i = 0; i < N; i++) { + h[i] = i % 251; + g[i] = i % 241 + 7; + } + uint32_t s = 0; + for (int r = 0; r < 60000; r++) s = (s + (uint32_t)hash(h, g, N)) & 0xfffffu; + return (int)s; +} + +// With no argument every kernel runs; with a kernel's name only that one does. +int main(int argc, char **argv) { + const std::string only = argc > 1 ? argv[1] : ""; + auto run = [&](const char *name) { return only.empty() || only == name; }; + std::printf("start...\n"); + if (run("saxpy")) std::printf("saxpy: %.0f\n", runSaxpy()); + if (run("triad")) std::printf("triad: %.0f\n", runTriad()); + if (run("stencil")) std::printf("stencil: %.0f\n", runStencil()); + if (run("dot")) std::printf("dot: %.0f\n", runDot()); + if (run("matmul")) std::printf("matmul: %.0f\n", runMatmul()); + if (run("hash")) std::printf("hash: %d\n", runHash()); + std::printf("done.\n"); +} diff --git a/tslang/benchmarks/numeric/nbench.ts b/tslang/benchmarks/numeric/nbench.ts new file mode 100644 index 000000000..914e438bc --- /dev/null +++ b/tslang/benchmarks/numeric/nbench.ts @@ -0,0 +1,182 @@ +// Tight numeric loops over arrays: the code a vectorizer is for. Six kernels over flat arrays, +// each repeated until it takes a measurable time, each ending in a check value every build prints +// (the Rust and C++ ports in rust/ and cpp/ print the same six). +// +// saxpy y[i] += a * x[i] element-wise, vectorizable as written +// triad z[i] = x[i] * y[i] + c element-wise, three arrays +// stencil out[i] = (in[i-1] + 2 in[i] + in[i+1]) / 4 neighbours of the same array +// dot s += x[i] * y[i] a floating-point reduction: strict IEEE +// order keeps it scalar without --fast-math +// matmul c[i][j] += a[i][k] * b[k][j] 64x64, flat arrays, i-k-j order +// hash h[i] = (h[i] * 31 + g[i]) & 0xffff integers, with an integer reduction +// +// No kernel depends on the order of floating-point operations being changed, so every build +// gives the same six numbers as long as it does not contract a*b+c into a fused multiply-add. +declare function floor(v: number): number; +declare function fmod(a: number, b: number): number; + +const N = 4096; +const M = 64; + +// a value's digits folded into an integer below 999983, which print() shows exactly +function check(v: number): number { + return fmod(floor(v * 1000.0), 999983.0); +} + +function makeArray(n: int, mul: number, mod: int, add: number): number[] { + let a: number[] = []; + a.length = n; + for (let i = 0; i < n; i++) { + a[i] = (i % mod) * mul + add; + } + + return a; +} + +function sum(a: number[]): number { + let s = 0.0; + for (let i = 0; i < a.length; i++) { + s += a[i]; + } + + return s; +} + +function saxpy(a: number, x: number[], y: number[], n: int) { + for (let i = 0; i < n; i++) { + y[i] += a * x[i]; + } +} + +function triad(x: number[], y: number[], z: number[], c: number, n: int) { + for (let i = 0; i < n; i++) { + z[i] = x[i] * y[i] + c; + } +} + +function stencil(src: number[], dst: number[], n: int) { + for (let i = 1; i < n - 1; i++) { + dst[i] = (src[i - 1] + 2.0 * src[i] + src[i + 1]) * 0.25; + } +} + +function dot(x: number[], y: number[], n: int): number { + let s = 0.0; + for (let i = 0; i < n; i++) { + s += x[i] * y[i]; + } + + return s; +} + +function matmul(a: number[], b: number[], c: number[], m: int) { + for (let i = 0; i < m; i++) { + for (let k = 0; k < m; k++) { + const aik = a[i * m + k]; + for (let j = 0; j < m; j++) { + c[i * m + j] += aik * b[k * m + j]; + } + } + } +} + +function hash(h: int[], g: int[], n: int): int { + let s: int = 0; + for (let i = 0; i < n; i++) { + h[i] = (h[i] * 31 + g[i]) & 0xffff; + s += h[i]; + } + + return s; +} + +function runSaxpy(): number { + const x = makeArray(N, 0.5, 97, 1.0); + const y = makeArray(N, 0.25, 89, 2.0); + for (let r = 0; r < 60000; r++) { + saxpy(0.000001, x, y, N); + } + + return check(sum(y)); +} + +function runTriad(): number { + const x = makeArray(N, 0.5, 97, 1.0); + const y = makeArray(N, 0.25, 89, 2.0); + const z = makeArray(N, 0.0, 1, 0.0); + let s = 0.0; + for (let r = 0; r < 60000; r++) { + triad(x, y, z, r * 0.001, N); + s += z[r % N]; + } + + return check(s + sum(z)); +} + +function runStencil(): number { + let a = makeArray(N, 0.5, 97, 1.0); + let b = makeArray(N, 0.5, 97, 1.0); + for (let r = 0; r < 30000; r++) { + stencil(a, b, N); + stencil(b, a, N); + } + + return check(sum(a)); +} + +function runDot(): number { + const x = makeArray(N, 0.5, 97, 1.0); + const y = makeArray(N, 0.25, 89, 2.0); + let s = 0.0; + for (let r = 0; r < 30000; r++) { + x[r % N] += 0.5; + s += dot(x, y, N) * 0.000001; + } + + return check(s); +} + +function runMatmul(): number { + const a = makeArray(M * M, 0.001, 13, 0.01); + const b = makeArray(M * M, 0.002, 7, 0.02); + const c = makeArray(M * M, 0.0, 1, 0.0); + for (let r = 0; r < 400; r++) { + matmul(a, b, c, M); + } + + return check(sum(c)); +} + +function runHash(): number { + let h: int[] = []; + let g: int[] = []; + h.length = N; + g.length = N; + for (let i = 0; i < N; i++) { + h[i] = i % 251; + g[i] = i % 241 + 7; + } + + let s: int = 0; + for (let r = 0; r < 60000; r++) { + s = (s + hash(h, g, N)) & 0xfffff; + } + + return s; +} + +// With no argument every kernel runs; with a kernel's name only that one does, so bench.sh can +// time them one by one. +function main(argc: int, args: Ref): int { + const only = argc > 1 ? Deref(args[1]) : ""; + const all = argc <= 1; + print("start..."); + if (all || only == "saxpy") print("saxpy:", runSaxpy()); + if (all || only == "triad") print("triad:", runTriad()); + if (all || only == "stencil") print("stencil:", runStencil()); + if (all || only == "dot") print("dot:", runDot()); + if (all || only == "matmul") print("matmul:", runMatmul()); + if (all || only == "hash") print("hash:", runHash()); + print("done."); + return 0; +} diff --git a/tslang/benchmarks/numeric/rust/Cargo.toml b/tslang/benchmarks/numeric/rust/Cargo.toml new file mode 100644 index 000000000..7131afdf9 --- /dev/null +++ b/tslang/benchmarks/numeric/rust/Cargo.toml @@ -0,0 +1,10 @@ +[package] +name = "nbench" +version = "0.1.0" +edition = "2021" + +[profile.release] +opt-level = 3 +codegen-units = 1 +lto = "fat" +panic = "abort" diff --git a/tslang/benchmarks/numeric/rust/src/main.rs b/tslang/benchmarks/numeric/rust/src/main.rs new file mode 100644 index 000000000..5640c104c --- /dev/null +++ b/tslang/benchmarks/numeric/rust/src/main.rs @@ -0,0 +1,134 @@ +// Rust port of tslang/benchmarks/numeric/nbench.ts: the same six kernels over Vec / Vec, +// written the way a Rust programmer would (slices and iterators where they fit), printing the same +// six check values. + +const N: usize = 4096; +const M: usize = 64; + +fn check(v: f64) -> f64 { (v * 1000.0).floor() % 999983.0 } + +fn make_array(n: usize, mul: f64, modulus: usize, add: f64) -> Vec { + (0..n).map(|i| (i % modulus) as f64 * mul + add).collect() +} + +fn sum(a: &[f64]) -> f64 { + let mut s = 0.0; + for v in a { s += *v; } + s +} + +#[inline(never)] +fn saxpy(a: f64, x: &[f64], y: &mut [f64]) { + for (yi, xi) in y.iter_mut().zip(x) { *yi += a * *xi; } +} + +#[inline(never)] +fn triad(x: &[f64], y: &[f64], z: &mut [f64], c: f64) { + for ((zi, xi), yi) in z.iter_mut().zip(x).zip(y) { *zi = *xi * *yi + c; } +} + +#[inline(never)] +fn stencil(src: &[f64], dst: &mut [f64]) { + let n = src.len(); + for (d, w) in dst[1..n - 1].iter_mut().zip(src.windows(3)) { + *d = (w[0] + 2.0 * w[1] + w[2]) * 0.25; + } +} + +#[inline(never)] +fn dot(x: &[f64], y: &[f64]) -> f64 { + let mut s = 0.0; + for (xi, yi) in x.iter().zip(y) { s += *xi * *yi; } + s +} + +#[inline(never)] +fn matmul(a: &[f64], b: &[f64], c: &mut [f64], m: usize) { + for i in 0..m { + for k in 0..m { + let aik = a[i * m + k]; + let row = &b[k * m..k * m + m]; + for (cj, bj) in c[i * m..i * m + m].iter_mut().zip(row) { *cj += aik * *bj; } + } + } +} + +#[inline(never)] +fn hash(h: &mut [i32], g: &[i32]) -> i32 { + let mut s: i32 = 0; + for (hi, gi) in h.iter_mut().zip(g) { + *hi = (hi.wrapping_mul(31).wrapping_add(*gi)) & 0xffff; + s = s.wrapping_add(*hi); + } + s +} + +fn run_saxpy() -> f64 { + let x = make_array(N, 0.5, 97, 1.0); + let mut y = make_array(N, 0.25, 89, 2.0); + for _ in 0..60000 { saxpy(0.000001, &x, &mut y); } + check(sum(&y)) +} + +fn run_triad() -> f64 { + let x = make_array(N, 0.5, 97, 1.0); + let y = make_array(N, 0.25, 89, 2.0); + let mut z = make_array(N, 0.0, 1, 0.0); + let mut s = 0.0; + for r in 0..60000usize { + triad(&x, &y, &mut z, r as f64 * 0.001); + s += z[r % N]; + } + check(s + sum(&z)) +} + +fn run_stencil() -> f64 { + let mut a = make_array(N, 0.5, 97, 1.0); + let mut b = make_array(N, 0.5, 97, 1.0); + for _ in 0..30000 { + stencil(&a, &mut b); + stencil(&b, &mut a); + } + check(sum(&a)) +} + +fn run_dot() -> f64 { + let mut x = make_array(N, 0.5, 97, 1.0); + let y = make_array(N, 0.25, 89, 2.0); + let mut s = 0.0; + for r in 0..30000usize { + x[r % N] += 0.5; + s += dot(&x, &y) * 0.000001; + } + check(s) +} + +fn run_matmul() -> f64 { + let a = make_array(M * M, 0.001, 13, 0.01); + let b = make_array(M * M, 0.002, 7, 0.02); + let mut c = make_array(M * M, 0.0, 1, 0.0); + for _ in 0..400 { matmul(&a, &b, &mut c, M); } + check(sum(&c)) +} + +fn run_hash() -> i32 { + let mut h: Vec = (0..N as i32).map(|i| i % 251).collect(); + let g: Vec = (0..N as i32).map(|i| i % 241 + 7).collect(); + let mut s: i32 = 0; + for _ in 0..60000 { s = s.wrapping_add(hash(&mut h, &g)) & 0xfffff; } + s +} + +// With no argument every kernel runs; with a kernel's name only that one does. +fn main() { + let only = std::env::args().nth(1); + let run = |name: &str| only.as_deref().map_or(true, |o| o == name); + println!("start..."); + if run("saxpy") { println!("saxpy: {}", run_saxpy()); } + if run("triad") { println!("triad: {}", run_triad()); } + if run("stencil") { println!("stencil: {}", run_stencil()); } + if run("dot") { println!("dot: {}", run_dot()); } + if run("matmul") { println!("matmul: {}", run_matmul()); } + if run("hash") { println!("hash: {}", run_hash()); } + println!("done."); +} From 207d70551d0ea20743d14723ba3d3a6817b1edbe Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 13:09:34 +0000 Subject: [PATCH 2/3] gc: the collector's allocations are noalias, so loops over arrays are vectorized An array is a header holding the pointer to its elements. GC_malloc and GC_realloc were declared without noalias, which LLVM gives malloc's and realloc's results by name, so a store to an element might have written the header: every loop loaded the elements' pointer again on each iteration and none was vectorized. The numeric benchmark ran 2 to 5 times slower under gc than under the other memory models; it is now level with them. The README's Performance section has the numeric benchmark's table, and the benchmarks' READMEs the results on Linux. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J8KRfVk4FT2FALW47Ux2Fp --- README.md | 21 ++++++++++ tslang/benchmarks/numeric/README.md | 60 ++++++++++++++++++---------- tslang/benchmarks/raytrace/README.md | 29 ++++++++++++++ tslang/lib/TypeScript/GCPass.cpp | 15 +++++++ 4 files changed, 104 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index e8254547f..def22efb9 100644 --- a/README.md +++ b/README.md @@ -791,6 +791,27 @@ calling methods nothing overrides directly, and testing a union for null without still pays for retains and releases in the inner loops. The benchmark's README has every step and the earlier results. +Loops over arrays are measured by [tslang/benchmarks/numeric](tslang/benchmarks/numeric/README.md): +six kernels over flat arrays of 4096 elements, with Rust and C++ ports (release build, Ubuntu 24.04 +x64 on a 4-core cloud VM, median of 9 runs, ms, measured 2026-11-10). + +| build | saxpy | triad | stencil | dot | matmul | hash | +| --- | --- | --- | --- | --- | --- | --- | +| `-mm=gc` | 85.5 | 93.3 | 97.5 | 93.2 | 23.2 | 44.6 | +| `-mm=rc` | 97.9 | 95.0 | 98.8 | 98.1 | 28.7 | 44.7 | +| `-mm=none` | 88.7 | 93.8 | 97.3 | 93.6 | 23.2 | 43.8 | +| `-mm=own` | 87.1 | 94.7 | 97.3 | 96.5 | 26.7 | 48.0 | +| Rust (`Vec`, iterators) | 68.2 | 76.0 | 103.4 | 94.9 | 20.6 | 44.4 | +| C++ (`std::vector`, clang -O3) | 69.0 | 75.2 | 107.4 | 93.5 | 30.1 | 42.8 | + +Here the memory model does not matter: the loops allocate nothing, and all four models vectorize +them. tslang is level with Rust and C++ on four of the kernels and takes about 1.25x as long on +saxpy and triad. Under `--emit=jit` the loops are vectorized for the host's CPU (AVX2 on that +machine) rather than the baseline x86-64 an executable targets, which made saxpy, triad, matmul +and hash 15-25% faster than the JIT of the release before it. The same ray tracer on that Linux +machine takes 128 ms under `-mm=gc`, 123 under `-mm=none`, 130 under `-mm=own` and 238 under +`-mm=rc`, against 103 ms for the C++ port; the benchmarks' READMEs have the full tables. + ### `-mm=own` `-mm=own` works the way Rust's borrow checker does. Every heap block has exactly one owner: a diff --git a/tslang/benchmarks/numeric/README.md b/tslang/benchmarks/numeric/README.md index 907ea4d5e..b0357ef9e 100644 --- a/tslang/benchmarks/numeric/README.md +++ b/tslang/benchmarks/numeric/README.md @@ -36,32 +36,50 @@ them one by one. A JIT figure includes the compiler's start-up, about 13 ms here ## Results 2026-11-10 -Release, Ubuntu 24.04 x64, a 4-core cloud VM (Xeon 2.1 GHz, AVX2); main d78c735. Median of 9 runs -after a warm-up, ms; every build prints the same check values. The machine is shared, so a few -percent either way is noise. +Release, Ubuntu 24.04 x64, a 4-core cloud VM (Xeon 2.1 GHz, AVX2); main d78c735 with the collector's +allocations declared `noalias` (below). Median of 9 runs after a warm-up, ms; every build prints +the same check values. The machine is shared, so a few percent either way is noise. | Build | saxpy | triad | stencil | dot | matmul | hash | |---|---|---|---|---|---|---| -| tslang gc | 190.5 | 225.3 | 206.9 | 94.1 | 91.2 | 220.4 | -| tslang rc | 84.0 | 94.2 | 96.6 | 92.5 | 28.0 | 43.4 | -| tslang none | 92.3 | 100.7 | 101.5 | 94.0 | 23.8 | 44.8 | -| tslang own | 89.0 | 97.3 | 94.6 | 96.3 | 23.3 | 44.1 | -| tslang gc (jit) | 172.9 | 189.7 | 223.7 | 105.5 | 102.7 | 220.2 | -| tslang rc (jit) | 86.7 | 84.0 | 97.0 | 104.5 | 32.8 | 50.4 | -| tslang none (jit) | 82.4 | 85.3 | 111.7 | 105.6 | 23.0 | 45.2 | -| tslang own (jit) | 86.2 | 92.8 | 103.0 | 102.9 | 22.4 | 44.2 | -| Rust | 73.0 | 82.6 | 103.6 | 96.5 | 21.2 | 46.6 | -| C++ (clang) | 66.4 | 72.4 | 107.2 | 92.9 | 31.5 | 43.4 | - -- rc, none and own are within about 1.2x of Rust and C++ on every kernel, and level with them on - stencil, dot, matmul and hash. Their loops are vectorized (SSE2 in an executable, which targets - the baseline x86-64 as the Rust and C++ builds do). -- **gc is 2 to 5 times slower than the other models on every kernel but dot.** Its executable has - no packed floating-point instruction at all: none of the loops is vectorized, where the same - source under none has them. dot is the same in every build because nothing vectorizes it. Why - the gc build's loops stay scalar has not been looked into. +| tslang gc | 85.5 | 93.3 | 97.5 | 93.2 | 23.2 | 44.6 | +| tslang rc | 97.9 | 95.0 | 98.8 | 98.1 | 28.7 | 44.7 | +| tslang none | 88.7 | 93.8 | 97.3 | 93.6 | 23.2 | 43.8 | +| tslang own | 87.1 | 94.7 | 97.3 | 96.5 | 26.7 | 48.0 | +| tslang gc (jit) | 101.2 | 93.4 | 118.6 | 106.9 | 32.6 | 57.2 | +| tslang rc (jit) | 91.4 | 86.9 | 96.4 | 108.2 | 33.3 | 42.0 | +| tslang none (jit) | 84.0 | 88.7 | 104.0 | 104.3 | 22.4 | 46.1 | +| tslang own (jit) | 90.9 | 91.0 | 108.0 | 103.3 | 25.6 | 48.5 | +| Rust | 68.2 | 76.0 | 103.4 | 94.9 | 20.6 | 44.4 | +| C++ (clang) | 69.0 | 75.2 | 107.4 | 93.5 | 30.1 | 42.8 | + +- The four memory models are level with each other, and level with Rust and C++ on stencil, dot, + matmul and hash. On saxpy and triad tslang takes about 1.25x as long as the two ports. All of + them vectorize these loops for SSE2: an executable targets the baseline x86-64, as the Rust and + C++ builds do. - dot is level across all ten rows: a strict floating-point reduction runs one addition at a time everywhere. `--fast-math` is what would change it. +- The JIT rows generate the same loops for the host's CPU (below) and carry the compiler's + start-up. The gc and none objects the JIT caches hold the same number of vector instructions; + what makes the gc row slower than the none row here was not looked into. + +### gc before its allocations were `noalias` + +An array is a header holding the pointer to its elements. Under rc, none and own the header and +the elements come from `malloc` and `realloc`, whose results LLVM knows by name to be blocks no +other pointer reaches. Under gc they come from `GC_malloc` and `GC_realloc`, which were declared +without that, so a store to an element might have written the header: every loop loaded the +elements' pointer again on each iteration, and none of them was vectorized. GCPass now declares +the results of the collector's allocation functions `noalias`. + +| Build | saxpy | triad | stencil | dot | matmul | hash | +|---|---|---|---|---|---|---| +| tslang gc, d78c735 | 190.5 | 225.3 | 206.9 | 94.1 | 91.2 | 220.4 | +| tslang gc, with `noalias` | 85.5 | 93.3 | 97.5 | 93.2 | 23.2 | 44.6 | + +Before the change the gc executable had no packed floating-point instruction; after it, its +optimized LLVM IR has as many vector operations as the none build's. The ray tracer in +`../raytrace` is unchanged by it (gc 123 ms before, 128 ms after, within that run's noise). ### The JIT before and after the optimizer was given the host's CPU diff --git a/tslang/benchmarks/raytrace/README.md b/tslang/benchmarks/raytrace/README.md index a52c9f9d3..60cdd5666 100644 --- a/tslang/benchmarks/raytrace/README.md +++ b/tslang/benchmarks/raytrace/README.md @@ -26,6 +26,35 @@ pwsh tslang/benchmarks/raytrace/bench.ps1 - Rust: `opt-level=3`, `lto="fat"`, `codegen-units=1`, `panic="abort"` - C++: `clang++ -std=c++20 -O3`, and `cl /std:c++20 /O2 /EHsc /MT` +## Results 2026-11-10: Linux + +Release, Ubuntu 24.04 x64, a 4-core cloud VM (Xeon 2.1 GHz); main d78c735 with the collector's +allocations declared `noalias`. `bench.ps1` is for Windows, so the same builds were made and timed +by hand: median of 15 runs, taken in turn so that the machine's drift falls on every build alike. +Every build prints the same checksum. The machine is shared, so a few percent either way is noise. + +| Build | Median | vs C++ idiomatic | +|---|---|---| +| tslang gc | 128 ms | 1.2x | +| tslang rc | 238 ms | 2.3x | +| tslang none | 123 ms | 1.2x | +| tslang own | 130 ms | 1.3x | +| Rust idiomatic | 164 ms | 1.6x | +| Rust Rc-boxed | 398 ms | 3.8x | +| C++ idiomatic (clang) | 103 ms | 1x | +| C++ shared_ptr (clang) | 571 ms | 5.5x | + +The ratios are against the C++ port here, because the Rust idiomatic port is 1.6x slower than it +on this machine, where on Windows the two are within 10% of each other. Built with +`-C target-cpu=native` the Rust port takes 109 ms, and the same flag does not change the C++ port; +what the baseline x86-64 Rust build loses the time to was not looked into. The tslang builds +target the baseline CPU too. + +Under `--emit=jit` with the JIT cache warm, the times are the executables' plus the compiler's +start-up (about 13 ms): the JIT's code for this benchmark is the executable's, and the same before +and after the optimizer was given the host's CPU (d78c735) - there are no loops here for it to +vectorize. `../numeric` is the benchmark that shows that change. + ## Results 2026-10-31: parameters captured by copy, direct method calls, null without a tag Release, Windows 11 x64, the same machine; main 1cd17c33 against the same tree with each change diff --git a/tslang/lib/TypeScript/GCPass.cpp b/tslang/lib/TypeScript/GCPass.cpp index 55970c0ed..9e35db968 100644 --- a/tslang/lib/TypeScript/GCPass.cpp +++ b/tslang/lib/TypeScript/GCPass.cpp @@ -220,6 +220,21 @@ class GCPass : public mlir::PassWrapper // each call is treated as returning a distinct, non-aliasing pointer. void markAsAllocatorIfNeeded(StringRef newName, LLVM::LLVMFuncOp funcOp) { + // No other pointer reaches the block an allocation returns, nor the one a resize returns: + // `noalias`, which LLVM gives malloc's and realloc's results by their names and cannot give + // these. An array is a header holding the pointer to its elements, and without this a + // store to an element might have written the header, so every loop over an array loaded + // the elements' pointer again on each iteration and none of them was vectorized - the + // numeric benchmark ran 2 to 5 times slower under gc than under the other models. + if (newName == "GC_malloc" || newName == "GC_malloc_atomic" || newName == "GC_memalign" || + newName == "GC_malloc_uncollectable" || newName == "GC_realloc") + { + if (funcOp.getNumResults() > 0) + { + funcOp.setResultAttr(0, LLVM::LLVMDialect::getNoAliasAttrName(), mlir::UnitAttr::get(funcOp->getContext())); + } + } + if (newName != "GC_malloc" && newName != "GC_malloc_atomic" && newName != "GC_memalign" && newName != "GC_malloc_uncollectable") { From 48bea045c3eae0370565b1d59637d74348856e3a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 13:23:56 +0000 Subject: [PATCH 3/3] Parser: a source position's line without copying the line table, and no regex over a whole file for a shebang - getLineStarts returned the file's table of line starts by value, and computeLineAndCharacterOfPosition and computeLineOfPosition took it by value again: three copies of the table for every source location. They now pass a reference. Compiling a small program with the default library executed 983M instructions, a quarter of them these copies; it is 725M. - '^#!.*' can only match at the start of the text, but regex_search tried it at every position of every file (3% of the same compile). The first two characters are tested before the regex is used. - getDirectiveFromComment took its regex by value, a copy per comment; it takes a reference, and a comment with no '@' is not searched. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J8KRfVk4FT2FALW47Ux2Fp --- tslang/ts-new-parser/scanner.cpp | 38 ++++++++++++++++++++++++++------ tslang/ts-new-parser/scanner.h | 10 ++++----- 2 files changed, 36 insertions(+), 12 deletions(-) diff --git a/tslang/ts-new-parser/scanner.cpp b/tslang/ts-new-parser/scanner.cpp index 13a55809c..dbdedd1de 100644 --- a/tslang/ts-new-parser/scanner.cpp +++ b/tslang/ts-new-parser/scanner.cpp @@ -1168,7 +1168,7 @@ auto Scanner::getPositionOfLineAndCharacter(SourceFileLike sourceFile, number li } /* @internal */ -auto Scanner::computePositionOfLineAndCharacter(std::vector lineStarts, number line, number character, string debugText, +auto Scanner::computePositionOfLineAndCharacter(const std::vector &lineStarts, number line, number character, const string &debugText, bool allowEdits) -> number { if (line < 0 || line >= lineStarts.size()) @@ -1209,7 +1209,7 @@ auto Scanner::computePositionOfLineAndCharacter(std::vector lineStarts, } /* @internal */ -auto Scanner::getLineStarts(SourceFileLike sourceFile) -> std::vector +auto Scanner::getLineStarts(SourceFileLike sourceFile) -> const std::vector & { if (!sourceFile->lineMap.empty()) { @@ -1225,7 +1225,7 @@ auto Scanner::getLineStarts(SourceFileLike sourceFile) -> std::vector } /* @internal */ -auto Scanner::computeLineAndCharacterOfPosition(std::vector lineStarts, number position) -> LineAndCharacter +auto Scanner::computeLineAndCharacterOfPosition(const std::vector &lineStarts, number position) -> LineAndCharacter { auto lineNumber = computeLineOfPosition(lineStarts, position); return LineAndCharacter({lineNumber, position - lineStarts[lineNumber]}); @@ -1235,7 +1235,7 @@ auto Scanner::computeLineAndCharacterOfPosition(std::vector lineStarts, * @internal * We assume the first line starts at position 0 and 'position' is non-negative. */ -auto Scanner::computeLineOfPosition(std::vector lineStarts, number position, number lowerBound) -> number +auto Scanner::computeLineOfPosition(const std::vector &lineStarts, number position, number lowerBound) -> number { auto lineNumber = binarySearch(lineStarts, position, &identity, &compareValues, lowerBound); if (lineNumber < 0) @@ -1258,7 +1258,7 @@ auto Scanner::getLinesBetweenPositions(SourceFileLike sourceFile, number pos1, n { if (pos1 == pos2) return 0; - auto lineStarts = getLineStarts(sourceFile); + const auto &lineStarts = getLineStarts(sourceFile); auto lower = std::min(pos1, pos2); auto isNegative = lower == pos2; auto upper = isNegative ? pos1 : pos2; @@ -1541,17 +1541,30 @@ auto Scanner::scanConflictMarkerTrivia(safe_string &text, number pos, std::funct regex Scanner::shebangTriviaRegex = regex(S("^#!.*")); +// `^#!.*` can only match at the start of the text, but regex_search still tries it at every +// position of the file. Asking for the two characters first keeps the regex for the files that +// do start with a shebang. +static auto startsWithShebang(const string &text) -> boolean +{ + return text.size() >= 2 && text[0] == S('#') && text[1] == S('!'); +} + /*@internal*/ auto Scanner::isShebangTrivia(string &text, number pos) -> boolean { // Shebangs check must only be done at the start of the file debug(pos == 0); - return regex_search(text, shebangTriviaRegex); + return startsWithShebang(text) && regex_search(text, shebangTriviaRegex); } /*@internal*/ auto Scanner::scanShebangTrivia(string &text, number pos) -> number { + if (!startsWithShebang(text)) + { + return pos; + } + auto words_begin = sregex_iterator(text.begin(), text.end(), shebangTriviaRegex); auto words_end = sregex_iterator(); for (auto i = words_begin; i != words_end; ++i) @@ -1593,6 +1606,11 @@ auto Scanner::getTrailingCommentRanges(string &text, pos_type pos) -> std::vecto /** Optionally, get the shebang */ auto Scanner::getShebang(string &text) -> string { + if (!startsWithShebang(text)) + { + return string(); + } + auto words_begin = sregex_iterator(text.begin(), text.end(), shebangTriviaRegex); auto words_end = sregex_iterator(); for (auto i = words_begin; i != words_end; ++i) @@ -3198,8 +3216,14 @@ auto Scanner::appendIfCommentDirective(std::vector commentDire return commentDirectives; } -auto Scanner::getDirectiveFromComment(string &text, regex commentDirectiveRegEx) -> CommentDirectiveType +auto Scanner::getDirectiveFromComment(string &text, const regex &commentDirectiveRegEx) -> CommentDirectiveType { + // both directive patterns need an '@'; most comments have none + if (text.find(S('@')) == string::npos) + { + return CommentDirectiveType::Undefined; + } + auto words_begin = sregex_iterator(text.begin(), text.end(), commentDirectiveRegEx); auto words_end = sregex_iterator(); if (words_begin == words_end) diff --git a/tslang/ts-new-parser/scanner.h b/tslang/ts-new-parser/scanner.h index 7ab84d0d2..1becc3f20 100644 --- a/tslang/ts-new-parser/scanner.h +++ b/tslang/ts-new-parser/scanner.h @@ -316,20 +316,20 @@ class Scanner auto getPositionOfLineAndCharacter(SourceFileLike sourceFile, number line, number character, bool allowEdits = true) -> number; /* @internal */ - auto computePositionOfLineAndCharacter(std::vector lineStarts, number line, number character, string debugText, + auto computePositionOfLineAndCharacter(const std::vector &lineStarts, number line, number character, const string &debugText, bool allowEdits = true) -> number; /* @internal */ - auto getLineStarts(SourceFileLike sourceFile) -> std::vector; + auto getLineStarts(SourceFileLike sourceFile) -> const std::vector &; /* @internal */ - auto computeLineAndCharacterOfPosition(std::vector lineStarts, number position) -> LineAndCharacter; + auto computeLineAndCharacterOfPosition(const std::vector &lineStarts, number position) -> LineAndCharacter; /** * @internal * We assume the first line starts at position 0 and 'position' is non-negative. */ - auto computeLineOfPosition(std::vector lineStarts, number position, number lowerBound = 0) -> number; + auto computeLineOfPosition(const std::vector &lineStarts, number position, number lowerBound = 0) -> number; /** @internal */ auto getLinesBetweenPositions(SourceFileLike sourceFile, number pos1, number pos2); @@ -632,7 +632,7 @@ class Scanner auto appendIfCommentDirective(std::vector commentDirectives, string text, regex commentDirectiveRegEx, pos_type lineStart) -> std::vector; - auto getDirectiveFromComment(string &text, regex commentDirectiveRegEx) -> CommentDirectiveType; + auto getDirectiveFromComment(string &text, const regex &commentDirectiveRegEx) -> CommentDirectiveType; auto reScanTemplateToken(boolean isTaggedTemplate) -> SyntaxKind;