This file records what this project cannot currently do, and what could not be verified in the environment it was developed in. Everything here is a statement about measurements and coverage, not about intentions.
$ perf stat -e cycles,instructions true
access to performance monitoring and observability operations for processes
without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
perf_event_paranoid setting is 3
kernel.perf_event_paranoid is 3 on the development machine, so perf stat
cannot read hardware counters at all — not even for the calling process.
Consequently:
- No IPC, cache-miss, branch-miss or stall-cycle measurement was taken.
- Bottleneck claims in this repository rest on timing evidence and static analysis only (vectoriser reports, disassembly, size sweeps). Where a bottleneck was not demonstrated, it is not claimed.
scripts/collect_system_info.pyrecordsperf.usable = falseso this is visible in every result set.
On a host with perf_event_paranoid <= 1 the commands in
docs/PERFORMANCE_METHODOLOGY.md work unchanged. Re-running them there would
let the "why is this faster" questions be answered with counters instead of
inference.
The development machine has libblas.so.3 at runtime but no cblas.h, so
PCL_HAVE_BLAS is off and gemm::matmul_blas() returns false.
Status of the BLAS baseline: NOT RUN locally. The code path is written and
unit-tested for the "no BLAS" contract (it must return false and leave C
untouched — MatmulBasics.BlasAvailabilityIsReported checks exactly that), and
the CI benchmark-smoke job installs libopenblas-dev so the comparison runs
there. Until that job's artifact has been inspected, no speedup against a
third-party BLAS is claimed anywhere in this repository.
The NEON kernels in src/simd/simd_kernels.cpp are written for AArch64 and are
compiled by the aarch64-neon CI job, but they were not executed on the
development host, which is x86-64. Their local status is therefore
NOT RUN, not "verified".
Consequences to keep in mind when reading those numbers:
- The double-precision NEON path relies on the compiler contracting a 2-lane
loop into
vld1q/vst1q+vfmaqpairs rather than using explicit intrinsics, so its vectorisation depends on the compiler and was not inspected on real hardware here. - The AArch64 SIMD GEMM micro-kernel assumes
ncols >= 4forfloat; the narrower cases are dispatched to a scalar loop by the caller.
CPU: Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz (4 cores / 8 threads, 1 socket)
Caches: L1d 32 KiB x4, L2 256 KiB x4, L3 6 MiB x1
Memory: ~7.5 GiB
Scaling: CPU scaling enabled (governor is not `performance`)
intel_pstatescaling is active, so timings drift with thermal and power state. Google Benchmark warns about this on startup. Repetitions, random interleaving and median reporting mitigate it; they do not eliminate it.- Only 8 logical cores means the maximum thread count is small and the scaling curves are short. Nothing in the published results should be read as a statement about large-scale machines.
- It is a laptop, so it is shared with whatever else is running. The recorded
load_averageand dispersion columns are the evidence for how much that mattered.
These are choices, not accidents:
- No
Backend::bestthat includes OpenMP.Backend::bestpicks the fastest single-threaded variant. Folding threading into an autotuning choice would make "best vs reference at T=1" meaningless. Thread scaling is measured separately and explicitly. - No explicit SIMD kernel for
min/max. Comparison-based reductions are branch bound and did not justify hand-written intrinsics.Backend::simdfor those maps to the serial fold, and the benchmark registers that entry anyway so the claim is visible in the data rather than only in this file. - No hand-written assembly. No profiling result on any available host demonstrated a need for it.
- No custom task scheduler. OpenMP's
static/dynamic/guidedare measured first; nothing in the results identified a scheduling problem that would justify writing one. - No AVX-512 kernels. The development CPU has no AVX-512. Writing kernels that cannot be executed here would violate the "never execute an unsupported instruction" rule in spirit and could not be validated.
- No f32 element-wise benchmarks. The library implements
floatand tests it, but the benchmark suite measuresf64for element-wise and memory workloads andf64for GEMM (plus one float GEMM numerical test). f32 bandwidth numbers are not available.
- Reduction and GEMM results are only defined to within a tolerance, because parallel and SIMD variants reassociate the sum. The reference variant's accumulation order is the strict left-to-right one; everything else is unspecified and documented as such.
meanissum / n. It is not more accurate thansum, and on adversarial input (large dynamic range) it is the less stable of the two. The library exposes it because the workloads need it, not because it is good.- A tree reduction cannot recover information a flat summation also loses.
tests/numericaldocuments this limit rather than hiding it.
scripts/analyze_results.pyneedsmatplotlibfor plots. Without it the CSV and markdown report are still produced and a note says the plots were skipped.- Dispersion statistics are computed over the repetitions Google Benchmark recorded in a single process. They describe within-run variability, not run-to-run reproducibility across machines or days. Repeating a run and comparing the medians is the way to check that.
rel_iqrisNaNfor fewer than 4 repetitions, because a quartile of one or two samples is not informative. The column is left empty rather than filled with a misleading number.
Recorded so they are not re-tried blindly; the numbers behind these statements
are in results/analysis/report.md.
-march=nativeas the supported configuration. Rejected: it produces a binary that cannot be compared against, or even run on, another machine. It survives only as thegcc-avx2-nativepreset, used for inspecting generated code.Backend::best= "fastest overall". Rejected: it silently mixes thread counts into what is otherwise a single-threaded comparison.- A hand-written OpenMP scheduling queue. Rejected before implementation: the scheduling experiment measures OpenMP's own policies first, and no measured gap justified a bespoke scheduler.