ParallelComputeLab provides an end-to-end benchmarking infrastructure using Google Benchmark, combined with Python automation scripts for metadata logging, result validation, statistical aggregation, and chart generation.
Run the automated benchmark suite:
python3 ParallelComputeLab/scripts/run_benchmarks.py \
--binary build/release/parallel_compute_benchmarks \
--output-dir ParallelComputeLab/results \
--repetitions 3 \
--min-time 0.2This command automatically:
- Gathers hardware topology, cache hierarchy, and compiler metadata into
results/system_metadata.json. - Executes Google Benchmark with specified repetition and timing settings.
- Saves raw JSON measurements into
results/raw_benchmark_results.jsonand CSV intoresults/raw_benchmark_results.csv. - Runs
validate_results.pyto ensure schema conformance and measurement integrity.
Process the raw results to produce summary metrics, speedup/efficiency calculations, markdown tables, and plots:
python3 ParallelComputeLab/scripts/analyze_results.py \
ParallelComputeLab/results/raw_benchmark_results.json \
ParallelComputeLab/resultsGenerated Output Artifacts:
-
results/summary_metrics.json: Median, mean, stddev, IQR, bandwidth, and speedup for each benchmark. -
results/summary_table.md: Formatted Markdown table comparing variants, scaling factors, and efficiency. -
results/scaling_threads.png: Strong scaling speedup$S(p)$ and parallel efficiency$E(p)$ plots. -
results/workload_comparison.png: Comparison of Scalar, Optimized, SIMD, and Parallel variants across working sets. -
results/memory_bandwidth.png: Strided memory access latency showing cache eviction penalties. -
results/scheduling_comparison.png: Runtime comparison of OpenMP static, dynamic, and guided scheduling policies.
The parallel_compute_benchmarks binary accepts standard Google Benchmark flags:
# Filter benchmarks by name regex
./build/release/parallel_compute_benchmarks --benchmark_filter="BM_MatMul.*"
# Run 5 repetitions per configuration
./build/release/parallel_compute_benchmarks --benchmark_repetitions=5
# Set minimum time per repetition
./build/release/parallel_compute_benchmarks --benchmark_min_time=0.5s
# Output to JSON
./build/release/parallel_compute_benchmarks --benchmark_out=results/custom.json --benchmark_out_format=jsonTo differentiate cache-resident computation from memory-bandwidth bottlenecks, benchmarks are evaluated across four representative memory levels:
| Working Set Size | Element Count (Float) | Memory Footprint | Primary Memory Subsystem |
|---|---|---|---|
| Small | 1,024 elements | 4 KiB | L1 Data Cache (32 KiB) |
| Medium | 16,384 elements | 64 KiB | L2 Cache (256 KiB) |
| Large | 262,144 elements | 1 MiB | L3 Shared Cache (6 MiB) |
| DRAM | 4,194,304 elements | 16 MiB | Main Memory (DRAM) |
The raw benchmark output follows the Google Benchmark JSON schema version 1, enriched with custom context:
{
"context": {
"cpu_model": "Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz",
"cpu_vendor": "GenuineIntel",
"os_name": "Linux 6.12.113+deb13-amd64",
"architecture": "x86_64",
"logical_cores": "8",
"physical_cores": "4",
"openmp_max_threads": "8",
"l1d_cache_bytes": "32768",
"l2_cache_bytes": "262144",
"l3_cache_bytes": "6291456",
"compiler_id": "GNU",
"compiler_version": "14.2.0",
"build_type": "Release",
"has_avx2": "true",
"has_fma": "true"
},
"benchmarks": [
{
"name": "BM_VectorAdd_SIMD/4194304",
"iterations": 83,
"real_time": 8234125.12,
"cpu_time": 8233400.05,
"time_unit": "ns",
"bytes_per_second": 6112349000.0,
"items_per_second": 509362416.0
}
]
}