[nahime0]

The historical benchmark suites measure process wall time, which hides the remaining costs inside mixed workloads. Add an opt-in comparison runner that compiles one shared PHP fixture with explicitly selected compiler revisions and measures arithmetic, closure calls, string append, dynamic eval, JSON decode with teardown, total body time and process wall time separately.

The runner requires real PHP, checks every warmup and measured checksum, rotates variant order, preserves raw samples and executable/input fingerprints, isolates runtime caches, and rejects noisy phases using median absolute deviation. Defaults are two warmups, 21 rounds and a 5% noise limit. Historical data and workflows are unchanged.

Validation: all 42 benchmark tooling tests pass with the exact CI discovery command, the PHP fixture passes php -n -l, and git diff --check passes. A quiet Linux x86_64 comparison used two warmups and 21 rotating samples on CPU11 with 64MiB native heaps. Body medians for published v0.27.3 / merged main f96ee9e / PHP8.5.11 are 1004.094 / 21.887 / 8.099 ms; process wall medians are 1009.222 / 26.884 / 34.526 ms. Every phase passed the 5% MAD gate (maximum 2.28%). These ratios describe this fixture only.

Require a source revision for every compiler label before any subprocess starts. SHA-256 fingerprints use bounded streaming reads without depending on Python 3.11 APIs; regression tests cover empty, short and multichunk files plus missing and partially supplied revisions.

[nahime0]

@greptileai please review the current head 7e772ce.

[greptile-apps]

Confidence Score: 5/5

[Low impact] The PR appears safe to merge; both previous findings are fixed and no new actionable issues remain.

Summary

Adds an opt-in runner that compares mixed-workload phases across selected compilers and PHP.

• Compiler comparisons now show mixed-workload phase and process times.

No new actionable issues were found in the changes since the previous review.

_{Reviews (2) · Last reviewed commit: "fix(benchmark): require revisions and po..." · Reviewed by Greptile}

[nahime0]

@greptileai please review current head 4380c55. Both reported issues are fixed, with regression tests.