You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Subscribes roadrunner to async and echo-ws-limited: adds the /delay/{ms} endpoint and a deeper h1 accept queue.
PR Commands — comment on this PR to trigger (requires collaborator approval):
Command
Description
/benchmark -f <framework>
Run every test the framework subscribes to
/benchmark -f <framework> -t <test>
Run one test only
/benchmark -f <framework> --save
Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save
Run one test and save results
/benchmark -f <framework> --compare <other>
Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,...
Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save
No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test>
Benchmark all enabled frameworks subscribed to <test> and save the results
For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.
What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:
The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.
Run benchmarks locally
You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.
⚠️/benchmark --save cannot start: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.
⚠️/benchmark --save aborted: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Subscribes roadrunner to
asyncandecho-ws-limited: adds the/delay/{ms}endpoint and a deeper h1 accept queue.PR Commands — comment on this PR to trigger (requires collaborator approval):
/benchmark -f <framework>/benchmark -f <framework> -t <test>/benchmark -f <framework> --save/benchmark -f <framework> -t <test> --save/benchmark -f <framework> --compare <other>/benchmark-multiple -f <fw1>,<fw2>,...-tand--savetoo; saved results land in a single commit/benchmark-multiple --save-fneeded: benchmark and save every framework the PR touches/benchmark-test -t <test><test>and save the resultsFor
/benchmark, always specify-f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer/benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another.--compareworks on single-framework runs only.What the deltas are measured against. By default, this framework's own results published on
main- answering "did this change help?". When you are tuning a variant or a successor entry,--comparere-bases them on another entry instead:The reply states which baseline it used, and profiles the other framework does not run show
n/arather than a delta.Run benchmarks locally
You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.
Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk) are built as self-contained Docker images on first run.