Skip to content

varnish: set thread_stats_rate=1000 - #1351

Draft
gquintard wants to merge 2 commits into
MDA2AV:mainfrom
gquintard:varnish-thread-stats-rate
Draft

varnish: set thread_stats_rate=1000#1351
gquintard wants to merge 2 commits into
MDA2AV:mainfrom
gquintard:varnish-thread-stats-rate

Conversation

@gquintard

@gquintard gquintard commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Sets thread_stats_rate=1000 (default 10) to defer per-thread worker-stat aggregation under high request rates. Experiment to isolate whether stats aggregation overhead is contributing to the baseline/pipelined RPS ceiling.
  • Fixes a missing line-continuation after vsl_mask=none that made -s malloc,256m dead code, never reaching varnishd.

Test plan

@gquintard @claude
varnish: set thread_stats_rate=1000
6068a32 
Deferred per-thread stat aggregation is cheaper under high request
rates than the default (10). Experiment to see if reduced worker-stat
aggregation overhead moves the needle on baseline/pipelined RPS.

Also fixes a missing line-continuation after vsl_mask=none that made
-s malloc,256m dead code, never reaching varnishd.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

👋 Heads up! This PR modifies the following frameworks:

@gquintard

gquintard commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f varnish --save

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: varnish | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 192,378 6488.4% 576MiB -6.8% +6.7%
baseline 4096 186,371 6361.9% 829MiB -9.0% -1.9%
pipelined 512 274,345 6440.6% 545MiB -12.8% +8.6%
pipelined 4096 261,097 6419.8% 862MiB -8.0% -35.2%
limited-conn 512 159,568 6266.6% 572MiB -9.2% -0.7%
limited-conn 4096 174,825 6288.2% 794MiB -7.8% +1.1%
baseline-h2 256 201,437 6436.7% 4.6GiB -5.8% +2.2%
baseline-h2 1024 190,938 6445.7% 4.8GiB -7.4% ~0%
static-h2 256 150,707 6548.4% 2.1GiB -0.6% -4.5%
static-h2 1024 138,289 6491.0% 4.7GiB -1.7% ~0%
Full log 
Process Request Failure:1179
Process Request Failure:936
Process Request Failure:1245
Process Request Failure:1254
Process Request Failure:1354
Process Request Failure:182
Process Request Failure:1101
Process Request Failure:1599
Process Request Failure:1434
Process Request Failure:1330
Process Request Failure:886
Process Request Failure:1593
Process Request Failure:1364
Process Request Failure:1490
Process Request Failure:1892
Process Request Failure:1695
Process Request Failure:1776
Process Request Failure:1632
Process Request Failure:1665
Process Request Failure:1665
Process Request Failure:1603
Process Request Failure:1479
Process Request Failure:1680
Process Request Failure:1244
Process Request Failure:1818
Process Request Failure:1448
Process Request Failure:1740
Process Request Failure:1286
Process Request Failure:944
Process Request Failure:1306
Process Request Failure:1293
Process Request Failure:1333
Process Request Failure:1410
Process Request Failure:1788
Process Request Failure:1730
Process Request Failure:1494
Process Request Failure:1278
Process Request Failure:1360
Process Request Failure:1780
Process Request Failure:1453
Process Request Failure:1295
Process Request Failure:1734
Process Request Failure:1335
Process Request Failure:1328
Process Request Failure:1337
Process Request Failure:1384
Process Request Failure:1489
Process Request Failure:1710
Process Request Failure:1542
Process Request Failure:1849
Process Request Failure:1194
Process Request Failure:1313
Process Request Failure:1412
Process Request Failure:1820
Process Request Failure:1378
Process Request Failure:1507
Process Request Failure:1433
Process Request Failure:1724
Process Request Failure:1348
Process Request Failure:1391
Process Request Failure:1749
Process Request Failure:1570
Process Request Failure:1486
Process Request Failure:1684
Process Request Failure:1335
Process Request Failure:1184
Process Request Failure:1538
Process Request Failure:541
Process Request Failure:1759
Process Request Failure:1425
Process Request Failure:1706
Process Request Failure:1620
Process Request Failure:1804
Process Request Failure:1448
Process Request Failure:1542
Process Request Failure:1567
Process Request Failure:1375
Process Request Failure:1725
Process Request Failure:1170
Process Request Failure:1223
Process Request Failure:1321
Process Request Failure:1608
Process Request Failure:1544
Process Request Failure:1463
Process Request Failure:1689
Process Request Failure:346
Process Request Failure:1752
Process Request Failure:1509
Process Request Failure:1691
Process Request Failure:1043
Process Request Failure:1500
Process Request Failure:1304
Process Request Failure:1856
Process Request Failure:1314
Process Request Failure:1576
Process Request Failure:1569
Process Request Failure:1900
Process Request Failure:1946
Process Request Failure:1645
Process Request Failure:1394
Process Request Failure:1396
Process Request Failure:1271
Process Request Failure:1407
Process Request Failure:1525
Process Request Failure:1183
Process Request Failure:1462
Process Request Failure:1814
Process Request Failure:1804
Process Request Failure:1074
Process Request Failure:1218
Process Request Failure:1590
Process Request Failure:1906
Process Request Failure:271
Process Request Failure:1458
Process Request Failure:1547
Process Request Failure:1108
Process Request Failure:1774
Process Request Failure:1462
Process Request Failure:1520
Process Request Failure:1841
Process Request Failure:1455
Process Request Failure:754
Process Request Failure:1505
Process Request Failure:1313
Process Request Failure:1820
Process Request Failure:1393
Process Request Failure:1620
Process Request Failure:1987
Process Request Failure:1805
Process Request Failure:1694
Process Request Failure:1407
Process Request Failure:1361
Process Request Failure:1892
Process Request Failure:1172
Process Request Failure:1631
Process Request Failure:1609
Process Request Failure:1840
Process Request Failure:1563
Process Request Failure:1814
Process Request Failure:1513
Process Request Failure:1645
Process Request Failure:1850
Process Request Failure:1443
Process Request Failure:1857
Process Request Failure:1875
Process Request Failure:1590
Process Request Failure:1948
Process Request Failure:2022
Process Request Failure:1458
Process Request Failure:1617
Process Request Failure:1953
Process Request Failure:1458
Process Request Failure:1738
Process Request Failure:1986
Process Request Failure:1892
Process Request Failure:1833
Process Request Failure:1364
Process Request Failure:1935

5Stopped all clients for thread #19

. Stopping all clients.Stopped all clients for thread #46
168. Stopping all clients.


finished in 5.06s, 136813.40 req/s, 8.15GB/s
requests: 761375 total, 793060 started, 761375 done, 684067 succeeded, 77308 failed, 77308 errored, 0 timeout
status codes: 699746 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 40.73GB (43729203329) total, 119.00MB (124782591) headers (space savings 21.21%), 40.58GB (43569962442) data
                     min         max         mean         sd        +/- sd
time for request:     2.83ms       1.81s    130.77ms     46.16ms    76.05%
time for connect:     3.67ms       1.81s    649.23ms    597.76ms    57.32%
time to 1st byte:    10.35ms       1.87s    689.28ms    614.90ms    55.94%
req/s           :       0.00      367.28      188.60       69.74    77.54%
[info] CPU 6491.0% | Mem 4.7GiB

=== Best: 138289 req/s (CPU: 6491.0%, Mem: 4.7GiB) ===
[info] saved results/static-h2/1024/varnish.json
httparena-bench-varnish
httparena-bench-varnish
[info] skip: varnish does not subscribe to baseline-h2c
[info] skip: varnish does not subscribe to json-h2c
[info] skip: varnish does not subscribe to baseline-h3
[info] skip: varnish does not subscribe to static-h3
[info] skip: varnish does not subscribe to gateway-64
[info] skip: varnish does not subscribe to gateway-h3
[info] skip: varnish does not subscribe to production-stack
[info] skip: varnish does not subscribe to unary-grpc
[info] skip: varnish does not subscribe to unary-grpc-tls
[info] skip: varnish does not subscribe to echo-ws
[info] skip: varnish does not subscribe to echo-ws-pipeline
[info] skip: varnish does not subscribe to echo-ws-limited
[info] skip: varnish does not subscribe to latency-1m
[info] skip: varnish does not subscribe to async
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/varnish.json - 10 new, 10 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

@gquintard
gquintard marked this pull request as draft August 28, 2026 05:11
@gquintard

gquintard commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

well, that's surprising

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MDA2AV MDA2AV Awaiting requested review from MDA2AV MDA2AV is a code owner
@Kaliumhexacyanoferrat Kaliumhexacyanoferrat Awaiting requested review from Kaliumhexacyanoferrat Kaliumhexacyanoferrat is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gquintard