Skip to content

Update roadrunner to 0.3.0 with configurable listener limits#769

Closed
williamthome wants to merge 3 commits into
MDA2AV:mainfrom
williamthome:feat/roadunner-0.3.0
Closed

Update roadrunner to 0.3.0 with configurable listener limits#769
williamthome wants to merge 3 commits into
MDA2AV:mainfrom
williamthome:feat/roadunner-0.3.0

Conversation

@williamthome
Copy link
Copy Markdown
Contributor

@williamthome williamthome commented May 31, 2026

Description

Bumps the roadrunner dependency from 0.2.3 to hex 0.3.0, with configurable listener limits, the handler_spawn memory option, and the quic 1.5.0 transport.


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

roadrunner defaults to 150 concurrent connections and rejects connections
over the cap, but the HttpArena profiles drive 512-16384 connections, so the
default throttled every high-conn test. A local json run at 4096 conns showed
the cap rejecting and forcing ~2M connection retries; raising it collapses
that to the reconnect floor and roughly doubles throughput. Sizing the
connection cap to the environment is standard deployment config for a
Production entry.

Pins the roadrunner dep to the conn_spawn memory branch (swap to a hex
version before merge).
@williamthome williamthome changed the title [roadrunner] Update roadrunner to 0.3.0 Update roadrunner to 0.3.0 with configurable listener limits May 31, 2026
@williamthome williamthome marked this pull request as ready for review May 31, 2026 20:14
@williamthome
Copy link
Copy Markdown
Contributor Author

/benchmark -f roadrunner

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: roadrunner | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 805,649 6115.6% 274MiB +4.4% -13.0%
baseline 4096 749,298 6081.6% 353MiB +9.1% +15.0%
pipelined 512 1,195,637 6330.0% 201MiB -5.4% +2.6%
pipelined 4096 1,203,699 6267.7% 247MiB -4.1% +22.3%
limited-conn 512 378,636 3729.2% 212MiB ~0% -4.9%
limited-conn 4096 375,309 3664.3% 224MiB -2.2% -12.8%
json 4096 209,344 6163.1% 405MiB +7.0% -28.4%
json-comp 512 101,943 6341.5% 665MiB +4.7% -16.7%
json-comp 4096 100,033 6194.6% 654MiB -2.9% -23.3%
json-comp 16384 104,010 6281.1% 684MiB -5.0% -15.2%
json-tls 4096 162,665 5940.4% 821MiB +212.4% +77.7%
upload 32 2,876 1965.8% 372MiB ~0% -22.3%
upload 256 2,920 5147.0% 684MiB -0.9% +9.3%
api-4 256 16,741 399.6% 157MiB -8.2% -3.7%
api-16 1024 40,157 1445.1% 264MiB -11.3% -24.4%
static 1024 39,992 4584.4% 256MiB -67.5% +29.9%
static 4096 42,789 4590.7% 409MiB -64.4% +103.5%
static 6800 36,597 4549.4% 560MiB -73.1% +175.9%
async-db 1024 116,836 3789.8% 713MiB +37.6% +13.4%
crud 4096 414,025 4307.0% 776MiB +109.1% +104.2%
fortunes 1024 125,313 4671.7% 845MiB +195.2% -6.9%
baseline-h2 256 452,259 6433.3% 476MiB -2.6% -15.0%
baseline-h2 1024 441,697 6277.3% 933MiB -6.5% +58.9%
static-h2 256 5,565 4365.3% 560MiB -74.1% -41.1%
static-h2 1024 17,254 4471.0% 1.5GiB +44.4% +112.2%
baseline-h2c 256 791,606 6446.8% 525MiB -5.1% -6.1%
baseline-h2c 1024 735,909 6312.6% 929MiB -17.1% +59.3%
baseline-h2c 4096 661,762 6232.2% 1.8GiB -25.6% +233.3%
json-h2c 1024 106,269 5193.7% 5.5GiB -11.1% +223.5%
json-h2c 4096 101,381 4845.8% 12.6GiB -25.9% +600.0%
baseline-h3 64 61,300 6150.4% 1.0GiB +7.6% ~0%
static-h3 64 3,187 4469.9% 568MiB +8.2% -22.2%
echo-ws 512 1,323,177 6368.3% 230MiB +48.7% -32.6%
echo-ws 4096 1,352,577 6599.8% 333MiB +173.6% +27.1%
echo-ws 16384 1,250,000 6096.3% 616MiB +47.4% +89.5%
echo-ws-pipeline 512 1,687,591 6524.3% 183MiB -12.2% -9.4%
echo-ws-pipeline 4096 1,602,517 6565.9% 340MiB -4.4% +33.3%
echo-ws-pipeline 16384 1,594,879 6114.3% 389MiB +42.2% +82.6%
Full log

==============================================
=== roadrunner / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   26.98ms   28.20ms   44.10ms   59.50ms   75.70ms

  8037970 frames sent in 5.00s, 7975967 frames received
  Throughput: 1.59M req/s
  Bandwidth:  10.73MB/s
  WS upgrades: 4096
  WS frames:   7975967
  Latency samples: 7975960 / 7975967 responses (100.0%)
[info] CPU 6382.6% | Mem 305MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   28.42ms   30.40ms   43.90ms   55.30ms   82.30ms

  8071899 frames sent in 5.00s, 8012586 frames received
  Throughput: 1.60M req/s
  Bandwidth:  10.77MB/s
  WS upgrades: 4095
  WS frames:   8012586
  Latency samples: 8012574 / 8012586 responses (100.0%)
[info] CPU 6565.9% | Mem 340MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   27.76ms   28.70ms   44.50ms   59.10ms   82.60ms

  7980746 frames sent in 5.00s, 7919417 frames received
  Throughput: 1.58M req/s
  Bandwidth:  10.65MB/s
  WS upgrades: 4096
  WS frames:   7919417
  Latency samples: 7919406 / 7919417 responses (100.0%)
[info] CPU 6197.2% | Mem 347MiB

=== Best: 1602517 req/s (CPU: 6565.9%, Mem: 340MiB) ===
[info] saved results/echo-ws-pipeline/4096/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner

==============================================
=== roadrunner / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   27.55ms   27.90ms   45.50ms   60.40ms   77.60ms

  8004261 frames sent in 5.00s, 7930117 frames received
  Throughput: 1.59M req/s
  Bandwidth:  10.68MB/s
  WS upgrades: 4666
  WS frames:   7930117
  Latency samples: 7930117 / 7930117 responses (100.0%)
[info] CPU 6023.4% | Mem 327MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   28.41ms   29.70ms   46.80ms   65.10ms   81.30ms

  7897330 frames sent in 5.00s, 7822946 frames received
  Throughput: 1.56M req/s
  Bandwidth:  10.54MB/s
  WS upgrades: 4649
  WS frames:   7822946
  Latency samples: 7822946 / 7822946 responses (100.0%)
[info] CPU 6504.3% | Mem 360MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   31.28ms   31.70ms   48.30ms   69.40ms   94.90ms

  8049007 frames sent in 5.00s, 7974399 frames received
  Throughput: 1.59M req/s
  Bandwidth:  10.74MB/s
  WS upgrades: 4919
  WS frames:   7974399
  Latency samples: 7974399 / 7974399 responses (100.0%)
[info] CPU 6114.3% | Mem 389MiB

=== Best: 1594879 req/s (CPU: 6114.3%, Mem: 389MiB) ===
[info] saved results/echo-ws-pipeline/16384/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/fortunes-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@williamthome williamthome marked this pull request as draft May 31, 2026 20:53
@williamthome
Copy link
Copy Markdown
Contributor Author

/benchmark -f roadrunner

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: roadrunner | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 799,612 6168.7% 246MiB +3.6% -21.9%
baseline 4096 801,421 6232.2% 310MiB +16.7% +1.0%
pipelined 512 1,194,221 6503.4% 208MiB -5.5% +6.1%
pipelined 4096 1,183,372 6354.8% 276MiB -5.7% +36.6%
limited-conn 512 385,066 3709.9% 218MiB +1.7% -2.2%
limited-conn 4096 378,536 4056.9% 222MiB -1.4% -13.6%
json 4096 221,051 6287.1% 398MiB +13.0% -29.7%
json-comp 512 103,924 6285.2% 712MiB +6.7% -10.8%
json-comp 4096 105,122 6351.4% 627MiB +2.1% -26.5%
json-comp 16384 105,935 6053.6% 665MiB -3.3% -17.6%
json-tls 4096 157,180 5937.9% 915MiB +201.9% +98.1%
upload 32 2,844 2161.4% 339MiB -1.1% -29.2%
upload 256 2,944 5514.1% 637MiB -0.1% +1.8%
api-4 256 16,856 401.0% 156MiB -7.6% -4.3%
api-16 1024 42,936 1532.9% 244MiB -5.2% -30.1%
static 1024 43,283 4456.2% 241MiB -64.9% +22.3%
static 4096 39,475 4518.3% 395MiB -67.1% +96.5%
static 6800 33,970 4536.9% 495MiB -75.0% +143.8%
async-db 1024 100,284 3077.0% 716MiB +18.1% +13.8%
crud 4096 410,416 4887.3% 730MiB +107.3% +92.1%
fortunes 1024 119,367 4764.4% 812MiB +181.2% -10.6%
baseline-h2 256 418,330 6238.8% 410MiB -9.9% -26.8%
baseline-h2 1024 416,899 6367.1% 473MiB -11.8% -19.4%
static-h2 256 7,313 4315.8% 446MiB -66.0% -53.1%
static-h2 1024 15,604 4699.1% 930MiB +30.6% +28.5%
baseline-h2c 256 765,798 6438.0% 268MiB -8.2% -52.1%
baseline-h2c 1024 734,702 6464.8% 361MiB -17.2% -38.1%
baseline-h2c 4096 645,146 6227.4% 553MiB -27.5% ~0%
json-h2c 1024 112,654 4773.2% 3.6GiB -5.8% +111.8%
json-h2c 4096 113,879 4616.6% 6.4GiB -16.8% +255.6%
baseline-h3 64 56,362 5621.6% 1.1GiB -1.0% +10.0%
static-h3 64 2,417 4402.4% 542MiB -17.9% -25.8%
echo-ws 512 1,336,989 6247.3% 238MiB +50.3% -30.2%
echo-ws 4096 1,309,796 6367.8% 376MiB +165.0% +43.5%
echo-ws 16384 1,159,807 5969.0% 554MiB +36.8% +70.5%
echo-ws-pipeline 512 1,687,964 6555.9% 183MiB -12.2% -9.4%
echo-ws-pipeline 4096 1,643,222 6363.2% 315MiB -2.0% +23.5%
echo-ws-pipeline 16384 1,609,095 6108.9% 308MiB +43.5% +44.6%
Full log

==============================================
=== roadrunner / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   25.81ms   24.70ms   43.60ms   60.50ms   79.30ms

  8278098 frames sent in 5.00s, 8216114 frames received
  Throughput: 1.64M req/s
  Bandwidth:  11.05MB/s
  WS upgrades: 4096
  WS frames:   8216114
  Latency samples: 8216106 / 8216114 responses (100.0%)
[info] CPU 6363.2% | Mem 315MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   27.38ms   29.30ms   43.30ms   59.40ms   82.40ms

  8133463 frames sent in 5.00s, 8074278 frames received
  Throughput: 1.61M req/s
  Bandwidth:  10.86MB/s
  WS upgrades: 4096
  WS frames:   8074278
  Latency samples: 8074263 / 8074278 responses (100.0%)
[info] CPU 6425.4% | Mem 347MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   27.09ms   29.40ms   44.00ms   59.50ms   80.60ms

  8158655 frames sent in 5.00s, 8098527 frames received
  Throughput: 1.62M req/s
  Bandwidth:  10.89MB/s
  WS upgrades: 4096
  WS frames:   8098527
  Latency samples: 8098507 / 8098527 responses (100.0%)
[info] CPU 6399.2% | Mem 364MiB

=== Best: 1643222 req/s (CPU: 6363.2%, Mem: 315MiB) ===
[info] saved results/echo-ws-pipeline/4096/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner

==============================================
=== roadrunner / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   25.54ms   24.10ms   42.40ms   68.70ms   84.80ms

  8115699 frames sent in 5.00s, 8045475 frames received
  Throughput: 1.61M req/s
  Bandwidth:  10.83MB/s
  WS upgrades: 4389
  WS frames:   8045475
  Latency samples: 8045475 / 8045475 responses (100.0%)
[info] CPU 6108.9% | Mem 308MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   30.50ms   31.10ms   48.00ms   59.00ms   92.70ms

  7957203 frames sent in 5.00s, 7880051 frames received
  Throughput: 1.58M req/s
  Bandwidth:  10.62MB/s
  WS upgrades: 4822
  WS frames:   7880051
  Latency samples: 7880051 / 7880051 responses (100.0%)
[info] CPU 6562.2% | Mem 328MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   32.03ms   32.90ms   49.30ms   62.20ms   70.80ms

  7979040 frames sent in 5.00s, 7901487 frames received
  Throughput: 1.58M req/s
  Bandwidth:  10.65MB/s
  WS upgrades: 4921
  WS frames:   7901487
  Latency samples: 7901487 / 7901487 responses (100.0%)
[info] CPU 6196.3% | Mem 364MiB

=== Best: 1609095 req/s (CPU: 6108.9%, Mem: 308MiB) ===
[info] saved results/echo-ws-pipeline/16384/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/fortunes-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@williamthome williamthome force-pushed the feat/roadunner-0.3.0 branch from 1ecfd96 to 5a76543 Compare June 1, 2026 00:03
@williamthome williamthome marked this pull request as ready for review June 1, 2026 01:24
@williamthome
Copy link
Copy Markdown
Contributor Author

/benchmark -f roadrunner --save

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

Benchmark Results

Framework: roadrunner | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 799,249 6202.3% 254MiB +3.6% -19.4%
baseline 4096 757,877 5992.9% 392MiB +10.3% +27.7%
pipelined 512 1,215,078 6542.4% 200MiB -3.9% +2.0%
pipelined 4096 1,223,760 6223.1% 251MiB -2.5% +24.3%
limited-conn 512 377,635 3649.5% 220MiB -0.2% -1.3%
limited-conn 4096 381,703 3814.2% 221MiB -0.6% -14.0%
json 4096 219,759 6139.7% 393MiB +12.3% -30.6%
json-comp 512 100,387 6312.4% 638MiB +3.1% -20.1%
json-comp 4096 98,428 6385.4% 645MiB -4.4% -24.4%
json-comp 16384 99,597 6249.0% 674MiB -9.0% -16.5%
json-tls 4096 164,486 5222.5% 884MiB +215.9% +91.3%
upload 32 2,882 1940.3% 316MiB +0.2% -34.0%
upload 256 2,964 5222.5% 679MiB +0.5% +8.5%
api-4 256 16,242 396.4% 155MiB -11.0% -4.9%
api-16 1024 42,207 1497.2% 262MiB -6.8% -24.9%
static 1024 39,176 4693.3% 256MiB -68.2% +29.9%
static 4096 36,747 4521.5% 392MiB -69.4% +95.0%
static 6800 40,718 4561.3% 553MiB -70.1% +172.4%
async-db 1024 142,135 5454.6% 697MiB +67.4% +10.8%
crud 4096 413,433 4253.6% 754MiB +108.8% +98.4%
fortunes 1024 109,161 3927.3% 824MiB +157.2% -9.3%
baseline-h2 256 456,364 6472.0% 526MiB -1.7% -6.1%
baseline-h2 1024 447,930 6242.3% 910MiB -5.2% +55.0%
static-h2 256 8,963 4275.2% 551MiB -58.3% -42.1%
static-h2 1024 0 4242.0% 1.1GiB -100.0% +55.6%
baseline-h2c 256 801,598 6466.2% 533MiB -3.9% -4.7%
baseline-h2c 1024 736,621 6402.5% 915MiB -17.0% +56.9%
baseline-h2c 4096 656,539 6279.4% 1.9GiB -26.2% +251.8%
json-h2c 1024 110,377 5551.1% 6.1GiB -7.7% +258.8%
json-h2c 4096 103,582 4809.5% 14.0GiB -24.3% +677.8%
baseline-h3 64 57,701 6002.5% 1.1GiB +1.3% +10.0%
static-h3 64 4,087 4532.5% 592MiB +38.8% -18.9%
echo-ws 512 1,323,241 6404.6% 263MiB +48.7% -22.9%
echo-ws 4096 1,358,189 6383.1% 292MiB +174.8% +11.5%
echo-ws 16384 1,067,802 5836.0% 636MiB +25.9% +95.7%
echo-ws-pipeline 512 1,719,777 6525.9% 182MiB -10.5% -9.9%
echo-ws-pipeline 4096 1,629,720 6368.2% 290MiB -2.8% +13.7%
echo-ws-pipeline 16384 1,632,958 6174.5% 324MiB +45.6% +52.1%
Full log

==============================================
=== roadrunner / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   26.14ms   28.10ms   43.80ms   57.90ms   67.30ms

  8210537 frames sent in 5.00s, 8148600 frames received
  Throughput: 1.63M req/s
  Bandwidth:  10.96MB/s
  WS upgrades: 4096
  WS frames:   8148600
  Latency samples: 8148595 / 8148600 responses (100.0%)
[info] CPU 6368.2% | Mem 290MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   27.80ms   30.30ms   44.10ms   59.10ms   85.20ms

  8114462 frames sent in 5.00s, 8054187 frames received
  Throughput: 1.61M req/s
  Bandwidth:  10.83MB/s
  WS upgrades: 4096
  WS frames:   8054187
  Latency samples: 8054301 / 8054187 responses (100.0%)
[info] CPU 6574.9% | Mem 332MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   28.12ms   29.60ms   44.40ms   58.90ms   77.30ms

  8007692 frames sent in 5.00s, 7947612 frames received
  Throughput: 1.59M req/s
  Bandwidth:  10.69MB/s
  WS upgrades: 4096
  WS frames:   7947612
  Latency samples: 7947603 / 7947612 responses (100.0%)
[info] CPU 6161.0% | Mem 343MiB

=== Best: 1629720 req/s (CPU: 6368.2%, Mem: 290MiB) ===
[info] saved results/echo-ws-pipeline/4096/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner

==============================================
=== roadrunner / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   27.11ms   28.70ms   44.90ms   61.60ms   70.60ms

  8239560 frames sent in 5.00s, 8164792 frames received
  Throughput: 1.63M req/s
  Bandwidth:  11.00MB/s
  WS upgrades: 4673
  WS frames:   8164792
  Latency samples: 8164792 / 8164792 responses (100.0%)
[info] CPU 6174.5% | Mem 324MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   29.83ms   30.60ms   47.20ms   66.10ms   83.40ms

  8129972 frames sent in 5.00s, 8052340 frames received
  Throughput: 1.61M req/s
  Bandwidth:  10.85MB/s
  WS upgrades: 4852
  WS frames:   8052340
  Latency samples: 8052340 / 8052340 responses (100.0%)
[info] CPU 6454.2% | Mem 385MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   45.99ms   49.40ms   61.70ms   77.10ms   101.80ms

  7862029 frames sent in 5.00s, 7765373 frames received
  Throughput: 1.55M req/s
  Bandwidth:  10.49MB/s
  WS upgrades: 6041
  WS frames:   7765373
  Latency samples: 7765373 / 7765373 responses (100.0%)
[info] CPU 6121.0% | Mem 458MiB

=== Best: 1632958 req/s (CPU: 6174.5%, Mem: 324MiB) ===
[info] saved results/echo-ws-pipeline/16384/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/fortunes-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@williamthome
Copy link
Copy Markdown
Contributor Author

I'm closing that one in favor of a new roadrunner version. I'll open a new PR with the latest stable version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant