test server performance by MDA2AV · Pull Request #779 · MDA2AV/HttpArena

MDA2AV · 2026-06-02T00:46:25Z

Description

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command	Description
`/benchmark -f <framework>`	Run all benchmark tests
`/benchmark -f <framework> -t <test>`	Run a specific test
`/benchmark -f <framework> --save`	Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.

Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

MDA2AV · 2026-06-02T00:46:44Z

/benchmark -f ringzero -t baseline

github-actions · 2026-06-02T00:46:51Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-06-02T00:48:33Z

Benchmark Results

Framework: ringzero | Test: baseline

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	4,356,593	6278.6%	4.1GiB	+4.9%	~0%
baseline	4096	4,252,170	6403.6%	4.2GiB	+4.9%	~0%

Full log

#18 exporting attestation manifest sha256:e50e549f692d6229b11998ae2094ee65f971700395eac0ca1d6cb2fc9a853df1 0.0s done
#18 exporting manifest list sha256:cb5a4860565b5da2619c1fc508c4001e90670f86daf55b8a6a9ef63a8ef16ec5 0.0s done
#18 naming to docker.io/library/httparena-ringzero:latest done
#18 unpacking to docker.io/library/httparena-ringzero:latest
#18 unpacking to docker.io/library/httparena-ringzero:latest done
#18 DONE 0.1s
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches

==============================================
=== ringzero / baseline / 512c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    116us    116us    136us    153us    167us

  21783577 requests in 5.00s, 21783576 responses
  Throughput: 4.35M req/s
  Bandwidth:  348.86MB/s
  Status codes: 2xx=21782967, 3xx=0, 4xx=609, 5xx=0
  Latency samples: 21783510 / 21783576 responses (100.0%)
  Per-template: 7284643,7279417,7219449
  Per-template-ok: 7284234,7279217,7219449

  WARNING: 609/21783576 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6278.6% | Mem 4.1GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    117us    116us    137us    154us    172us

  21640562 requests in 5.00s, 21640562 responses
  Throughput: 4.33M req/s
  Bandwidth:  346.58MB/s
  Status codes: 2xx=21639871, 3xx=0, 4xx=691, 5xx=0
  Latency samples: 21640496 / 21640562 responses (100.0%)
  Per-template: 7233729,7225544,7181223
  Per-template-ok: 7233271,7225311,7181222

  WARNING: 691/21640562 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6402.6% | Mem 4.1GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    117us    117us    137us    154us    169us

  21613637 requests in 5.00s, 21613635 responses
  Throughput: 4.32M req/s
  Bandwidth:  346.16MB/s
  Status codes: 2xx=21612976, 3xx=0, 4xx=659, 5xx=0
  Latency samples: 21613559 / 21613635 responses (100.0%)
  Per-template: 7232021,7219877,7161661
  Per-template-ok: 7231589,7219650,7161661

  WARNING: 659/21613635 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6341.9% | Mem 4.1GiB

=== Best: 4356593 req/s (CPU: 6278.6%, Mem: 4.1GiB) ===
[info] input BW: 336.54MB/s (avg template: 81 bytes)
[info] saved results/baseline/512/ringzero.json
httparena-bench-ringzero
httparena-bench-ringzero

==============================================
=== ringzero / baseline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    978us    958us   1.04ms   1.13ms   4.04ms

  21046166 requests in 5.00s, 21046166 responses
  Throughput: 4.21M req/s
  Bandwidth:  337.04MB/s
  Status codes: 2xx=21044313, 3xx=0, 4xx=1853, 5xx=0
  Latency samples: 21046146 / 21046166 responses (100.0%)
  Per-template: 7017320,7013232,7015594
  Per-template-ok: 7016130,7012571,7015592

  WARNING: 1853/21046166 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6000.4% | Mem 4.2GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    962us    965us   1.04ms   1.11ms   3.90ms

  21262732 requests in 5.00s, 21262731 responses
  Throughput: 4.25M req/s
  Bandwidth:  340.55MB/s
  Status codes: 2xx=21260851, 3xx=0, 4xx=1880, 5xx=0
  Latency samples: 21262681 / 21262731 responses (100.0%)
  Per-template: 7090118,7084878,7087685
  Per-template-ok: 7088828,7084288,7087685

  WARNING: 1880/21262731 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6403.6% | Mem 4.2GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    966us    969us   1.05ms   1.11ms   3.94ms

  21179704 requests in 5.00s, 21179701 responses
  Throughput: 4.23M req/s
  Bandwidth:  339.19MB/s
  Status codes: 2xx=21177792, 3xx=0, 4xx=1909, 5xx=0
  Latency samples: 21179601 / 21179701 responses (100.0%)
  Per-template: 7063106,7057163,7059332
  Per-template-ok: 7061810,7056549,7059332

  WARNING: 1909/21179701 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6175.3% | Mem 4.2GiB

=== Best: 4252170 req/s (CPU: 6403.6%, Mem: 4.2GiB) ===
[info] input BW: 328.47MB/s (avg template: 81 bytes)
[info] saved results/baseline/4096/ringzero.json
httparena-bench-ringzero
httparena-bench-ringzero
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

MDA2AV · 2026-06-02T00:49:20Z

@EdmondDantes opened this to bench true-async-server

benchmarked ringzero baseline which is performing well

MDA2AV · 2026-06-02T00:49:36Z

/benchmark -f true-async-server -t baseline

github-actions · 2026-06-02T00:49:44Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-06-02T00:51:27Z

Benchmark Results

Framework: true-async-server | Test: baseline

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,515,833	6522.0%	186MiB	-29.7%	+2.2%
baseline	4096	1,454,891	6327.4%	216MiB	-35.6%	-1.8%

Full log


#8 [4/4] COPY entry.php PostgreSQL.php SQLite.php /app/
#8 CACHED

#9 exporting to image
#9 exporting layers done
#9 exporting manifest sha256:fa06b414309886129b1962e9f955a3977c262fc7103bf2af80e6212f6759ceb7 0.0s done
#9 exporting config sha256:d6444e418dd66f7c96033574f12c01008dec568b93357553cce98356b0850db3
#9 exporting config sha256:d6444e418dd66f7c96033574f12c01008dec568b93357553cce98356b0850db3 0.0s done
#9 exporting attestation manifest sha256:7b613707ffd0e0777301e587bec230a4d6fb09b8fc03cc6deab62d7152b6a397 0.0s done
#9 exporting manifest list sha256:936bd29b029ef7371c75e4ec4eff0b12141ff57b222994c2ded3c96515998f8d 0.0s done
#9 naming to docker.io/library/httparena-true-async-server:latest
#9 naming to docker.io/library/httparena-true-async-server:latest done
#9 unpacking to docker.io/library/httparena-true-async-server:latest 0.1s done
#9 DONE 0.2s
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] starting postgres sidecar
[info] postgres ready (seeded)

==============================================
=== true-async-server / baseline / 512c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    338us    315us    543us    939us   1.55ms

  7543470 requests in 5.00s, 7543470 responses
  Throughput: 1.51M req/s
  Bandwidth:  192.75MB/s
  Status codes: 2xx=7543470, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 7545970 / 7543470 responses (100.0%)
  Per-template: 2599396,2478499,2465547
  Per-template-ok: 2599396,2478499,2465547
[info] CPU 6589.2% | Mem 187MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    337us    309us    530us    953us   1.82ms

  7579168 requests in 5.00s, 7579167 responses
  Throughput: 1.52M req/s
  Bandwidth:  193.66MB/s
  Status codes: 2xx=7579167, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 7579142 / 7579167 responses (100.0%)
  Per-template: 2621815,2524686,2432641
  Per-template-ok: 2621815,2524686,2432641
[info] CPU 6522.0% | Mem 186MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    339us    317us    529us    891us   1.59ms

  7535862 requests in 5.00s, 7535862 responses
  Throughput: 1.51M req/s
  Bandwidth:  192.56MB/s
  Status codes: 2xx=7535862, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 7535836 / 7535862 responses (100.0%)
  Per-template: 2622645,2476711,2436480
  Per-template-ok: 2622645,2476711,2436480
[info] CPU 6591.5% | Mem 185MiB

=== Best: 1515833 req/s (CPU: 6522.0%, Mem: 186MiB) ===
[info] input BW: 117.09MB/s (avg template: 81 bytes)
[info] saved results/baseline/512/true-async-server.json
httparena-bench-true-async-server
httparena-bench-true-async-server

==============================================
=== true-async-server / baseline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.83ms   2.85ms   3.30ms   4.24ms   7.00ms

  7258564 requests in 5.00s, 7258564 responses
  Throughput: 1.45M req/s
  Bandwidth:  185.47MB/s
  Status codes: 2xx=7258564, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 7258543 / 7258564 responses (100.0%)
  Per-template: 2426605,2418831,2413106
  Per-template-ok: 2426605,2418831,2413106
[info] CPU 6146.4% | Mem 207MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.81ms   2.79ms   3.37ms   4.38ms   6.89ms

  7276122 requests in 5.00s, 7274458 responses
  Throughput: 1.45M req/s
  Bandwidth:  185.88MB/s
  Status codes: 2xx=7274458, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 7274430 / 7274458 responses (100.0%)
  Per-template: 2428331,2423907,2422192
  Per-template-ok: 2428331,2423907,2422192
[info] CPU 6327.4% | Mem 216MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.83ms   2.83ms   3.32ms   3.94ms   6.26ms

  7233068 requests in 5.00s, 7232748 responses
  Throughput: 1.45M req/s
  Bandwidth:  184.81MB/s
  Status codes: 2xx=7232748, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 7232715 / 7232748 responses (100.0%)
  Per-template: 2420777,2397978,2413960
  Per-template-ok: 2420777,2397978,2413960
[info] CPU 6194.9% | Mem 217MiB

=== Best: 1454891 req/s (CPU: 6327.4%, Mem: 216MiB) ===
[info] input BW: 112.39MB/s (avg template: 81 bytes)
[info] saved results/baseline/4096/true-async-server.json
httparena-bench-true-async-server
httparena-bench-true-async-server
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

MDA2AV · 2026-06-02T00:55:57Z

Ok, I tested locally on the server with an older HEAD before true-async-server changes and the performance holds on 2.2M, so it seems this is a real regression for baseline on true-async-server @EdmondDantes

EdmondDantes · 2026-06-02T07:05:47Z

Ok, I tested locally on the server with an older HEAD before true-async-server changes and the performance holds on 2.2M, so it seems this is a real regression for baseline on true-async-server @EdmondDantes

Thank you very much, I'll try to figure out what's causing it!

test server performance

54be82a

MDA2AV requested a review from Kaliumhexacyanoferrat as a code owner June 2, 2026 00:46

Repository owner deleted a comment from github-actions Bot Jun 2, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test server performance#779

test server performance#779
MDA2AV wants to merge 1 commit into
mainfrom
test-server-perf

MDA2AV commented Jun 2, 2026

Uh oh!

MDA2AV commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

MDA2AV commented Jun 2, 2026

Uh oh!

MDA2AV commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

MDA2AV commented Jun 2, 2026 •

edited

Loading

Uh oh!

EdmondDantes commented Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

MDA2AV commented Jun 2, 2026

Description

Uh oh!

MDA2AV commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Benchmark Results

Uh oh!

MDA2AV commented Jun 2, 2026

Uh oh!

MDA2AV commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Benchmark Results

Uh oh!

MDA2AV commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EdmondDantes commented Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MDA2AV commented Jun 2, 2026 •

edited

Loading