Skip to content

perf(bench): add jemalloc/tcmalloc baselines and p99 tail-latency reporting #111

Description

@danielPoloWork

Summary

Extend the pool-vs-malloc microbenchmark with (1) external allocator baselines
(jemalloc / tcmalloc) and (2) p99 (tail) percentile reporting alongside the existing
min/median/mean/max/stddev. This is the only substantive part of the spec review's §6.3
critique (#105) still standing after the ADR-0014
methodology was verified as already delivered.

Current state (already solid)

The harness is not the naive single-loop the original spec described — ADR-0014
already gives: warm-up (repeat 0 discarded), min/median/mean/max/stddev, anti-optimization
barriers, disclosed compiler flags + host, per-release reports under docs/bench/, a
non-asserting CI smoke, and a contended concurrent scenario (MUTEX vs LOCKFREE). So this
is an enhancement, not a fix.

What's missing

  • Tail latency (p99). median is effectively p50; there is no p99/p999 column. For an
    allocator, tail behaviour (e.g. a growth event on a dynamic pool) is exactly what a
    latency-sensitive consumer cares about. Adding p99 means switching the inner loop from
    aggregate-time-per-repeat to a per-operation timing sample set (or an HdrHistogram-style
    bucketed recorder) so percentiles are meaningful.
  • Modern baselines. Today the only baseline is the system malloc. jemalloc / tcmalloc
    are the allocators a reviewer will compare against; without them the headline ratios are
    only vs glibc/MSVC malloc. Baselines must be optional (built only when the dep is
    present) so the default build keeps zero external dependencies (spec §3.3).

Design notes / decisions to record in the ADR update

  • Keep the ADR-0014 TSV column contract stable — add columns, don't reshape existing rows
    (ADR-0014 §6 was written to absorb new rows/columns).
  • Per-op sampling has measurement overhead; decide whether p99 is a separate --percentiles
    run (higher-fidelity, slower) vs. always-on, to avoid perturbing the existing ns/op numbers.
  • jemalloc/tcmalloc as optional baselines behind a CMake feature-detect; never a hard dep.
    CI can add a cell where the allocator is available (Linux) without touching the MSVC leg.
  • Update ADR-0014 (or a superseding note) to record the percentile method and the
    baseline-selection policy.

Scope & compatibility

  • Benchmark/tooling only — no product-code or ABI change. SemVer-neutral. Per ADR-0004.

Acceptance criteria

  • Benchmark emits a p99 column (method documented in the ADR update).
  • jemalloc and/or tcmalloc baselines build optionally (feature-detected); default
    build stays dependency-free.
  • ADR-0014 updated (or superseded) to record the percentile method + baseline policy.
  • docs/bench/ report format and the bench README reflect the new columns/baselines.
  • CI smoke still runs and stays non-asserting.

References

Metadata

Metadata

Labels

enhancementNew feature or requestperfPerformance improvement (Conventional Commit: perf)

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions