Summary
Extend the pool-vs-malloc microbenchmark with (1) external allocator baselines
(jemalloc / tcmalloc) and (2) p99 (tail) percentile reporting alongside the existing
min/median/mean/max/stddev. This is the only substantive part of the spec review's §6.3
critique (#105) still standing after the ADR-0014
methodology was verified as already delivered.
Current state (already solid)
The harness is not the naive single-loop the original spec described — ADR-0014
already gives: warm-up (repeat 0 discarded), min/median/mean/max/stddev, anti-optimization
barriers, disclosed compiler flags + host, per-release reports under docs/bench/, a
non-asserting CI smoke, and a contended concurrent scenario (MUTEX vs LOCKFREE). So this
is an enhancement, not a fix.
What's missing
- Tail latency (p99).
median is effectively p50; there is no p99/p999 column. For an
allocator, tail behaviour (e.g. a growth event on a dynamic pool) is exactly what a
latency-sensitive consumer cares about. Adding p99 means switching the inner loop from
aggregate-time-per-repeat to a per-operation timing sample set (or an HdrHistogram-style
bucketed recorder) so percentiles are meaningful.
- Modern baselines. Today the only baseline is the system
malloc. jemalloc / tcmalloc
are the allocators a reviewer will compare against; without them the headline ratios are
only vs glibc/MSVC malloc. Baselines must be optional (built only when the dep is
present) so the default build keeps zero external dependencies (spec §3.3).
Design notes / decisions to record in the ADR update
- Keep the ADR-0014 TSV column contract stable — add columns, don't reshape existing rows
(ADR-0014 §6 was written to absorb new rows/columns).
- Per-op sampling has measurement overhead; decide whether p99 is a separate
--percentiles
run (higher-fidelity, slower) vs. always-on, to avoid perturbing the existing ns/op numbers.
- jemalloc/tcmalloc as optional baselines behind a CMake feature-detect; never a hard dep.
CI can add a cell where the allocator is available (Linux) without touching the MSVC leg.
- Update ADR-0014 (or a superseding note) to record the percentile method and the
baseline-selection policy.
Scope & compatibility
- Benchmark/tooling only — no product-code or ABI change. SemVer-neutral. Per ADR-0004.
Acceptance criteria
References
Summary
Extend the pool-vs-malloc microbenchmark with (1) external allocator baselines
(jemalloc / tcmalloc) and (2) p99 (tail) percentile reporting alongside the existing
min/median/mean/max/stddev. This is the only substantive part of the spec review's §6.3
critique (#105) still standing after the ADR-0014
methodology was verified as already delivered.
Current state (already solid)
The harness is not the naive single-loop the original spec described — ADR-0014
already gives: warm-up (repeat 0 discarded), min/median/mean/max/stddev, anti-optimization
barriers, disclosed compiler flags + host, per-release reports under
docs/bench/, anon-asserting CI smoke, and a contended concurrent scenario (
MUTEXvsLOCKFREE). So thisis an enhancement, not a fix.
What's missing
medianis effectively p50; there is no p99/p999 column. For anallocator, tail behaviour (e.g. a growth event on a dynamic pool) is exactly what a
latency-sensitive consumer cares about. Adding p99 means switching the inner loop from
aggregate-time-per-repeat to a per-operation timing sample set (or an HdrHistogram-style
bucketed recorder) so percentiles are meaningful.
malloc. jemalloc / tcmallocare the allocators a reviewer will compare against; without them the headline ratios are
only vs glibc/MSVC malloc. Baselines must be optional (built only when the dep is
present) so the default build keeps zero external dependencies (spec §3.3).
Design notes / decisions to record in the ADR update
(ADR-0014 §6 was written to absorb new rows/columns).
--percentilesrun (higher-fidelity, slower) vs. always-on, to avoid perturbing the existing ns/op numbers.
CI can add a cell where the allocator is available (Linux) without touching the MSVC leg.
baseline-selection policy.
Scope & compatibility
Acceptance criteria
build stays dependency-free.
docs/bench/report format and the bench README reflect the new columns/baselines.References
docs/specs/01_spec_cpp_memory_pool.md§7.1 (deferred list) via docs(spec): reconcile the memory-pool spec with the as-built system #110