Skip to content

feat(aiperf): make AIPerf the primary benchmark path#12

Open
thangquang09 wants to merge 30 commits into
mainfrom
pr/aiperf-main-benchmark
Open

feat(aiperf): make AIPerf the primary benchmark path#12
thangquang09 wants to merge 30 commits into
mainfrom
pr/aiperf-main-benchmark

Conversation

@thangquang09

@thangquang09 thangquang09 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Integrate AIPerf as the primary benchmark client path for agentic replay workloads.
  • Add coding benchmark scenario references for MiniMax and GLM smoke runs.
  • Cover both hackathon trace datasets and public InferenceX/Weka datasets.

Squash commit

  • eb6b1388 feat(aiperf): make AIPerf primary benchmark path

Testing

  • git diff --check origin/main..HEAD
  • uv run --with pytest --with pydantic --with pyyaml python -m pytest utils/matrix_logic/ utils/bench_serving/test_aiperf_adapter.py -v
    • 197 passed

Ngô Quang Hòa and others added 30 commits May 22, 2026 13:25
@claude
Add 1x H100 GreenNode runner type and gpt-oss-20b smoke config
c80256b 
- runners.yaml: register `h100-1x` runner type with our GreenNode H100 node
- nvidia-master.yaml: add `gptoss-fp4-h100-1x-vllm` (openai/gpt-oss-20b at FP4,
  TP=1, conc 4-16) — sized to fit a single 80GB H100, used as a smoke test
- runners/launch_h100-greennode.sh: plain-Docker launch script for
  single-VM GPU boxes (existing scripts assume Slurm + enroot, which we
  don't have on GreenNode)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Point submodules at vngcloud forks
9a282b3 
callanjfox/kv-cache-tester → vngcloud/kv-cache-tester
cquil11/aiperf → vngcloud/aiperf

Lets a single fine-grained PAT scoped to vngcloud org authorize both
the main repo and submodule clones during actions/checkout, without
needing a two-step anonymous-checkout pattern. Pinned commit SHAs
already exist in the forks, so no submodule re-pin required.
Add RTX 5090 GreenNode runner + DeepSeek-R1-Qwen3-8B BF16 smoke config
57db195 
- runners.yaml: register rtx5090-1x runner type pointing at our 5090 node
- nvidia-master.yaml: dsr1qwen3-bf16-rtx5090-1x-vllm config
  (deepseek-ai/DeepSeek-R1-0528-Qwen3-8B, BF16, TP=1, conc 4-16)
- runners/launch_rtx5090-greennode.sh: plain-Docker launcher
- benchmarks/single_node/dsr1qwen3_bf16_rtx5090.sh: vLLM serve + bench
  script for the 8B distill on a single 5090
Add 8k1k workload, FP8 5090 variant, and agentic-coding scenarios
2a7cbdb 
- nvidia-master.yaml: 8k1k fixed-seq-len + agentic-coding scenarios
  added to gptoss-fp4-h100-1x and both dsr1qwen3-rtx5090 configs.
  8k1k single-node entries auto-trigger lm-eval at median/highest conc
  via InferenceX's mark_eval_entries logic — so this also enables eval.
- New dsr1qwen3-fp8-rtx5090-1x-vllm config (vLLM on-the-fly FP8 quant).
- Agentic scripts for dsr1qwen3 on RTX 5090 (bf16 + fp8 variants).
  TORCH_CUDA_ARCH_LIST=12.0 for sm_120 Blackwell consumer.
- dsr1qwen3_bf16_rtx5090.sh: MAX_MODEL_LEN now respects the env var
  (${MAX_MODEL_LEN:-9472}) so 8k1k fits.

Matrix: 30 entries (gptoss h100-1x: 10, dsr1qwen3-bf16 5090: 10,
dsr1qwen3-fp8 5090: 10 — each = 6 fixed-seq-len + 4 agentic).
Add 2x H100 runner type and gptoss-fp4-h100-2x-vllm config
517f706 
Mirrors the 1x config (same gpt-oss-20b model, image, scenarios) but
TP=2 on h100-greennode_01. Gives direct apples-to-apples scaling
comparison: 1x H100 (TP=1) vs 2x H100 (TP=2) on identical workload.
Drop stray trailing 'c' typo in launch_h100-greennode.sh
8a0f026 
The single character at EOF made the script exit 127 after every
successful gpt-oss benchmark, so every gptoss-fp4-h100-1x/-2x job in
run 26285383625 ran to completion (full TTFT/TPOT emitted) then was
marked failed. dsr1qwen3 jobs on rtx5090 were unaffected (different
launch script).
Add gemma4-fp8-h100-2x-vllm-mtp lab config for MEP-0006
e3b63ce 
Native Gemma 4 MTP via vLLM v1 --speculative-config (PR
vllm-project/vllm#41745, merged 2026-05-08). Targets the 2x H100
GreenNode box (h100-greennode_01), uses google/gemma-4-31B-it
target + google/gemma-4-31B-it-assistant drafter (Q-only decoder
sharing target's KV cache, ~few hundred MB extra weights, no
separate process).

Image pinned to v0.22.0 as the first vLLM release post-PR.
Drafter HF slug is the presumed 31B name following the
E2B/E4B/26B pattern from the PR test plan — may need adjusting
if Google's actual upload differs.

Sweep: 1k1k + 8k1k * conc {4,8,16}, all spec-decoding: mtp.
Comparable to the MEP-0002 prior-art Gemma 4 baseline.
Fix gemma4 MTP image tag: v0.22.0 → v0.21.0
9c1a53a 
v0.22.0 doesn't exist on Docker Hub. v0.21.0 (tagged 2026-05-15) is
the first stable release after PR #41745 was merged (2026-05-06), so
it contains the Gemma 4 MTP code path. After v0.21.0 vLLM only ships
nightly builds.
Disable agentic-coding scenarios pending issue #2
4c6a430 
aiperf SystemController times out at 'Configure Profiling' on all
GreenNode runners. Commented out the agentic-coding blocks for:
  gptoss-fp4-h100-1x-vllm
  gptoss-fp4-h100-2x-vllm
  dsr1qwen3-bf16-rtx5090-1x-vllm
  dsr1qwen3-fp8-rtx5090-1x-vllm
Restore once #2 is resolved. The fixed-seq-len scenarios still run.
Gemma 4 MTP bench: bump max-num-batched-tokens 2048 → 8192
b0cb3e2 
Gemma 4 is multimodal; vLLM startup fails with:
  ValueError: max_tokens_per_mm_item (2496) is larger than
              max_num_batched_tokens (2048)
because the vision encoder emits 2496 tokens per image and chunked
MM input is disabled by default. 8192 covers MM items and also
matches our 8k1k workload upper bound.
Gemma4 MTP bench: combine baseline + MTP in one config (Approach A)
b195592 
Rename gemma4-fp8-h100-2x-vllm-mtp → gemma4-fp8-h100-2x-vllm-bench
and add 'spec-decoding: none' rows alongside the existing
'spec-decoding: mtp' rows. Matrix now emits 12 entries (6 mtp + 6
none) per dispatch, so MTP-vs-baseline rows share a wall-clock
window — no between-run drift in HF mirror speed, runner thermal,
or vLLM patch level.

Also fix unrelated YAML typo on line 17 (prefill:wr → prefill:)
that was blocking matrix validation.
Gemma4 MTP: add N=4 / N=6 sweep configs
e251de4 
Validator forbids extra fields on single-node search-space entries,
so num_speculative_tokens can't ride the matrix entry directly.
Workaround: distinct model-prefix per N, the script derives N from
$MODEL_PREFIX. Script filename variants (gemma4n4_fp8_h100.sh,
gemma4n6_fp8_h100.sh) symlink to gemma4_fp8_h100.sh.

Also drop the broken /workspace/spec_metrics_*.txt curl — server.log
(already uploaded by the workflow's Upload server logs step) already
captures vLLM's 'SpecDecoding metrics:' lines every ~10s during
inference, so the curl was redundant. It was also broken: workspace
gets wiped by next job's actions/checkout 'clean: true', so only the
last job's file survived (empty).
@claude
Promote num-speculative-tokens + max-num-batched-tokens to first-clas…
14a42a0 
…s search-space fields

Both knobs were previously inexpressible in the matrix schema:
- N was smuggled into model-prefix (gemma4n4/n6) and decoded by a case
  statement in gemma4_fp8_h100.sh — this polluted infmax_model_prefix in
  agg_bmk.json with fake "models" and forced any consumer to string-parse
  the suffix to recover N.
- max-num-batched-tokens was hardcoded to 8192 in the bench script.

Adds them as typed Optional[int] fields on SingleNodeSearchSpaceEntry +
SingleNodeMatrixEntry, plumbs through generate_sweep_configs.py, the
workflow templates (benchmark-tmpl.yml inputs + env + RESULT_FILENAME
slug so concurrent jobs at different knob values don't collide),
launch_h100-greennode.sh RUN_ENV, the bench script, and
process_result.py (emits flat columns in agg_bmk.json) +
compare_results.py (DB lookup key).

Collapses gemma4-fp8-h100-2x-vllm-bench-{n4,n6} back into the single
gemma4-fp8-h100-2x-vllm-bench key with N=2/4/6 as search-space rows,
and deletes the n4/n6 script symlinks. Adds new
gemma4-fp8-h100-1x-vllm-bench (TP=1, mnbt sweep 4k/8k/16k) as the
first config exercising the new max-num-batched-tokens knob.

Dashboard handoff: agg_bmk.json rows gain two optional int columns
(num_speculative_tokens, max_num_batched_tokens). RESULT_FILENAME slug
gains _n<value>_mnbt<value> segments. compare_results.build_config_params
passes both fields as DB lookup params for the comparator to consume
once the schema is migrated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude
benchmark_lib.sh: persist HF_TOKEN to disk so vLLM workers see it
be8bd1b 
vLLM's APIServer / EngineCore are launched via multiprocessing.spawn,
which starts fresh Python interpreters. Those re-import huggingface_hub
and call get_token(); if HF Hub was imported before the env var was
read on a given code path, the worker emits
  "You are sending unauthenticated requests to the HF Hub ..."
even though the parent shell has HF_TOKEN set (confirmed via the
`docker run -e HF_TOKEN` plumbing in launch_*.sh + benchmark-tmpl.yml).

Writing the token to ~/.cache/huggingface/token sidesteps env-var
propagation entirely — every HF Hub code path falls back to that file.
Also exports HUGGING_FACE_HUB_TOKEN for legacy library paths.

Auto-invoked at source time so every bench script that does
  source "$(dirname "$0")/../benchmark_lib.sh"
gets the fix without having to add a call. No-op when HF_TOKEN is
unset (local smoke runs without GHA secrets).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude
Add qwen3.5-27b-fp8-h100-1x-vllm-bench lab config
d5e5dc1 
Qwen3.5-27B (dense) FP8 on the single H100 GreenNode box (h100-1x ->
h100-greennode_00), vLLM v0.21.0. Mirrors the gemma4-fp8-h100-1x-vllm-bench
sweep: isl {1k,8k} x osl 1k x conc {4,8,16}, sweeping max-num-batched-tokens
4k/8k/16k (18 jobs total).

Distinct model-prefix (qwen3.5-27b) so the inferencex.com dashboard does not
conflate this dense model with the existing qwen3.5-397B-A17B MoE configs.

HF slug Qwen/Qwen3.5-27B-FP8 is the presumed name following the
<repo>-<size>-FP8 pattern; confirm against Qwen's actual upload before dispatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add vLLM launch script for qwen3.5-27b-fp8-h100-1x-vllm-bench
28094aa 
The config qwen3.5-27b-fp8-h100-1x-vllm-bench (d5e5dc1) had no matching
launch script, so the runner died at "Launch job script" with exit 127
(benchmarks/single_node/qwen3.5-27b_fp8_h100.sh: not found).

Add it as a vLLM single-H100 script modeled on gemma4_fp8_h100.sh:
- vLLM auto-detects fp8 from the Qwen/Qwen3.5-27B-FP8 checkpoint's
  quantization_config (native fp8, dynamic activation scheme), so no
  --quantization flag; --dtype bfloat16 sets the compute dtype.
- Serve flags per the production config: --reasoning-parser qwen3,
  --enable-auto-tool-choice, --tool-call-parser qwen3_xml,
  --gpu-memory-utilization 0.90, --trust-remote-code, TP=1.
- max-model-len + max-num-batched-tokens threaded from the matrix via
  config.yaml so the mnbt 4k/8k/16k sweep works; --max-num-seqs=$CONC.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add gemma4-fp8-h100-2x-sglang baseline config (engine comparison)
92c5d47 
SGLang counterpart to gemma4-fp8-h100-2x-vllm-bench so Gemma 4 31B can be
compared vLLM-vs-SGLang on the same 2x H100 / TP=2 baseline (spec-none).
No SGLang gemma4 config existed before; gemma4 had only ever run on vLLM.

Three pieces:
- runners/launch_h100-greennode.sh: prefer a framework-tagged single_node
  script (<prefix>_<prec>_h100_<framework>.sh) with a fallback to the
  historical engine-less name. Lets two engines coexist for one model on
  this box (b200/b300 launchers already do this). NO spec suffix is added:
  scripts like gemma4_fp8_h100.sh branch internally on $SPEC_DECODING, and
  no *_h100*_vllm.sh scripts exist, so every current vLLM config (gemma4,
  qwen3.5-27b, gptoss) keeps falling back to its legacy name unchanged.
- benchmarks/single_node/gemma4_fp8_h100_sglang.sh: SGLang launch, on-the-fly
  fp8 (--quantization fp8) + fp8 KV to mirror the vLLM bench, TP from matrix.
- gemma4-fp8-h100-2x-sglang config: 6 jobs (isl{1k,8k} x conc{4,8,16}).

MTP omitted on purpose — SGLang's gemma4 MTP path differs from vLLM's
--speculative-config drafter; this is baseline-vs-baseline. First dispatch
will also confirm whether SGLang v0.5.12 supports the Gemma 4 arch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add bench-config skill documenting the benchmark config flow
77218c2 
Documents the 7-layer path from a config entry in
.github/configs/*-master.yaml through generate_sweep_configs.py,
e2e-tests.yml, benchmark-tmpl.yml, runners/launch_<family>.sh, to the
benchmarks/single_node/*.sh launch script and the dashboard.

Captures the edit recipes for the common changes (engine, sweep params,
card count, new model), the derived launch-script naming rule, the two
silent-failure rules (a new config needs a matching launch script or it
dies at "Launch job script" with exit 127; runner != card count), the
HF-slug/arch pre-checks, local validation commands, the vngcloud dispatch
command, and run-watching/debugging tips.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
gemma4 SGLang: use pre-quantized fp8 checkpoint to skip vision tower
95c0e16 
First SGLang dispatch confirmed SGLang v0.5.12 DOES support Gemma 4
(gemma4_mm.py/gemma4_vision.py load, server starts), but on-the-fly
--quantization fp8 crashes in the vision tower: gemma4_vision.py forward
-> fp8 apply -> triton_scaled_mm assert (scale_b.shape mismatch), so the
scheduler dies and the server never becomes healthy.

Fix: serve the pre-quantized RedHatAI/gemma-4-31B-it-FP8-dynamic
checkpoint (compressed-tensors, ignore=['re:.*vision.*','lm_head',
're:.*embed_tokens.*']) and drop --quantization. SGLang auto-detects
compressed-tensors and honours the ignore list, keeping the vision
encoder in bf16 while running the LLM in fp8 — exactly "fp8 LLM, vision
unquantized". Slightly different fp8 recipe than the vLLM bench's
on-the-fly fp8 (noted in the config comment).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
bench-config skill: add SGLang multimodal fp8 vision-tower gotcha
9acb32a 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
bench-config skill: trim to a minimal cheat-sheet
ba18e1f 
Cut the prose walkthrough, HF curl recipes, verbose flow diagram and Notes
section; keep the load-bearing facts (flow, field reference, derived
script-name rule, edit table, the two silent-failure rules, validate +
dispatch commands, engine gotchas). ~196 -> ~60 lines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add qwen3.5-27b-fp8-h100-1x-sglang (SGLang counterpart to the vLLM be…
a459668 
…nch)

SGLang baseline for Qwen3.5-27B on a single H100, mirroring the vLLM
qwen3.5-27b-fp8-h100-1x-vllm-bench on the SAME checkpoint
(Qwen/Qwen3.5-27B-FP8) for an engine-to-engine comparison. 6 jobs
(isl {1k,8k} x conc {4,8,16}); no max-num-batched-tokens sweep since
that's a vLLM knob (SGLang uses --chunked-prefill-size, fixed 8192).

Launch script qwen3.5-27b_fp8_h100_sglang.sh adapted from the 397B-A17B
qwen3.5_fp8_h100.sh: dense single-GPU (dropped expert-parallel-size and
the multi-GPU flashinfer allreduce fusion), and — crucially — NO
--quantization. Qwen/Qwen3.5-27B-FP8 is a pre-quantized block-fp8
checkpoint whose quantization_config.modules_to_not_convert already
excludes the whole vision tower (model.visual.*), lm_head, embeddings and
the linear-attn conv1d/in_proj layers; SGLang auto-detects that and keeps
the vision encoder in bf16. Forcing on-the-fly fp8 would quantize the
vision tower and crash in triton_scaled_mm (the gemma4 failure mode).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
bench-config skill: document dashboard publishing (auto-ingest + forc…
37c903b 
…e-ingest)

Dashboard ingests only runs you mark ready. Document the two paths:
[ingest] commit-prefix convention for the 15-min auto-scan (with the
current startswith caveat), and the force-ingest workflow_dispatch by
run id that works today.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
bench-config skill: warn to force-ingest one run at a time
5aa23cd 
The auto-ingest concurrency group (cancel-in-progress: false) keeps only
one running + one pending, so dispatching several run_urls at once gets
the middle ones cancelled. Dispatch sequentially.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
bench-config skill: note new model-prefix must be declared in dashboa…
7d87ea6 
…rd or rows skip

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Run sweep on dev branch too
cf8dbc6 
Add dev to the push and pull_request branch filters in run-sweep.yml so
the perf sweep fires on dev as well as main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@aistackdev
Merge pull request #5 from vngcloud/chore/run-sweep-on-dev
e390a6f 
Run sweep on dev branch too
@thangquang09 @claude
Merge AIPerf Integration (#11)
f0072cc 
* Add AIPerf benchmark client integration

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: route gemma4 h100 sweep to 2x runner

* test: narrow gemma4 h100 sweep

* test: benchmark gemma4 with aiperf only

* docs: add aiperf integration guide

* chore: remove local h100 launcher docs

* Trigger Run Sweep when a PR is opened

Add the 'opened' pull_request type so the sweep fires as soon as a
non-draft PR is opened (still gated by the perf-changelog.yaml path
filter), not only on ready_for_review/synchronize/label.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: AIPerf agentic-replay for 3 trace datasets (+ optional --tokenizer) (#9)

## What

Integrates the **agentic-replay** scenario type into the InferenceX AIPerf path and wires up three trace datasets, so a recorded `mooncake_trace` JSONL can be replayed once (closed-loop, with think-time) or driven back-to-back as a capacity sweep — all through official AIPerf v0.9.0.

## Datasets added (`benchmarks/single_node/agentic/datasets/`)
- `minimax_claude_code_prod_v3.jsonl` — Claude-Code MiniMax production trace (think-time)
- `agentic_coding_1variant_64k_150s.jsonl` — synthetic agentic-coding 64k tier (think-time)
- `gemma_blend_prod.jsonl` — Gemma blend_prod, single-turn back-to-back (`strip-trace-delays`)

## Plumbing
- **Adapter** (`utils/bench_serving/aiperf_adapter.py`): duration-mode stop condition, `--no-fixed-schedule`, `--inter-turn-delay-cap-seconds`, `--dataset-sampling-strategy`, `--benchmark-grace-period`, goodput, and an optional **`--tokenizer`** passthrough (defaults to the served model when unset).
- **Matrix/validation** (`generate_sweep_configs.py`, `validation.py`): `agentic-replay` scenario + `tokenizer` field, both flowing through `e2e-tests.yml` → `benchmark-tmpl.yml` → launcher → adapter.
- **Launcher** (`qwen3-4b-2507_bf16_h100_vllm.sh`, `runners/launch_h100-greennode.sh`): trace-subset (`#N`), `STRIP_TRACE_DELAYS`, duration-based replay, canonical `REPLAY_ARGS`.
- **Configs**: three `*-smoke*` keys on the qwen3-4b-2507 path + matching `perf-changelog.yaml` entries.

## Docs / tests
- ADR-0002 (keep both agentic paths: think-time vs back-to-back) + MVP-vs-Mode1 note.
- `agentic-replay-run` Claude skill (runbook; inherits config mechanics from `bench-config`).
- Unit tests: adapter (tokenizer set/omit), validation, sweep-config, process_result — green.

## Validation
3 datasets validated live at conc=4 on Qwen3-4B (h100-greennode_00): prefix-cache hit MiniMax prod **51.5%**, 64k 1-variant **95.7%**, Gemma blend b2b **3.5%** — spanning the structural spectrum.

* feat(aiperf): add utils/aiperf-mooncake submodule (clean v0.9.0 fork)

Second aiperf submodule (utils/aiperf-mooncake) pinned to a clean v0.9.0
fork (thangquang09/aiperf) wired via AIPERF_SOURCE_DIR so the mooncake
agentic-replay path installs our fork instead of stock PyPI 0.9.0. The
weka path (utils/aiperf) is unaffected. See ADR-0003.

Also update the agentic-replay-run skill to require the AIPERF_SOURCE_DIR
export in the launch script for all three datasets, and CONTEXT.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(skill): add optional DCGM toggle to agentic-replay-run

Ask the user (Q6) whether to enable DCGM; when yes, the agent edits the
runner launch script to start a dcgm-exporter sidecar (--network host) so
AIPerf reaches localhost:9400/metrics for richer GPU telemetry. gpu_metrics.csv
unchanged. Includes the copy-paste bash block and first-run port-9400 check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(agentx-weka): Qwen3-4B vLLM launcher + agentx-weka-run skill

Adds the agentx-weka (inferencex-agentx-mvp / cc-traces-weka corpus) path
for vLLM on h100-greennode, validated by a 90s smoke at conc 2/4 (0 errors).

- benchmarks/single_node/agentic/qwen3-4b-weka_bf16_h100_vllm.sh: launcher
  that isolates aiperf in a clean venv so its anyio/starlette upgrade never
  reaches vLLM's system python (v0.21.0 crashes otherwise).
- benchmark_lib.sh: WEKA_NUM_DATASET_ENTRIES caps the 949-trace corpus
  (default 949, backward-compatible).
- nvidia-master.yaml: qwen3-4b-weka smoke config (no perf-changelog entry, so
  it does not auto-trigger a sweep; dispatch manually).
- .claude/skills/agentx-weka-run: skill documenting the path + the venv /
  ordering / trace-load / disk gotchas.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(aiperf): bump aiperf-mooncake submodule to include non-finite metrics fix

Bumps utils/aiperf-mooncake 524fac8b -> 1d2377f5, which merges a backport of
upstream ai-dynamo/aiperf#1025 onto the pinned benchtool/aiperf-0.9.0 line.

Without the fix, SGLang's NaN sglang:fwd_occupancy gauge (uninitialized before
the first forward pass) slips past the old `== float("inf")` filter, gets
orjson-encoded as JSON null, fails ServerMetricsRecordMessage validation, and
silently drops the ENTIRE scrape — taking cache_hit_rate / cached_tokens with
it. The agentic-replay path (/agentic-replay-run, mooncake_trace) scrapes
SGLang /metrics with --enable-metrics, so it was affected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(skill): agentic-replay-run smoke uses 2-request warmup

Default 20-request warmup runs before profiling and can eat a short
smoke window (observed ~750s on 31B/131072-ctx). For duration 90, set
--warmup-request-count to 2 in the launch script (not env-plumbed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [dataset] add minimax CC v4 weka-format dataset (234 traces, hackathon week)

Production-grounded agentic-coding dataset from vMonitor gateway logs:
- Date range: June 10-17, 2026 (hackathon week)
- Full capture: 260 sessions pulled, 234 traces (>=2 turns), 21,449 requests
- Keyed by x-claude-code-session-id (real CC conversations)
- Weka trace format: block_size=64, LCP-based hash_ids
- Compaction modeled via reset_context (>10% context change, 6.2% reset rate)
- Concurrent bursts linearized
- think_time stored RAW (no cap) — runtime capping via scenario config
- Latency: latencies.request (full wall-clock)
- Validated: pydantic WekaTrace 234/234 pass, cache 86.5%

Stats: p50=37 turns, ISL p50=63K, OSL p50=121, think_time p50=0.8s p90=47.6s

* feat(agentic): add minimax v4 weka smoke

* chore(runner+skill): add DCGM sidecar to h200-greennode launcher; update agentic-replay-run skill

- runners/launch_h200-greennode.sh: DCGM exporter sidecar always-on before docker run
- .claude/skills/agentic-replay-run/SKILL.md: centralize flow, DCGM default-on,
  flexible yyyy/mm/dd run-naming pattern

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* submodule+skill(aiperf-mooncake): switch to benchtool/agentx-weka; weka via aiperf-mooncake

utils/aiperf-mooncake now tracks benchtool/agentx-weka (forked from ajc/agentx
in the thangquang09 repo), which carries the weka_trace loader AND the
data_collector math.isfinite NaN filter. This lets weka_trace run through the
agentic-replay path on the same submodule as mooncake_trace, with no SGLang
runtime patch needed.

- .gitmodules: aiperf-mooncake branch benchtool/aiperf-0.9.0 -> benchtool/agentx-weka
- pointer: 1d2377f5 -> a75c4612
- agentic-replay-run skill: weka_trace AIPerf source utils/aiperf -> utils/aiperf-mooncake;
  fork-pin section + confirm-fork log note updated
- datasets README: same source update + rationale

Do not use utils/aiperf (vngcloud fork) for weka any more: its data_collector.py
uses '== float("inf")' which never catches NaN, so SGLang's fwd_occupancy=NaN
drops the entire /metrics scrape and would require the runtime patch
(patches/aiperf-skip-nonfinite-server-metrics.patch).

---------

Co-authored-by: Thắng. Lý Quang (5) <thanglq5@vng.com.vn>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ngô Quang Hòa <hoanq3@vng.com.vn>
Revert "Merge AIPerf Integration (#11)"
48a703a 
This reverts commit f0072cc.
feat(aiperf): make AIPerf primary benchmark path
eb6b138 
Integrate AIPerf as the main benchmark client path for agentic replay workloads.

Add coding benchmark scenario references for MiniMax and GLM on the hackathon and public InferenceX datasets.
@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@thangquang09 @aistackdev