[unitest] small change in test_deepgemm_precision.py#7834
[unitest] small change in test_deepgemm_precision.py#7834zhoutianzi666 wants to merge 15 commits into
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #7834 +/- ##
==========================================
Coverage ? 63.33%
==========================================
Files ? 462
Lines ? 64372
Branches ? 9872
==========================================
Hits ? 40768
Misses ? 20836
Partials ? 2768
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览❌ 存在 1 个 Required 失败任务，需处理后方可合并。
2 任务状态汇总2.1 Required任务 : 7/8 通过
2.2 可选任务 — 18/21 通过
3 失败详情（仅 required）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — 覆盖率不达标（置信度: 高）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage
失败用例:
根因详情: 关键日志: 修复建议:
修复建议摘要: 为 关联变更: |
zhoutianzi666
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览❌ 存在 1 个 Required 失败任务需要处理，其余 Required 任务已通过。
2 任务状态汇总2.1 Required任务 : 9/10 通过
2.2 可选任务 — 29/32 通过
3 失败详情（仅 required）Pre Commit — 代码规范（置信度: 高）Pre Commit
根因详情: 关键日志: 修复建议:
修复建议摘要: 删除或重命名 test_flashmla_precision.py:60 未使用变量 关联变更: PR 修改文件： 链接: 查看日志 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览
2 任务状态汇总2.1 Required 任务：0/0 通过
2.2 可选任务 — 6/9 通过
3 失败详情（仅 required）无 required 失败任务。 可选任务失败说明Run iluvatar Tests / run_iluvatar_cases
Check PR Template
CI_HPU
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
we |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-05-19 19:53:47
📋 Review 摘要
PR 概述：重构 MLA KV Cache 写入逻辑，引入 slot_mapping 替换 max_seq_len 参数，并统一 forward_mixed 中 prefill/decode 路径的缓存写入调用；同时优化算子测试精度计时方式。
变更范围：custom_ops/gpu_ops/append_attn/、fastdeploy/model_executor/layers/attention/mla_attention_backend.py、tests/operators/
影响面 Tag：[OP] [KVCache]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🔴 Bug | custom_ops/gpu_ops/append_attn/mla_cache_kernel.cuh:214 |
调试代码残留：printf + asm volatile("trap;") 在生产环境触发会终止整个 CUDA context |
| ❓ 疑问 | fastdeploy/model_executor/layers/attention/mla_attention_backend.py:846 |
decode 分支移除 decode_mla_write_cache，统一用 prefill_mla_write_cache，但 decode token（batch_id=-1）的实际写缓存语义需确认 |
📝 PR 规范检查
PR 标题缺少官方 Tag，描述所有 section（Motivation / Modifications / Usage or Command / Accuracy Tests / Checklist）均为空。
标题建议（可直接复制）：
[OP][KVCache] Refactor MLA write cache to use slot_mapping and unify prefill/decode path in forward_mixed
PR 描述建议（可直接复制，必须复刻 checklist §D2 模板的完整结构）：
## Motivation
重构 MLA KV Cache 写入接口，以 `slot_mapping` 替换 `max_seq_len` 参数，消除历史冗余参数；在 `forward_mixed` 中统一 prefill 与 decode token 的 KV Cache 写入路径，减少代码重复。同时优化 `test_deepgemm_precision.py` 和 `test_flashmla_precision.py` 中的计时方式，改用 CUDA Event 实现更精确的性能测量。
## Modifications
- `custom_ops/gpu_ops/append_attn/mla_cache_kernel.cuh`：`prefill_absorb_cache_kernel` 新增 `slot_mapping` 参数与 CT 模板参数；将 `batch_id_per_token` 类型从 `uint32_t` 改为 `int32_t` 以支持 -1 标记；新增 `ori_bi == -1` 跳过逻辑
- `custom_ops/gpu_ops/append_attn/mla_cache_kernel.cu`：`PrefillMLAWriteCache` / `PrefillMLAWriteCacheKernel` 新增 `slot_mapping` 参数，移除 `max_seq_len`；`DecodeMLAWriteCache` / `DecodeMLAWriteCacheKernel` 移除 `max_seq_len`
- `custom_ops/gpu_ops/cpp_extensions.cc`：同步更新函数签名声明
- `fastdeploy/model_executor/layers/attention/mla_attention_backend.py`：`forward_extend` / `forward_decode` / `forward_mixed` 同步更新调用签名；`forward_mixed` 中将缓存写入提升至分支外，统一处理 prefill + decode token；`mla_blackwell` / `flashmla_baseline` 新增 float8 类型支持
- `tests/operators/test_deepgemm_precision.py`：改用 CUDA Event 精确计时，新增 TFLOPs/s 输出，扩展测试用例
- `tests/operators/test_flashmla_precision.py`：改用 CUDA Event 精确计时，新增带宽输出，支持 float8 dtype
## Usage or Command
N/A
## Accuracy Tests
N/A
## Checklist
- [ ] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [x] Format your code, run `pre-commit` before commit.
- [x] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.总体评价
本 PR 完成了 MLA KV Cache 写入接口的有效重构，引入 slot_mapping 并统一了 mixed batch 的处理路径，逻辑方向正确。但 CUDA kernel 中残留了 asm volatile("trap;") 调试断言，在生产环境触发将导致整个 GPU Context 崩溃，属于 P0 阻塞问题，需在合入前删除；同时 decode token 在新路径下的缓存写入语义需作者进一步确认。
| const uint32_t block_offset = ori_seq_id % block_size; | ||
|
|
||
| const int32_t block_idx1 = slot_mapping[token_idx] / block_size; | ||
| if (block_idx1 != block_idx) { |
There was a problem hiding this comment.
🔴 Bug CUDA kernel 中残留调试代码：printf + asm volatile("trap;") 在生产环境会导致 GPU 崩溃。
asm volatile("trap;") 相当于 GPU 上的 abort()，一旦触发将终止整个 CUDA context，导致服务不可用。这段代码明显是用于对齐验证 slot_mapping 与 block_tables 两种寻址路径是否一致的临时调试代码，不应合入主干。
建议修复：
- 验证通过后，直接删除整个 block（第 213-219 行）
- 如需保留作为 debug 模式开关，应改为
#ifdef DEBUG_MLA_CACHE ... #endif
| @@ -845,20 +845,6 @@ def forward_mixed( | |||
|
|
|||
| # Decode branch: k is None | |||
There was a problem hiding this comment.
❓ 疑问 forward_mixed 中 decode 分支移除了 decode_mla_write_cache 调用，但 decode token 的 KV 写缓存通过提升到分支前的 prefill_mla_write_cache 统一处理。
请确认：prefill_mla_write_cache 在 batch_id_per_token[token_idx] == -1（decode token 在 mixed batch 中被标记为 -1）时通过 if (ori_bi == -1) continue; 正确跳过，但 decode token 的实际写缓存是否仍然有效？原 decode_mla_write_cache 使用 seq_lens_decoder/seq_lens_encoder 参数顺序，而新代码使用 seq_lens_this_time/seq_lens_decoder，语义是否一致？
5 participants
Motivation
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.