Search results

[Feature]:[Question / DeepSeek-V4] KV cache FP8 layout for SWA / COMPRESS pools differs from the DeepSeek-V4 reference implementation (FlashMLA / SGLang) — any plan to align?

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#14327 In NVIDIA/TensorRT-LLM;

· inference666 opened on May 19, 2026

[Feature]: AutoDeploy: Integrate with CppMambaHybridCacheManager

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14320 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on May 19, 2026

[Feature]: AutoDeploy: MTP for Qwen3.5 MoE VLM

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14288 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on May 19, 2026

[Feature]: AutoDeploy: MTP for Qwen3.5 MoE Text Model

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14287 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on May 19, 2026

MLIR-fusion seems to corrupt CUDA graph replay in Gemma4 e2b

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

CUDA Graph

Status: Open.

#14272 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on May 18, 2026

[AutoDeploy]: Investigate SuperV3 with MTP not scaling well

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14225 In NVIDIA/TensorRT-LLM;

· galagam opened on May 17, 2026

[AutoDeploy]: Performant UltraV3 with MTP

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14223 In NVIDIA/TensorRT-LLM;

· galagam opened on May 17, 2026

[AutoDeploy]: Pre-merge L0 cost-reduction

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14173 In NVIDIA/TensorRT-LLM;

· galagam opened on May 15, 2026

nvfp4_gemm produces incorrect numerical results on SM120 (Blackwell consumer)

bugSomething isn't workingSomething isn't working

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

waiting for feedback

Status: Open.

#14154 In NVIDIA/TensorRT-LLM;

· QilinWan opened on May 14, 2026

AutoDeploy Grafia Integration PoC

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Status: Open.

#14153 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on May 14, 2026

[Bug]: [CuteDSL MoE] generate_token_selected_experts: ghost-token assignment to zero-allocation experts + dead-on-multi-rank prioritization

bugSomething isn't workingSomething isn't working

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Status: Open.

#14146 In NVIDIA/TensorRT-LLM;

· leejnau opened on May 14, 2026

[AutoDeploy]: Add test to gate against memory leak regressions

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14141 In NVIDIA/TensorRT-LLM;

· galagam opened on May 14, 2026