Enable KV Cache Support for Scanned Decoder Layers and Improve vLLM Integration

[khatwanimohit]

Description

This PR enables passing KV caches through scanned decoder layers, which is essential for efficient decoding when layer scanning is enabled (a common optimization for large models). It also refines the vLLM adapter and attention layers to better handle sharding and JIT tracing.

If the change fixes a bug or a Github issue, please include a link, e.g.,:
FIXES: b/123456
FIXES: #123456

Notice 1: Once all tests pass, the "pull ready" label will automatically be assigned.
This label is used for administrative purposes. Please do not add it manually.

Notice 2: For external contributions, our settings currently require an approval from a MaxText maintainer to trigger CI tests.

Tests

Requires tpu-inference on branch: mohit/omit_jit

Tested Qwen3-30B using vllm serve on a v5p-8

VLLM_ENABLE_V1_MULTIPROCESSING=0 NEW_MODEL_DESIGN=1 TPU_BACKEND_TYPE=jax vllm serve Qwen/Qwen3-30B-A3B \
  --seed 42 \
  --max-model-len 2048 \
  --no-enable-prefix-caching \
  --disable-log-requests \
  --max-num-batched-tokens 8192 \
  --async-scheduling \
  --tensor_parallel_size 1 \
  --gpu-memory-utilization 0.3 \
  --max-num-seqs 256 \
  --hf_overrides '{"architectures": ["MaxTextForCausalLM"]}' \
  --additional-config='{"sharding":{"sharding_strategy": {"enable_dp_attention": true, "expert_parallelism": 4}}, "maxtext_config": {"model_name": "qwen3-30b-a3b", "scan_layers": true, "log_config":false, "load_parameters_path": "gs://maxtext-model-checkpoints/qwen3-30b/2025-11-11/pathways/scanned/0/items"}}'

Received a response

{
"id": "cmpl-9f0419af227ee518",
"object": "text_completion",
"created": 1773193183,
"model": "Qwen/Qwen3-30B-A3B",
"choices": [
  {
    "index": 0,
    "text": " city in the U.S. state of Washington, and the county seat of King",
    "logprobs": null,
    "finish_reason": "length",
    "stop_reason": null,
    "token_ids": null,
    "prompt_logprobs": null,
    "prompt_token_ids": null
  }
],
"usage": {
  "prompt_tokens": 3,
  "total_tokens": 19,
  "completion_tokens": 16,
  "prompt_tokens_details": null
},
"kv_transfer_params": null
}

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[NicoGrande]

Thank you for doing this @khatwanimohit !!

CC @ChingTsai and @chishuen who were interested in this feature.

[codecov]

Codecov Report

❌ Patch coverage is 64.28571% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/maxtext/layers/decoders.py	63.15%	5 Missing and 2 partials ⚠️
src/maxtext/layers/attentions.py	0.00%	2 Missing ⚠️
src/maxtext/models/qwen3.py	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!