vLLM v0.30.0

What it claims

vLLM v0.30.0 (762 commits, 315 contributors) is an upgrade that changes behaviour for existing deployments, not only for new models. An engineer upgrading needs to know about the breaking changes, several silently flipped kernel defaults, and a batch of security fixes. The release also reports concrete startup and kernel speedups.

Evidence

The figures below come from individual PRs. The page gives only the setting listed and compares against the prior code path, not a full benchmark setup.

Startup and RL

• Freezing gc during CUDA graph capture cut capture time from 12s to 2s and engine init from 28.9s to 8.2s on H200 (Model Runner V2).
• GPU-compacted --return-sampling-mask fixes an RL step-time regression of about 2x.

Model-specific

• Kimi K3:
• Removing mixed-batch KDA gather/scatter gives 5.2–7.7% E2E throughput.
• Grouped FP8 MLA cache insertion is a 4–6x kernel speedup at small batch.
• The strided DSV3 low-latency GEMM is 12–81% faster at the kernel level.
• GLM-5.3-Flash: FlashKDA chunked prefill is 1.7–3.8x faster than the Triton chunk path.
• Gemma 4 multimodal: pruned sliding-window tiles give up to 3–4x E2E.
• Nemotron latent-MoE: skipping a redundant all-reduce gives about a 13% decode gain at TP>1.

Hardware-specific

• NVIDIA: H20 block-FP8 MoE tuning gives +21%.
• ROCm:
• Preshuffled blockscaled FP8 GEMM gives +4–8% QPS on DSv3.
• Packed zero-points for W4A16 cut TPOT by 26% on Gemma 4 AWQ.
• AITER custom AG/RS cuts TPOT by about 3%.
• Arm CPU: the LM head is about 6% faster on Llama-3.1-8B W8A8.

How to use it

Artifacts. Wheels ship for:

• CUDA 12.9 and CUDA 13.0, each on x86_64 and arm64
• CPU on x86_64, arm64 and macOS
• XPU on x86_64

There is also a dedicated Zen5 CPU image, and nightly CPU images are now on Docker Hub.

New switches named on the page:

• Fast Start: --load-format ipc_cache maps post-quantized, TP-sharded weights from a per-GPU daemon over CUDA IPC. It covers FP4 checkpoints and multi-node TP.
• Scale-out endpoints (/render, /derender, /inference/v1/generate): now require --enable-scale-out on plain vllm serve.
• gRPC: use vllm serve --grpc instead of python -m vllm.entrypoints.grpc_server.
• B12X attention on SM120/SM121: --attention-backend B12X_ATTN.
• DeepSelect sparse-indexer top-k: --sparse-indexer-topk-backend.
• Engram tensor parallelism: --engram-config.
• Watermarking: per-request opt-out. With speculative decoding, use dual-key mode via allow_target_only_watermarking.

New defaults and how to revert them:

• DeepEP v2 combine overlap is on. Set VLLM_DEEPEP_V2_COMBINE_OVERLAP=0 to disable it.
• ROCm AITER custom AG/RS is on for DP attention + TP experts. Set VLLM_ROCM_USE_AITER_CUSTOM_AR=0 to disable it.
• NVIDIA kernel defaults (no off-switch is named on the page):
• FlashInfer CuTeDSL NVFP4 W4A16 replaces Marlin on SM100/103.
• W4A4 NVFP4 is preferred on SM120/121.
• The BF16x3 router GEMM is the default on SM100.
• Audio resampler: the default moved from PyAV to torchaudio.

Limits

These are the breaking changes to check before upgrading:

• GPTQ activation ordering removed. g_idx is now ignored, and the related Marlin, GPTQ, CPU and RDNA3 kernels are deleted.
• YaRN aligned with Transformers. Derived max_model_len can drop: TeleChat3-36B-Thinking goes from 131072 to 32768, and sarvam-105b from 5242880 to 131072. attn_factor and extrapolation_factor are now ignored.
• Removed env vars and aliases:
• VLLM_PREFIX_CACHE_RETENTION_INTERVAL
• VLLM_MM_HASHER_ALGORITHM
• VLLM_ENABLE_SCALE_OUT_ENDPOINTS
• use_fp4_indexer_cache
• the CUDA_VISIBLE_DEVICES fallback on ROCm (use HIP_VISIBLE_DEVICES)
• DCP backend support must now be declared. DCP with ROCm standard attention, Triton, FlexAttention or TurboQuant fails at backend selection.
• MoRI-IO WRITE mode with hybrid KV cache groups needs --disable-hybrid-kv-cache-manager.
• Mamba all cache mode is deprecated and falls back to Model Runner V1.
• Fast Start errors on platforms other than CUDA and ROCm.

These are the security fixes, which argue for upgrading exposed servers:

• Validation-error response bodies are now bounded, closing about a 5,300x response amplification.
• An unvalidated cache_salt could crash the engine via LMCache.
• Sparse embeddings were not bounded before densification.
• Request-controlled video sampling is now capped for GLMGA and Qwen-VL.
• Late-interaction caches were keyed on the client X-Request-Id.
• Credentials leaked into benchmark logs.

Dependency pins:

• CUTLASS 4.7.1
• Transformers 5.16.1
• openai >= 2.25.0
• huggingface_hub >= 1.31.0
• DeepGEMM pinned to the vllm-project fork 2.8.0
• ROCm base moved to ROCm 10.0
• CPU Docker base moved to Ubuntu 25.04 with GCC 15