vLLM 0.32.0 and 0.31.0: Breaking Changes, Low‑Precision KV Caches and Faster Startup

Which breaking changes affect your deployment? #

Each row names the release that lists it, so check the version you are moving to before applying an action. The v0.32.0 entries are in the Release v0.32.0 notes, and the v0.31.0 entries are in the Release v0.31.0 notes. Most rows change configuration or client code rather than the server's runtime behaviour, so the work is mainly a review of your configs, pins and clients.

v0.32.0: Am I affected by its breaking changes? #

You are affected by v0.32.0 if your environment runs Python 3.10, because that interpreter is dropped. The same notes restrict default CORS origins, so browser clients that relied on the previous open default need an explicit origin list. Dependency bounds also change to transformers 5.16.1 up to 5.20.0 and tokenizers 0.23.2 or newer, which affects projects with pins outside that range.

Several v0.32.0 removals change configuration. Cascade attention, mamba_cache_mode="all", and the torch.compile sequence parallelism and AsyncTP passes are listed as removed. Check your launch configs and deployment scripts for these names, since a removed option will no longer be honoured.

Two v0.32.0 changes affect request handling rather than configuration. max_tokens is no longer reserved from the context window, which matters for clients that send prompts close to the limit. Responses API reasoning stream events were renamed, so any consumer that matches event names needs an update.

v0.31.0: Am I affected by its breaking changes? #

Only deployments that accept per-request multimodal options are blocked by v0.31.0 until they set a flag. Per-request mm_processor_kwargs and media_io_kwargs are rejected unless --trust-request-mm-kwargs is set, as listed in the Release v0.31.0 notes. Before enabling that flag, confirm that only trusted callers can reach the server.

The other v0.31.0 breaks are option-level. tokenizer_mode="slow" is removed, the Mamba prefix-cache flag is renamed, quantization="fp8" is replaced by the fp8_per_tensor shorthand, and the AllSpark INT8 W8A16 backend is removed. Scripts and launch configs that use these names should be updated before rollout.

Two v0.31.0 changes alter run behaviour. --enforce-eager now also disables JIT kernel warmup, and XPU graphs are enabled by default, which removes the VLLM_XPU_ENABLE_XPU_GRAPH variable. Eager-mode and XPU deployments should measure startup and first-request timing after upgrading.

v0.32.0: Low-precision KV caches across GPU generations #

NVFP4 KV cache now runs on more GPU generations in v0.32.0. The notes list NVFP4 KV cache on SM8x and SM12x, on SM90, and with XQA decode on SM12x, and MiniMax-M3 sparse attention on SM100. Check your GPU architecture against these entries before planning KV memory. ROCm gains a 4-bit ultraquant_4bit KV cache, which the notes describe as about 2x the density of FP8.

The GLM-5.x change alters a default. GLM-5.x now defaults to an FP8 KV cache on SM100, which the notes say halves KV memory per rank. In the PR's benchmark, that setup gave 2.3-5.5% more throughput at a 7-11% TTFT cost. Because those figures come from one benchmark, operators on SM100 should measure TTFT on their own workload before accepting the default.

For GLM-5.3-Flash on SM90, the notes state the following:

fp8_ds_mla on SM90 gives GLM-5.3-Flash up to 31% more output throughput ( #59246 )

Release v0.32.0

The notes do not state whether fp8_ds_mla on SM90 is enabled by default or needs a configuration change, so check the default in your setup before expecting the gain.

v0.32.0: Startup, kernel caching and API worker routing #

v0.32.0 adds two startup commands for FlashInfer kernels and autotuning. vllm download-kernels installs FlashInfer's precompiled kernels, and vllm preload --preload-autotune keeps the FlashInfer autotune table warm across restarts. Tokenizers and processors also load from the local snapshot. This suggests that services restarting often may spend less time on these steps, but the notes do not give timing figures for that.

vllm download-kernels
vllm preload --preload-autotune

When --api-server-count is above 1, each connection goes to the least-loaded API worker. The PR's benchmark reports 43-85% more req/s on short requests, measured with Qwen3-0.6B at 32 input and 1 output tokens. That is a single benchmark setup, so apply the range only to workloads with similar request sizes. The same release also adds a shared-memory tensor arena that broadcasts large multimodal tensors to workers with one copy.

v0.32.0: Security fixes and stricter input limits #

v0.32.0 contains fixes for three advisories, so deployments exposed to them should upgrade. The fixed advisories are GHSA-4hhp-h66f-j5j7 (chat template resource exhaustion), GHSA-q97c-p368-75c4 (tokenizer max_token_id) and GHSA-frvg-qw88-hww9 (P2P ZMQ socket exhaustion). The notes also restrict CORS defaults, which is covered under breaking changes above.

Input handling is stricter in v0.32.0. Untrusted images are limited to raster formats, JSON schemas nested deeper than 64 levels are rejected, and shared-memory multimodal cache handles are signed. Services that accept user images or schemas should test those requests after the upgrade, because inputs outside these limits are refused.

v0.32.0: Model Runner V2 and speculative decoding #

Model Runner V2 is now the default on ROCm in v0.32.0. The model-specific MRV1 exceptions are removed. MRV2 also gains a GPU n-gram speculator with variable-length drafts, Elastic EP, pipeline parallelism with prefill context parallelism for MLA models, and a stock torch.compile mode. ROCm users running model-specific paths should confirm they now run on MRV2.

Speculative and draft paths also change in v0.32.0. GLM-5.3 sparse MLA index conversions are reused across layers, which the notes give as 3.5-3.9x on that kernel. Fused multi-step draft decode is listed at 13.3% at concurrency 1. These are separate items with separate setups, so compare each against its own measurement.

v0.32.0: New models and the structured decision endpoint #

v0.32.0 adds a POST /v1/systemone endpoint, served by default. It answers choice, noul and score questions over a structured state. The notes also list Cloudflare/clef-omni, LiquidAI/d1-3B and DiffusionGemma as running on it. Teams building on structured decisions should check whether they need this route.

Other v0.32.0 model additions, all listed in the Release v0.32.0 notes, include:

• EmbeddingGemma 2 (google/embeddinggemma-2), an omni-modal embedding model.
• MiniCPM-V 4.7, PLaMo 3 reasoning and tool parsers, and LoRA for BERT, RoBERTa and XLM-RoBERTa embedding models.
• Nemotron 3.5 ASR for complete-audio transcription; native realtime streaming is listed as not yet supported.
• Video input in the Transformers modeling backend.

The HiSparse per-request residency caching change gives up to 58% more throughput on GLM-5.3-NVFP4 on B200, with long, fully GPU-resident contexts at high concurrency. That figure applies only to that setup, so check whether your workload matches it before expecting similar gains.

v0.31.0: DeepSeek-V4.1-Flash and Qwen performance changes #

In v0.31.0, FlashMLA mega attention with the V4.1 NVFP4 compressed KV cache is the SM100 default for DeepSeek-V4.1-Flash. The Release v0.31.0 notes also list DeepGEMM sparse MQA logits for the indexer, Mega-Gate, fused TP all-reduce with mHC input preparation, and an MXFP8 wo_b GEMM fused with sequence-parallel reduce-scatter. Operators running V4.1 on SM100 should review these defaults against their current setup.

For Qwen3.8-Flash-Next (Qwen4Exp), v0.31.0 lists merged QSA projections in one GEMM at 1.45-1.68x, and SM100 low-latency decode GEMM plans at up to 2.7x against cuBLAS on small-M. Each figure is tied to the setup named in its own entry, so do not apply them to other models or batch shapes.

v0.31.0: Fast restart and experimental snapshots #

The new vllm preload CLI in v0.31.0 keeps post-quantized weights resident in GPU memory across engine restarts. The weight-cache daemon later gained data parallelism, MTP draft models, a /health endpoint and a readiness wait, as listed in the v0.31.0 notes. Teams that restart engines often can use it to avoid reloading weights each time.

Experimental initialized-engine snapshots (vllm snapshot create and vllm snapshot restore) use CRIU to restore a fully initialized TP1 engine. Since the feature is marked experimental, this suggests testing it on non-critical hosts first.

v0.31.0: Scheduling controls and HiSparse hardening #

v0.31.0 changes how prefill chunking behaves under --long-prefill-token-threshold. The threshold now adapts to the number of waiting prefills instead of chunking a lone request. A new --max-num-active-seqs flag caps RUNNING admission independently of max_num_seqs. Schedulers that tuned prefill chunking should re-test their latency numbers.

The release also reworks the waiting queue so requests already holding KV blocks are scheduled first, and it fixes a KV connector and MTP deadlock under KV pressure. HiSparse receives several fixes, including a chunked-prefill preemption livelock. Operators using HiSparse should verify that admission and preemption metrics still behave as expected.

v0.31.0: Security hardening and prefix-cache isolation #

Prefix-cache keys are now separated by source in v0.31.0, so a LoRA name and a cache_salt can no longer collide. The LoRA path is also part of the block hash. Stale multimodal receiver-cache entries can no longer replace fresh payloads. Multi-tenant deployments that rely on LoRA names or cache salts should review their cache-key assumptions.

v0.31.0: Other model and backend changes #

v0.31.0 also adds structured generation for DiffusionGemma and a Granite 4.2 parser. Smaller items from the notes include:

• DiffusionGemma structured generation mode with bounded single-token choices.
• Cohere2MoE auxiliary hidden states for EAGLE3 and DFlash drafters.
• A built-in granite_thinking_parser for Granite 4.2.
• Kimi-K3 vision patch embedder as a GEMM, and a fused KimiViT QK RoPE path listed at up to 29x.
• Engram tables shared across DP replicas by default.

How do I upgrade safely? #

Start by checking your Python version and dependency pins against the v0.32.0 notes. The v0.32.0 release lists the image vllm/vllm-openai:v0.32.0, and the v0.31.0 release lists vllm/vllm-openai:v0.31.0. Review each version's section separately, because the notes present their changes separately.

1. Confirm your environment is not on Python 3.10, which v0.32.0 drops.
2. Check transformers and tokenizers pins against the v0.32.0 bounds.
3. Set explicit CORS origins for browser clients.
4. Remove or replace options removed in v0.32.0 and v0.31.0, including cascade attention, mamba_cache_mode="all", tokenizer_mode="slow" and quantization="fp8".
5. Update Responses API consumers for the renamed reasoning stream events.
6. Set --trust-request-mm-kwargs only where callers are trusted, and test multimodal requests.
7. If you run GLM-5.x on SM100, measure TTFT with the new FP8 KV default.
8. Test startup with vllm download-kernels and vllm preload if you use them.
9. Pull the image for your target release and run your regression suite before rollout.

docker pull vllm/vllm-openai:v0.32.0
docker pull vllm/vllm-openai:v0.31.0