vLLM 0.31.0 release introduces DeepSeek-V4.1-Flash optimizations and Model Runner V2

Brief written by the FOSSWire AI editor from vLLM How we write

October 6, 2026 1 min read

Key Takeaway

vLLM 0.31.0 significantly improves performance for DeepSeek models and introduces Model Runner V2 for enhanced speculative decoding.

Summary

The vLLM project has released version 0.31.0, featuring 717 commits from over 300 contributors. This update focuses on high-performance serving for DeepSeek-V4.1-Flash via FlashMLA and NVFP4 compressed KV cache, alongside the introduction of Model Runner V2. It also includes new scheduling controls, security hardening for multimodal requests, and experimental engine snapshots using CRIU.

Technical advancements in vLLM 0.31.0

DeepSeek-V4.1-Flash optimizations: Implementation of FlashMLA mega attention with NVFP4 compressed KV cache as the SM100 default.
Model Runner V2: Supports draft-model speculative decoding and custom logits processors.
Fast Restart: A new vllm preload CLI launches a weight-cache daemon to keep post-quantized weights in GPU memory across restarts.
Large-scale serving: Introduces MoonEP balanced EP all2all backend and prefill context parallelism with data parallelism.
Security: Added requirement for --trust-request-mm-kwargs to accept per-request multimodal kwargs.

Why it matters

This release accelerates the deployment of large-scale Mixture-of-Experts (MoE) models like DeepSeek by optimizing memory management and kernel fusion for specific hardware architectures.

What to do

Update to vLLM version 0.31.0.
Set the --trust-request-mm-kwargs flag if you need to accept per-request multimodal kwargs.
Note that tokenizer_mode="slow" has been removed and --enable-mamba-fine-grained-prefix-cache has been renamed to --enable-mamba-shared-prefix-checkpoint.

Original reference: github.com →