vLLM 0.31.0 improves DeepSeek-V4.1-Flash performance and large-scale serving

The vLLM 0.31.0 release includes 717 commits and introduces deep optimizations for DeepSeek-V4.1-Flash, including FlashMLA mega attention with NVFP4 compressed KV cache. The update also features Model Runner V2 with improved speculative decoding, new large-scale serving backends like MoonEP, and a new preload CLI to keep weights resident in GPU memory across restarts.

Technical highlights of vLLM 0.31.0

• DeepSeek-V4.1-Flash optimizations: Implementation of FlashMLA mega attention, DeepGEMM sparse MQA logits, and Mega-Gate fusing.
• Model Runner V2: Supports draft-model speculative decoding and custom logits processors.
• Large-scale serving: Adds MoonEP balanced EP all2all backend and prefill context parallelism with data parallelism.
• Fast restart capability: A new vllm preload CLI uses a weight-cache daemon to maintain post-quantized weights in GPU memory.
• Security enhancements: Implements request-based gating for multimodal kwargs and prevents LoRA name/cache_salt collisions.

Why it matters

This release significantly advances the efficiency of serving massive Mixture-of-Experts (MoE) models like DeepSeek on high-end hardware.