ONNX Runtime WebGPU Plugin EP 0.5.0 adds MoE, sparse attention, and AMD subgroup kernels

Microsoft has released version 0.5.0 of the ONNX Runtime WebGPU Execution Provider (EP) plugin, which introduces extensive support for modern machine learning operators. This update focuses on enhancing performance for complex models through new kernels for Mixture-of-Experts (MoE), sparse attention mechanisms, and quantized operations.

Technical improvements target both efficiency and hardware utilization, including specialized kernels for AMD GPUs and optimized data movement for common tensor operations. The release also addresses runtime integration, offering better support for concurrent session execution and improved memory management when interfacing with the Dawn WebGPU implementation.

Technical advancements in 0.5.0

• Architectural Support: Added native kernels for GatedDeltaNet and various sparse attention mechanisms, including SparsePagedAttention and DynamicSparseAttention to support continuous batching.
• MoE and Quantization: Expanded Mixture-of-Experts (MoE) capabilities with unquantized support and specialized quantized execution for INT2 expert weights and block-scaled FP8 QMoE.
• GPU Performance: Enabled subgroup-matrix kernels specifically for AMD hardware and introduced FP16 GEMV specializations for single-row decode projections.
• Runtime and Integration: Introduced concurrent independent-Session execution on compatible ONNX Runtime hosts and added an adapterIndex to allow independent selection of physical Dawn adapters.
• Reliability Fixes: Resolved critical issues including out-of-bounds stores in subgroup-matrix MatMulNBits and fixed initializer upload failures on GPUs lacking Resizable BAR support.

Why it matters

This release enables more sophisticated, locally-run AI models to leverage WebGPU acceleration in web browsers and native environments. By optimizing for AMD hardware and supporting advanced quantization formats, it reduces the barrier for running high-performance, memory-efficient LLMs on consumer-grade hardware without vendor lock-in.