vLLM 0.33.0rc2 fixes AMD ROCm AITER GEMM overflow bug on gfx950 GPUs

vLLM, the Apache-2.0 licensed high-throughput serving engine for large language models, has published v0.33.0rc2 with a ROCm-specific bugfix contributed by AMD engineer Micah Williamson. The patch disables AITER (AMD's tuned GEMM library) when the product of matrix dimensions M and N reaches or exceeds 2^31, preventing numerical corruption on AMD's gfx950 GPU target. This is a cherry-picked fix from commit 6d01893, indicating it was deemed critical enough to fast-track into the release candidate branch rather than wait for the next full release cycle.

The bugfix directly affects operators running vLLM on AMD Instinct or Radeon Pro hardware using the ROCm stack, particularly those serving large models where batch sizes or sequence lengths produce substantial GEMM workloads. The gfx950 architecture corresponds to AMD's CDNA 3-based MI300 series accelerators, which are increasingly deployed as CUDA alternatives in open-source AI inference clusters. By bounding the AITER optimization path, the fix trades peak performance for correctness on oversized matrix operations.

ROCm GEMM kernel bounds and AITER behavior

• AITER integration boundary: The fix introduces a conditional skip for AITER-tuned GEMM kernels when the dimensional product M*N >= 2^31, indicating a signed 32-bit integer overflow threshold in AMD's matrix multiplication tuning.
• gfx950 target specificity: The patch is scoped exclusively to AMD's gfx950 instruction set architecture, which covers MI300X/MI300A Instinct accelerators using CDNA 3, not consumer RDNA GPUs.
• Cherry-pick urgency: The commit was cherry-picked from 6d01893 into the rc2 tag, suggesting the overflow caused production-visible failures that could not wait for v0.34.0.
• Performance trade-off: When the skip triggers, vLLM falls back to non-AITER GEMM paths, likely ROCm's standard rocBLAS or hipBLAS implementations, with potentially reduced throughput but correct results.
• Signed-off-by chain: The patch carries Micah Williamson's AMD corporate sign-off, reflecting upstream AMD engineering directly maintaining ROCm compatibility in vLLM rather than community-only porting.

Why it matters

This fix reinforces vLLM's viability as a vendor-neutral inference engine by demonstrating responsive maintenance of the ROCm backend alongside CUDA, reducing friction for data centers deploying AMD MI300 hardware as open-source alternatives to NVIDIA. For Linux AI infrastructure operators, the bound on AITER-tuned paths provides a correctness guarantee that prevents silent data corruption in large-batch serving scenarios, though profiling may be needed to identify workloads that now hit the fallback path.