AMD Quark 0.13 adds open EAGLE3 speculative-decoding draft training on MI350X GPUs

AMD has published a complete training pipeline for EAGLE3.1 speculative-decoding draft models, running natively on AMD Instinct MI350X GPUs through ROCm, vLLM, and TorchSpec. The release centers on AMD Quark 0.13, which packages the workflow into reusable YAML recipes and a CLI tool that generates on-policy data, trains drafts with FSDP2 sharding, exports vLLM-compatible checkpoints, and benchmarks acceptance length on SPEED-Bench. The team validated the pipeline end-to-end on Qwen3-8B and on amd/MiniMax-M3-MXFP4, a large mixture-of-experts target, demonstrating that draft training no longer requires NVIDIA hardware or CUDA.

EAGLE3.1 refines the original EAGLE3 method with added normalization in the draft head for training stability, while preserving the same feature-level drafting design and vLLM serving format. The draft is trained against the target model's own hidden states rather than generic text, making its proposals easier for the verifier to accept. The full loop operates on a single 8-GPU MI350X node, with some GPUs hosting vLLM-ROCm target engines and the remainder running the FSDP2 draft trainer.

ROCm-native training pipeline and EAGLE3.1 architecture

• EAGLE3.1 refinement: Adds normalization inside the draft head compared to EAGLE3, improving training stability while maintaining backward compatibility with the LlamaForCausalLMEagle3 vLLM serving format.
• Five-stage pipeline: On-policy data generation from the target model, multi-layer hidden-state extraction via live vLLM-ROCm engines, cold-start draft training with FSDP2 sharding, export to vLLM-loadable checkpoints, and SPEED-Bench evaluation.
• Hardware topology: Single 8-GPU AMD Instinct MI350X node splits roles between vLLM target engines and FSDP2 draft trainers, eliminating cross-node overhead for this workload.
• AMD Quark 0.13 packaging: Reusable YAML recipes and CLI commands (quark.experimental.speculative_decoding.setup and .run) provide validated, reproducible workflows with idempotent environment preparation.
• Training objective: Uses the training-time-test (TTT) objective with low-, mid-, and high-level target model features, aligning the draft closely with the verifier's internal representations rather than generic language modeling.

Why it matters

This release removes a significant barrier for ROCm-based LLM serving by closing the loop between draft training and deployment without leaving the AMD software stack. Independent developers and organizations running MI300X or MI350X clusters can now train custom speculative-decoding drafts for their own target models, reducing dependence on NVIDIA's CUDA ecosystem for state-of-the-art inference acceleration. The YAML recipe approach also lowers the barrier to reproducible research and production tuning on AMD hardware.