Amazon releases HOPE: second-order pruning method for Mixture-of-Experts LLMs

Amazon has published HOPE (Higher-Order Pruning of Experts), a new method for compressing Mixture-of-Experts language models by pruning expert MLPs while preserving their cooperative relationships. Unlike existing first-order approaches that score experts independently, HOPE formulates pruning as a quadratic optimization problem that captures pairwise interactions between experts through a surrogate objective upper-bounding the pruning error. The technique is particularly relevant for massive MoE models like Qwen3.5-122B, where 95% of the roughly 250 GB parameter footprint consists of expert MLPs despite only 8 of 256 experts per layer being active for any given token.

The method has been released with open-source code and a detailed implementation guide on Hugging Face. At aggressive 50% pruning rates, HOPE achieves the best average rank across evaluation tasks compared to competing methods, with particularly strong gains of up to +6.1% on agentic coding benchmarks. The approach requires no retraining, composes with quantization, and avoids the latency penalties of dynamic expert loading strategies.

How HOPE differs from first-order pruning

• Quadratic formulation: HOPE minimizes E[Z] over a calibration set, reformulated as p^T F p where F is an E x E interaction matrix, making it a binary quadratic program rather than independent scalar ranking.
• Surrogate objective: The method defines Z = [sum of pruned expert contributions]^2, which strictly upper-bounds the dominant component of true pruning error and naturally produces cross-terms between expert pairs.
• Pairwise interaction terms: Expanding Z yields g_i(x)g_j(x)||f_i(x)||_2||f_j(x)||_2 cross-terms that capture cooperative expert relationships discarded by first-order methods.
• No retraining required: Like other pruning approaches, HOPE permanently deletes experts without requiring fine-tuning or training, and composes with quantization for further compression.
• Replacement-aware error modeling: The formulation accounts for how pruned experts are replaced by the next-highest gate weight alternatives, though the full dependency is handled through the tractable surrogate rather than direct optimization.

Why it matters

For organizations deploying large MoE models on resource-constrained infrastructure, HOPE offers a principled way to halve memory requirements while preserving task performance that depends on expert cooperation patterns. The open-source release on Hugging Face with implementation guidance lowers barriers for independent researchers and downstream projects to apply second-order pruning to their own models, though the practical deployment story for AMD ROCm, Intel, or CPU inference backends remains to be documented by the community.