Reflection releases Beam, a 501B parameter open MoE model
Key Takeaway
Beam is a 501B parameter Mixture-of-Experts model designed for coding and agentic tasks with a 1M token context window.
Summary
The new US-based lab Reflection has released Beam, a large-scale Mixture-of-Experts (MoE) model under the Apache 2.0 license. While the model contains 501B total parameters, it only uses 23B active parameters per token, allowing for high-capacity reasoning at higher speeds. It is specifically optimized for coding, tool calling, and long-context agentic workflows.
Technical architecture of Beam
- Mixture-of-Experts (MoE) structure: 52 layers total, with 49 layers utilizing 256 experts; 8 experts are routed per token.
- Context Window: Supports up to 1M tokens using a hybrid attention mechanism (43 sliding-window layers and 9 global layers).
- Memory and Speed: Requires memory for 501B parameters but operates at the computational speed of a 23B parameter model.
- Precision Support: Available in BF16, FP8, and NVFP4 formats.
- Optimization: Includes a custom Rust-based inference harness and day-0 support for transformers, vLLM, and SGLang.
Why it matters
The Apache 2.0 release of a frontier-scale MoE model provides developers with a high-performance alternative for self-hosted agentic and coding applications.
Original reference: huggingface.co →