Introducing Beam: Reflection AI's 501B open-weight model

What it claims

Beam is a 501B-total / 23B-active sparse MoE, text-only, aimed at coding and agentic work. Reflection's pitch is inference efficiency, not peak capability. They position it as competitive with GLM 5.2 and approaching Qwen 3.8-Max on coding and agentic tasks, and they concede Kimi K3 is ahead on raw capability. On advanced reasoning benchmarks it reportedly matches GLM-5.2 while using 3–4× less inference compute. Midtraining extends effective context to 1M tokens. RL rollouts used a maximum of 256K tokens.

How it works

• Architecture: interleaved local and global attention, and fine-grained routed experts.
• Residual stream: a depth-based scaling scheme counters activation growth. It is combined with SandwichNorm, elementwise attention gating and FP32 residual accumulation.
• Load balancing: auxiliary-loss-free balancing with cosine-decayed expert-bias updates, plus sequence-level balancing. Reflection reports near-uniform expert utilization in the base model.
• Pretraining data: about 95% of raw web tokens were dropped. In-house classifiers retained about 1.8T high-quality tokens that conventional filters would miss, including 87% of curated web-code tokens. Code and technical content were repeated across epochs. STEM PDFs were processed with a VLM-based OCR pipeline.
• RL: asynchronous policy gradients, with every token tagged by the policy version that produced it. Reflection says training stays stable on rollouts generated more than a day earlier. The algorithm itself is not described.
• A controllable length penalty shaped reasoning length. Completion lengths first fell while scores rose, then grew again as agentic skill improved.
• About 1M environments, mostly synthetic, were filtered for difficulty (neither always solved nor impossible) and quality (not hackable, underspecified or guessable).
• Independent judges re-screened passing solutions for verifier exploits.
• Alignment: a separate safety teacher was trained from the pretrained checkpoint and merged with the RL teacher via multi-teacher on-policy distillation:

flowchart LR
  P[Pretrained base] --> M[Midtraining<br/>1M ctx] --> R[High-compute RL teacher]
  P --> S[Safety/alignment teacher<br/>SFT + RL]
  R --> D[Multi-teacher on-policy distillation]
  S --> D
  D --> B[Beam]

Safety training uses deliberative alignment with adversarial, iterated SFT data. Reward forecasting for generative-RM rewards predicted RL gains at r = 0.79, against r = 0.46 for a Best-of-N ceiling.

Evidence

• Pretraining: under 4 weeks on 6,144 GB300 NVL72 GPUs. Nine semi-automatic rewinds, caused by gradient-norm spikes or suspected silent data corruption. Goodput reached 92.3% late in the run.
• RL compute: 10.5K GB300s for 4 weeks.
• More than 100M rollouts, averaging 110K concurrent, using about 1.3B sandboxes (up to 170K concurrent).
• Inference-to-training GPU ratio ran between 3.9:1 and 5.4:1.
• Reflection reports no plateau in capability as RL compute grew.
• Infrastructure:
• Weight propagation to the inference fleet took a median of about 12s. It is hierarchical: RoCE across racks, then NVLink within a rack. This cut cross-rack traffic by 75% and made fleet-wide adoption 2.2× faster than every replica pulling directly.
• 71 inference incidents were survived without killing the job. Median recovery was 8 minutes, and lost capacity was 0.02% of serving GPU-minutes.
• Generalization: browsing improved even though no browsing tasks were in the RL mix.

The benchmark table is a figure that isn't in the page text, so per-benchmark scores and settings can't be checked here.

How to use it

• Access now: a waitlist and early-access sign-up only.
• Promised for "later this month" (October 2026):
• weights under Apache 2.0
• a technical report and model card
• the full stack for running, evaluating and fine-tuning the model
• integrations with open-source libraries and harnesses
• Inference control: a reasoning-effort parameter trades response length against performance. Its API shape isn't given.
• Hardware: no requirements are stated. Plan for serving 501B total parameters even though only 23B are active per token.

Limits

• This is a preview. Final red-teaming and evaluations are still in progress, and the weights are not yet downloadable.
• Safety evaluation results are deferred to the technical report.
• Beam is text-only. Other modalities work only when represented as text.
• Kimi K3 is acknowledged as stronger on raw capability.
• The efficiency claims (3–4× vs GLM-5.2) come without stated benchmarks or measurement settings in the text.
• The staleness-handling and training–inference mismatch algorithms are named but not explained. Wait for the report before trying to reproduce the RL recipe.