MiniMax-M2 is a 62-layer, 229.9B-parameter sparse MoE (9.8B active per token) built around the thesis that a mini activation footprint can reach frontier-tier agentic performance. Every block is a full-attention GQA layer (48 query / 8 KV heads, head dim 64, RoPE throughout) followed by an MoE FFN with 256 fine-grained experts, 8 active per token, routed by sigmoid gating with learnable per-expert bias terms that implicitly regulate utilization and let the auxiliary load-balancing loss be greatly reduced. The design is deliberately conservative on the mixing axis: after extensive hybrid sliding-window exploration (spanning ratios, RoPE variants, intra/inter-layer hybrids, and sink tokens), the team kept full attention in all 62 layers, arguing no sub-quadratic variant reliably matched full-attention quality in production settings — a counterpoint to the linear-attention hybrids in the same atlas generation. The 192K native context is reached by staged training (8K → 32K → 192K) rather than any positional-encoding extension. Capacity-per-token economics carry through to inference: a single MTP module (K=1, DeepSeek-V3-style, loss weight 0.3 annealed to 0.1) is expanded by weight copying to three draft modules (K=3) during the decay phase, giving multi-step speculative decoding, and the drafts are co-trained with the RL policy so acceptance survives distribution shift. The bulk of the report is agentic post-training: SFT on interleaved thinking traces, then CISPO agent RL over MDP-formulated trajectories with composite process/outcome/time rewards, run in the Forge system (windowed FIFO, prefix-tree merging up to 40× speedup, prefill-decode disaggregation), culminating in the M2.7 self-evolution checkpoint that debugs its own training runs.
What the report claims is novel
The MiniMax-M2 series: a 229.9B-total / 9.8B-active MoE family showing that 'mini activations can unleash maximum real-world intelligence' — frontier-tier agentic performance at ~1/20 of the activated footprint of closed frontier systems [Abstract, §1]
Agent-driven data pipelines producing large-scale verifiable trajectories across agentic coding (SWE, AppDev, Terminal-Gym) and agentic cowork, each grounded in an executable workspace with artifact-aligned rewards (Agent-as-a-Verifier for AppDev) [§1, §4]
Forge, an agent-native RL system: windowed-FIFO scheduling absorbing trajectory-length variance, prefix-tree merging with up to 40× training speedup, and clean training–inference–agent decoupling supporting both white-box and black-box agents [§1, §6]
M2.7 self-evolution: an early operational form where the model autonomously triages failed training runs, edits its own agent scaffold, and runs multi-round self-improvement, closing an expensive human-in-the-loop bottleneck [§1, §6.4]
MTP expansion via weight copying (K=1 → K=3) enabling multi-step speculative decoding, with MTP modules continuously co-trained with the RL policy via top-K KL divergence [§2.3, §6.3]
A large-scale negative result: full attention retained across all layers after extensive hybrid-SWA exploration (varying ratios, RoPE settings, intra/inter-layer hybrids, sink tokens) showed no sub-quadratic variant reliably matching full attention quality [§2.2.2]
Atlas assessment: Block-level novelty is modest: fine-grained experts, sigmoid gating with learnable bias (aux-loss-reduced load balancing), and MTP-for-speculative-decoding all follow established DeepSeek-V3/DeepSeekMoE/aux-free lineages the report itself cites. Its distinctive position in the atlas is negative — a deliberate return to full attention at 230B scale while peers (moonshot-kimi-k3, upstage-solar-open-2, zai-glm-5-class hybrids) move to linear-attention hybrids; M2 keeps GQA + RoPE and buys 192K context with staged training rather than sub-quadratic layers. The genuinely novel contributions are training/infra: the Forge RL system (windowed FIFO, prefix-tree merging, white/black-box agent decoupling) and the M2.7 self-evolution loop, neither of which is a block mechanism. MTP weight-copy expansion (K=1→K=3) is a small but real refinement over DeepSeek-V3's single-module MTP.
Relation to the atlas
Lineage in
Fine-grained experts — larger number of smaller experts (DeepSeekMoE organization) from DeepSeekMoE
Multi-Token Prediction (MTP) module following the DeepSeek-V3 design during pre-training from DeepSeek-V3
Sigmoid gating with learnable expert bias for auxiliary-loss-free load balancing from Aux-Free MoE (Wang et al., not in atlas)
Clipped Importance Sampling Policy Optimization (CISPO) for RL from MiniMax-M1 (not in atlas)
Influence out
No descendants recorded yet.
Notable omissions
No per-expert FFN dimension disclosed (only 'reducing per-expert FFN size')
Normalization type and pre/post placement not disclosed; activation function not named
RoPE base frequency not disclosed
No optimizer, learning-rate schedule, batch schedule, or training precision disclosed
No training hardware, FLOPs, or GPU-hours disclosed
Tokenizer never described (only the 200,064 vocab size)
Input/output embedding tying not disclosed
No per-variant configuration for M2.5 / M2.7 — the report treats them as checkpoints of one architecture and states only ~10B activated for M2.7
Model license not named (the arXiv license on the page covers the paper only)
The checkpoint-to-checkpoint evolution M2 → M2.5 → M2.7 (data, recipes, reward changes) is not detailed
No vision/multimodal input pipeline described despite the MM Claw multi-modal evaluation benchmark
Auxiliary load-balancing loss reduction is not quantified
sigmoid gating with learnable expert-specific bias terms and top-8 selection (no softmax zero-sum constraint)[15]
Load balancing
learnable per-expert bias terms in the gating function, optimized jointly with model parameters, implicitly regulating expert utilization; auxiliary load-balancing loss greatly reduced (not eliminated)[16]
Expert granularity
fine-grained: larger number of smaller experts with reduced per-expert FFN size (following DeepSeekMoE)[17]
multi-stage training curriculum: context window progressively extended from 8K through 32K to 192K during continued pre-training; no positional-encoding modification described[22]
comprehensive curated corpus: web documents, academic literature, books, programming code, and structured question-answering content; quality assessed by model-based reward scoring plus auxiliary classifiers; balanced sampling upweights high-quality content while retaining category diversity; code, mathematics, and STEM significantly upsampled relative to their natural distribution[24]
Curriculum / staging
two-phase pre-training: constant phase on 19.9T tokens, then a decay phase with a 9.3T budget mixing short-text decay data with long-context data (high-quality code concatenation, naturally long-form PDFs, thematically related document packing); progressive context extension 8K → 32K → 192K; MTP loss weight annealed 0.3 → 0.1[25]
agentic RL over MDP-formulated trajectories (LLM as policy, context management/tool execution/memory as environment): CISPO policy gradient with composite rewards — process reward (language-mixing and tool-format penalties, structured-reasoning rewards), outcome reward, and time-efficiency shaping; interleaved chain-of-thought with reasoning-state persistence (full thinking blocks appended to message history across turns); multi-stage context-length curriculum in RL[28]
Distillation
not disclosed
Modality
Type
multimodal
Attachment
not disclosed
Inference efficiency
KV-cache design
GQA with 8 KV heads across all 62 full-attention layers: KV cache ∝ 8 heads and grows linearly with sequence length (no linear-attention layers to bound the cache)
Quantisation shipped
none disclosed
Speculative decoding
MTP-based speculative decoding: three MTP modules (K=3) generate draft tokens verified by the main model in a single forward pass; MTP modules co-trained with the RL policy via a top-K KL divergence loss to keep acceptance rates high under distribution shift; copy-initialized from main-model weights[29]
Serving optimisations
heterogeneous prefill-decode disaggregation with per-phase parallelism strategies optimized for MoE (eliminating prefill/decode mutual interference); inference kernels co-designed with the deployment stack inside the Forge RL system[30]
Evaluation
Benchmarks (report's own numbers only)
Benchmark
Value
Ref
SWE-bench Pro
56.2
Abstract, §7.5
SWE-bench Multilingual
76.5
Abstract, §7.5
Multi-SWE-bench
52.7
Abstract, §7.5
Terminal-Bench 2.0
57.0
Abstract, §7.5
NL2Repo
39.8
§7.5
MLE Bench Lite (medal rate %)
66.6
§7.5
VIBE-Pro
55.6
§7.5
HyperTask
67.6
§7.5
MM Claw
62.7
Abstract, §7.3
BrowseComp
77.8
Abstract
GDPval-AA
50.0
Abstract
Toolathlon
46.3
Abstract
AIME 2026
94.2
Abstract
GPQA-Diamond
89.8
Abstract
Provenance references
stated · Title ('The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence')
stated · §2.1 ('with 8 experts activated per token')
stated · §2.1, §2.2.1 ('Routing is implemented using sigmoid gating with learnable expert-specific bias terms'); 'Instead of softmax-based top-k gating... we use sigmoid gating'
stated · §2.2.1 ('Expert Bias... implicitly regulate expert utilization, allowing the auxiliary load-balancing loss to be greatly reduced')
stated · §2.2.1 ('Fine-Grained Experts... a larger number of smaller experts, increasing the total expert count while reducing per-expert FFN size')
stated · §5 ('We conduct Supervised Fine-Tuning (SFT) to instill the desired interleaved thinking behavior in M2'); SFT data interleaves thinking traces with intermediate actions and observations
stated · §6.2.1 ('CISPO. We adapt Clipped Importance Sampling Policy Optimization (CISPO) [MiniMax-M1] to M2 series RL training'); clipped importance ratios with stop-gradient, zero lower bound, 1+ε upper bound