VibeThinker-3B is a post-training-only project: a fixed 3B dense base (Qwen2.5-Coder-3B, named but never described internally) is driven to frontier-level verifiable-reasoning performance through a five-stage pipeline built on the authors' Spectrum-to-Signal Principle. Stage 1 is a two-stage curriculum SFT: broad multi-domain coverage (math, code, STEM, chat, instruction following) over 5 epochs, then 2 epochs on a hard subset of traces ≥5K tokens filtered by VibeThinker-1.5B rollouts, with Diversity-Exploring Distillation merging domain-specialist checkpoints at the parameter level to preserve a wide solution 'spectrum'. Stage 2 runs MGPO — a GRPO-style clipped policy objective reweighting prompts by their proximity to the maximum-entropy point p=0.5, keeping training at the capability boundary — sequentially over Math, Code, and STEM RL at a single 64K context window (progressive window expansion is rejected as harmful at this scale), followed by Long2Short Math RL, which redistributes reward among correct trajectories toward shorter ones to compress reasoning length. Stage 3 is offline self-distillation: verified trajectories from the Math/Code/STEM checkpoints, selected by a learning-potential score, are distilled back into one student. Stage 4 is Instruct RL with rule-based and rubric rewards to restore strict instruction controllability. At inference, CLR scales test-time compute at the claim level: 32 trajectories, five decision-relevant claims each, self-verified and aggregated by a reliability-weighted vote. The report's thesis — the Parametric Compression-Coverage Hypothesis — is that verifiable reasoning compresses into a small 'reasoning core' while knowledge needs broad parameter coverage, which the GPQA-Diamond gap (70.2) versus frontier peers illustrates.
What the report claims is novel
Frontier-level verifiable reasoning at a strict 3B scale: 94.3 on AIME26 and 80.2 Pass@1 on LiveCodeBench v6, matching or exceeding flagship models orders of magnitude larger (DeepSeek V3.2, GLM-5, Gemini 3 Pro, Kimi K2.5) [Abstract, §1, Table 2]
Parametric Compression-Coverage Hypothesis: verifiable reasoning is a parameter-dense capability compressible into a compact reasoning core, while knowledge and general-purpose competence are parameter-expansive coverage problems [§1, §4]
Reasoning-Knowledge Decoupling Paradigm: small models are a complementary research trajectory to parameter scaling rather than a deployment compromise [§1]
A systematically upgraded Spectrum-to-Signal post-training pipeline: two-stage curriculum SFT, multi-domain reasoning RL (Math → Code → STEM), Long2Short Math RL for token-efficiency, offline self-distillation with learning-potential filtering, and Instruct RL [§2]
CLR (Claim-Level Reliability Assessment): a claim-level test-time scaling strategy that extracts and self-verifies decision-relevant claims, improving Pass@1 on answer-verifiable math (AIME26 94.3 → 97.1) at lower token cost than trace-level verification [§3.1]
Out-of-distribution generalization: 96.1% first-attempt acceptance (123/128) on recent unseen LeetCode contests, above GPT-5.2, Kimi K2.5, and Qwen3-Max under the same protocol [§1, §3.2, Table 3]
Atlas assessment: This is a post-training recipe report on a fixed Qwen2.5-Coder-3B base: the block is entirely undisclosed, so there is no architectural novelty relative to atlas entries. The post-training stack is a GRPO-family variant (MGPO adds maximum-entropy prompt weighting to the clipped objective — a modest, well-motivated twist on deepseek-r1's verifiable-RL recipe), plus Long2Short length-reward redistribution (echoing length-bias control seen in the deepseek-r1/post-training literature), offline self-distillation from own checkpoints (similar to deepseek-r1's R1-distill consolidation), and CLR, a claim-level test-time scaling scheme that is a genuine contribution relative to trace-level self-verification (deepseek-r1's 'verify' mode). The headline claims (matching 671B-class models on AIME26) are internally consistent but rest on 2026 competition sets and majority of evaluation is Pass@1 over 64 samples with LLM-as-judge assistance on complex answers; the knowledge gap (GPQA-D 70.2 vs 80+ for frontier peers) is honestly reported and matches the compression-coverage thesis. Nothing here contradicts atlas peers; the report's own framing admits reasoning-knowledge decoupling rather than general frontier parity.
Relation to the atlas
Lineage in
Spectrum-to-Signal Principle (SSP): SFT builds a diverse solution spectrum, RL amplifies the signal from VibeThinker-1.5B
MaxEnt-Guided Policy Optimization (MGPO): GRPO-style clipped objective with maximum-entropy prompt weighting from VibeThinker-1.5B
GRPO-style clipped policy objective with group-relative advantages from DeepSeekMath (DeepSeek)
Base model: Qwen2.5-Coder-3B dense foundation model from Qwen2.5-Coder-3B (Alibaba)
Progressive context-window expansion RL considered and rejected in favor of a single 64K window from DeepScaleR
Diversity-Exploring Distillation: domain-specialist checkpoint merging from VibeThinker-1.5B
On-policy RL stabilization against rollout-training probability mismatch from Mismatch Praxis / efficient-RL-framework analyses (not in atlas)
Influence out
No descendants recorded yet.
Notable omissions
Model architecture entirely undisclosed: layers, hidden dim, heads, activation, position encoding, vocab, normalization — none appear (post-training-only report)
No pretraining information: tokens, data, compute, hardware (base model is external)
Model weights release not stated; no HuggingFace/GitHub repository or model card linked
Organisation not named anywhere in the report text
No license for the model weights (arXiv 'CC Zero' header is the paper license only)
multi-domain verifiable reasoning RL (Math → Code → STEM) with MGPO at a single 64K long-context window (progressive window expansion rejected); Long2Short Math RL redistributing rewards toward shorter correct trajectories; CLR (Claim-Level Reliability Assessment) claim-level test-time scaling — K=32 candidate trajectories, M=5 decision-relevant claims extracted per trajectory, self-verification verdicts mapped through a nonlinear reliability score (r_k = (mean verdict)^M), reliability-weighted answer aggregation — evaluated over 8 independent runs; curriculum CoT SFT with a 5K-token trace threshold[13]
Distillation
offline self-distillation: verified high-learning-potential trajectories (S_LP = length-normalized student NLL, bucketed per domain/length) from the model's own Math/Code/STEM RL checkpoints are distilled back into a unified student via SFT; note: the SFT-stage 'multi-path distillation' uses external strong-teacher models to generate training data, which the schema treats as teacher-generated data (see prose), not distillation[14]