Nanbeige4.1-3B is a post-training recipe applied to the Nanbeige4-3B-Base checkpoint; the report discloses no architecture beyond the 3B parameter count and a 256K context, so the block itself (a dense decoder, per family context) is inherited unchanged. The contribution is a cascaded, capability-balanced training stack. SFT is re-mixed toward code and hard math, run through a three-stage context curriculum (32k → 64k → 256k, with the final stage mixing code 27%, deep-search 26%, STEM 23%, tool-use 13%, general 10%), and improved by scaling the Solution Refinement / CoT Reconstruction loop from the prior Nanbeige4-3B-2511 release. General RL then runs GRPO twice: point-wise RL with a general reward model (8 rollouts per prompt) that suppresses redundancy and formatting errors, then pair-wise RL with a binary win/loss reward and a swap-consistency regularizer to remove position bias. Code RL runs in two stages — correctness via a pass-rate reward, then a gated time-complexity reward active only when all tests pass, with a judge system (multi-language sandbox plus an instruct judge model) comparing against reference optimal complexity. Finally a lightweight agentic RL stage, trained on Wikipedia-graph multi-hop QA and turn-judged long trajectories (open-sourced as ToolMind-Web-QA), delivers stable long-horizon search — up to 600 tool-call turns. The result is a 3B model that leads small models on deep-search benchmarks and beats much larger peers on coding and alignment — at the cost of disclosing no architectural detail at all.
What the report claims is novel
First open-source small language model (3B) to combine strong agentic behavior, code generation, and general reasoning in a single model [Abstract]
Combination of point-wise and pair-wise reward modeling for reasoning and preference alignment (GRPO with a general reward model, then a pair-wise reward model with a swap-consistency regularizer) [§2.1]
Complexity-aware rewards in code RL: a gated time-complexity reward activated only when all test cases pass, with a judge system (multi-language sandbox + instruct judge model) comparing generated solutions against the reference optimal complexity bound [§2.3]
Deep-search data synthesis (Wikipedia entity-relation graph random walks with temporal-aware head-entity selection; trajectory synthesis with turn-level judgment on logical soundness, tool-call accuracy, and informational gain) enabling stable long-horizon tool interactions of up to 600 tool-call turns [§2.2]
Open-sourced the ToolMind-Web-QA search dataset on HuggingFace [§2.2.1]
A unified multi-stage training recipe (SFT → point-wise RL → pair-wise RL → two-stage code RL → agentic RL) that keeps a 3B model broadly competent and strongly specialized simultaneously [§2.4]
Deep-search performance at 3B scale that beats specialized search models at tens-of-billions scale and approaches 100B+ general-purpose models (GAIA 69.90, xBench-DeepSearch-05 75.00, SEAL-0 41.44) [§3.2]
Atlas assessment: A pure post-training contribution on the Nanbeige4-3B base: no architectural novelty is claimed or described (the report has no architecture section at all). The reward-engineering stack — point-wise + pair-wise RL, a gated time-complexity reward, and turn-level vs trajectory-level credit in agentic RL — extends the cascaded-RL line the report cites (Nemotron-Cascade); the gated complexity reward that fires only on fully correct solutions is the most distinctive element. The 600-turn agentic-stability and deep-search claims at 3B scale are strong but not independently verifiable. vs microsoft-phi-4 (the atlas's closest 'small generalist' peer): same small-model-broad-capabilities intent, but Nanbeige4.1 optimizes for agentic deep search and code efficiency rather than instruction-following density; vs weibo-vibethinker-3b: same 3B reasoning scale, but Nanbeige adds long-horizon tool use and alignment stages. The complete absence of architecture disclosure makes cross-atlas block comparison impossible.
Relation to the atlas
Lineage in
Base model and initialization (Nanbeige4.1-3B is built upon the Nanbeige4-3B-Base) from Nanbeige4-3B (Boss Zhipin)
Solution Refinement and CoT Reconstruction framework (scaled up from the previous version) from Nanbeige4-3B-2511
Point-wise reward modeling / cascaded RL paradigm from Nemotron-Cascade (NVIDIA)
Group Relative Policy Optimization (GRPO) from DeepSeekMath / DeepSeek-R1
Entity-relation graph random walks and multi-hop QA synthesis for tool-use data (ToolMind framework) from ToolMind (Boss Zhipin)
Unified pairwise RLHF framework with swap-consistency regularizer for position-bias mitigation from Unified pairwise RLHF framework (Xu et al. 2025)
Influence out
No descendants recorded yet.
Notable omissions
No architecture disclosure at all: layers, hidden dim, heads, activation, normalization, position encoding, tokenizer, vocab — none described (post-training recipe paper)
No pre-training details (token counts, data, compute) — the model is initialized from Nanbeige4-3B-Base
No license named for the release
No training hardware, optimizer, learning-rate schedule, precision, or parallelism
No FLOPs or GPU-hours
No inference-efficiency disclosures (KV cache, quantization, serving)
No direct measurement of the 600-tool-call-turn ceiling (only the claim)
General-evaluation settings are minimal (harness details given only for deep-search evals: Mindflow framework, Serper/Jina/E2B tools)
SFT mixture redesigned with a higher proportion of code-related data and higher ratios of challenging mathematics and general-domain problems; final 256k-stage mixture: code 27%, deep-Search 26%, STEM 23%, tool-use 13%, general domains 10%. Deep-search data: multi-hop QA pairs from Wikipedia entity-relation graphs (temporal-aware head-entity selection, conditional random walks) plus synthesized long-horizon trajectories with turn-level judgment (open-sourced as ToolMind-Web-QA). Code data: judge-system-filtered time-optimal solutions; RL data: point-wise (human preference), pair-wise (strong-weak model comparisons with checklist filtering), code (problem + test cases + time-complexity-optimal solution + optimal complexity label), agentic (1,827 synthesized task-oriented environments with tools and verifiers)[8]
Curriculum / staging
post-training pipeline on the Nanbeige4-3B-Base: extended SFT (three context stages, final 256k) → point-wise RL (GRPO, 8 rollouts per prompt, general reward model) → pair-wise RL (pairwise reward model, swap-consistency regularizer) → two-stage code RL (Stage 1 correctness pass-rate reward; Stage 2 gated time-complexity reward active only at PassRate=1) → lightweight agentic RL; on-policy difficulty filtering in code RL (n=8 rollouts, retain problems with k in [1,5] qualifying rollouts)[9]
cascaded multi-stage RL: point-wise RL with a general reward model (8 rollouts per prompt) suppressing redundancy/repetition (overlong truncation reduced from 5.27% to 0.38% on LiveCodeBench-v6); pair-wise RL with binary win/loss reward vs reference answers, multi-turn dialogue history concatenated into the reward model input; two-stage code RL with correctness pass-rate reward then a gated time-complexity reward (judge system compares generated solutions against reference optimal complexity); lightweight agentic RL for tool-use and search; deep-search training with turn-level judgment (logical soundness, tool-call accuracy, informational gain) and trajectory-level rewards, enabling stable long-horizon interaction of up to 600 tool-call turns[12]