Qwen3-Coder-Next is an 80-billion-parameter mixture-of-experts model that activates 3 billion parameters per token, built on the Qwen3-Next base with a hybrid-attention backbone — and that is the report's entire architectural disclosure. There is no configuration table: layers, hidden width, head counts, expert counts, vocabulary, normalization, and activation are all inherited from the base model and left unspecified. What the report documents instead is a complete agentic training recipe. Mid-training expands the context from 32,768 to 262,144 tokens so the model can carry multi-turn tool-use trajectories, mixing natural GitHub data (370 languages, roughly 600B repository-level tokens) with synthetic PR-derived tasks (~800K verifiable instances across 9+ languages), text–code grounding data rewritten by the larger Qwen3-Coder-480B-A35B-Instruct teacher, and FIM data in chat and search-and-replace forms. After continued pretraining, SFT on verified agentic trajectories is followed by specialization into four domain experts — web development, user experience/tool-format adherence, single-turn QA, software engineering — and distillation back into a single unified model. Execution-verifiable RL then runs in two regimes: single-turn, unit-test-verified coding, and multi-turn agentic SWE interaction with trajectory-level rewards, token-level tool-format penalties, and a reinforced reward-hacking blocker. The result is a small-footprint coding agent that matches models with an order of magnitude more active compute on the SWE-Bench suites, arguing that scaling agentic training, not model size, drives coding-agent capability. Architecturally this record is the thinnest in the atlas; as a training-recipe disclosure it is among the most detailed.
What the report claims is novel
Scaling agentic training rather than model size: a compact 80B-A3B model, trained on large-scale synthesized verifiable coding tasks with environment feedback, matches models with an order of magnitude more active compute on agentic benchmarks [Abstract, §1]
Large-scale task-synthesis pipeline: GitHub PR mining with constructed runnable Docker environments plus extension of open-source executable datasets (SWE-Smith, SWE-Flow, SWE-Rebench, Multi-SWE-RL), yielding ~800K verifiable SWE task instances across 9+ programming languages, with quality-assurance-agent filtering [§2.1]
MegaFlow, a fully cloud-native (Alibaba Cloud Kubernetes) orchestration system enabling production-scale parallel execution, evaluation, and data generation for agentic coding workloads [§2.2]
Repository-level mid-training recipe: context expanded 32,768 → 262,144 tokens, GitHub language coverage expanded 92 → 370 languages, ~600B repository-level tokens, best-fit packing, and masking of highly repetitive segments [§3.1.1, §3.2]
Tool-chat-template diversity training and the new XML-style qwen3_coder tool-calling format for string-heavy arguments, improving format-invariant tool use across IDE/CLI scaffolds [§4.2.2]
Expert specialization followed by distillation: Web Development, User Experience, Single-turn QA, and Software Engineering experts consolidated into one unified deployment model without expert routing [§4.2, §4.2.5]
Execution-verifiable RL beyond competitive programming (single-turn unit-test-verified coding) plus multi-turn agentic RL with a reinforced reward-hacking blocker for GitHub-based environments, eliminating agent exploits of future-commit leakage [§4.2.3, §4.2.4]
First comparative evaluation of a coding model against frontier models on cybersecurity benchmarks (AthenaBench, PrimeVul-Paired, SecCodeBench, CWEval) [§A.4]
Atlas assessment: Every claimed contribution is in the training recipe and data infrastructure — none is architectural: the hybrid-attention MoE backbone is inherited from the Qwen3-Next base and the report discloses no config details at all. vs atlas peers, the mid-training → SFT → multi-expert → distillation → execution-RL pipeline extends the Qwen2.5-Coder/Qwen3-Coder lineage (repository-level pretraining with special concatenation tokens is explicitly carried over from Qwen2.5-Coder), and tool-template diversity plus the reward-hacking blocker are practical engineering contributions of the same family as the agentic-RL recipes seen in Kimi K3, though K3's report is far more architecture-heavy. The genuinely distinctive claims are (1) the demonstration that a 3B-active model can match models with ~10× active compute via scaled agentic training, and (2) the first security-benchmark comparison for a coding agent. Architecturally this is the thinnest record in the atlas: layers, experts, attention internals, and normalization are simply not discussed.
Relation to the atlas
Lineage in
Hybrid attention + MoE backbone (80B total / 3B active per token) from Qwen3-Next (pretrained base)
Repository-level pretraining with special concatenation tokens from Qwen2.5-Coder
Influence out
No descendants recorded yet.
Notable omissions
No architecture/config table: layers, hidden dim, head counts, expert counts, FFN dims, vocab size, activation, and normalization are all undisclosed (inherited from Qwen3-Next)
Exact attention composition not disclosed beyond 'hybrid attention' — no full/linear layer pattern, no GQA/MLA details, no attention heads
No model license named (only the arXiv paper license line appears)
No pretraining details: the Qwen3-Next base pretraining is out of scope; mid-training volume given only as 'trillions of tokens'
No optimizer, learning rate, batch size, precision, parallelism, training hardware, or compute (FLOPs/GPU-hours) disclosed
RL algorithm not named (no GRPO/PPO/RLOO); only 'execution-based rewards' are described
Deployed context length and any positional-encoding extension method not disclosed
Tokenizer not described (FIM tokens and ChatML format referenced only)
No KV-cache, quantization, speculative-decoding, or serving disclosures (MegaFlow covers rollout infrastructure, not serving)
The 30B-A3B 'Flash' sibling is not covered by this report
mid-training corpus primarily natural data with a smaller synthetic portion: (1) GitHub source code with language support expanded from 92 to 370 languages, file- and repository-level data (~600B repository-level tokens; repository concatenation via special tokens), (2) text–code grounding data from Common Crawl and math/programming/education domains, rewritten into normalized Markdown by Qwen3-Coder-480B-A35B-Instruct, (3) PR-based structured SWE tasks mined from GitHub, (4) synthetic single-turn QA and multi-turn agentic trajectories generated by multiple agent frameworks (SWE-agent, Mini-SWE-agent, OpenHands, Claude-Code, Qwen-Code, Terminus) with Qwen3-Coder-480B-A35B-Instruct as teacher, (5) a small instruction-following mix, (6) FIM data from Stack-V2 (chat-FIM and search-and-replace FIM); pretraining corpus updated through Sep 30, 2025[10]
Curriculum / staging
staged pipeline: continued pretraining (mid-training) on code/agent data with context 32,768 → 262,144 tokens → SFT → specialization of four domain experts → expert distillation into one unified model → single-turn and multi-turn execution-verifiable RL; best-fit packing (BFP) for sample packing; masking of highly repetitive segments[11]
execution-verifiable RL in two regimes: single-turn RL on unit-test-verified coding tasks (competitive programming, library-usage, multilingual, secure coding) with majority-vote unit tests; multi-turn agentic RL on SWE tasks with trajectory-level completion rewards plus unfinished-trajectory and turn-level tool-format penalties and a reinforced reward-hacking blocker; long-horizon behavior emerged (average agent turns 50 → 130)[14]
Distillation
expert distillation: knowledge from domain-specialized experts (Web Development, User Experience, Single-turn RL/QA, Software Engineering) distilled into the unified SFT model[15]
stated · Abstract, §1 ('activates only 3 billion per forward pass'); Table 3 (Size 80A3)
stated · §1 ('based on Qwen3-Next with hybrid attention and Mixture-of-Experts'); §6 ('hybrid mixture-of-experts architecture')
stated · §3.1.1, §3.2 ('we expand the training context length from 32,768 tokens to 262,144 tokens'; 'we extend the context length beyond typical pretraining settings to 262,144 tokens')
stated · §3.1, §3.1.1, §3.1.2, §3.1.4
stated · §1, §3.2, §4
stated · §4.1 ('we first perform supervised fine-tuning (SFT)'); data from in-house corpora, verified agentic trajectories, and documentation-grounded QA, filtered by a Mini-SWE-agent-based user-simulator verifier and pairwise judging
stated · §4.1
stated · §4.2.3, §4.2.4
stated · §4.2.5
n/a · §6 (text-only; visual capability planned only for future models)