medium
taxonomy axis: scale_class
Decision rule: 1B ≤ p < 10B
Anchor: Llama 3.1 8B
Models in this category (12)
| Model | Org | Date | Block | Attention | Context | Total | Active |
|---|---|---|---|---|---|---|---|
| Antares | Cisco Foundation AI | not disclosed | dense | GQA | 131,072 | 1B | 1B |
| Nanbeige4.2 | Nanbeige LLM Lab, Boss Zhipin | not disclosed | dense | not disclosed | 262,144 | 3B | 3B |
| GPT-2 | OpenAI | not disclosed | dense | 1,024 | 1.542B | 1.542B | |
| Laguna | poolside | not disclosed | sparse-MoE | hybrid | 131,072 | 33.4B | 3B |
| Phi-3 | Microsoft | 2024-04-22 | dense | MHA | 4,096 | 3.8B | 3.8B |
| Phi-4 | Microsoft Research | 2024-12-12 | dense | MHA | 4,096 | 14B | 14B |
| OLMo 2 | OLMo Team, Allen Institute for AI (Ai2) | 2024-12-31 | dense | MHA | 4,096 | 7B | 7B |
| xLSTM-7B | NXAI (NX-AI) | 2025-03-17 | dense | linear/state-space | 8,192 | 6.87B | 6.87B |
| Nanbeige4.1 | Nanbeige LLM Lab, Boss Zhipin | 2026-02-13 | dense | not disclosed | 262,144 | 3B | 3B |
| Tiny Aya | Cohere Labs / Cohere | 2026-03-12 | dense | hybrid | 8,192 | 3.35B | 3.35B |
| ZAYA1 | Zyphra | 2026-05-06 | sparse-MoE | GQA | 131,072 | 8.4B | 760M |
| VibeThinker | 2026-06-15 | dense | not disclosed | 65,536 | 3B | 3B |
What varies within this category
attention variant varies — GQA, MHA, hybrid, linear/state-space, not disclosed. block type varies — dense, sparse-MoE. trained context varies — 1024, 131072, 262144, 4096, 65536, 8192. openness varies — open-weights, open-weights-open-data, undisclosed.