text-only

taxonomy axis: modality

Decision rule: Report describes no non-text input or output.

Anchor: Llama 3.1

Models in this category (41)

Model	Org	Date	Block	Attention	Context	Total	Active
Antares	Cisco Foundation AI	not disclosed	dense	GQA	131,072	1B	1B
Nanbeige4.2	Nanbeige LLM Lab, Boss Zhipin	not disclosed	dense	not disclosed	262,144	3B	3B
GPT-2	OpenAI	not disclosed	dense		1,024	1.542B	1.542B
Laguna	poolside	not disclosed	sparse-MoE	hybrid	131,072	33.4B	3B
INTELLECT-3	Prime Intellect (Prime Intellect, Inc.)	not disclosed	sparse-MoE	not disclosed	65,536	106B	12B
Mixtral	Mistral AI	2024-01-08	sparse-MoE	GQA	32,768	47B	13B
Jamba	AI21 Labs	2024-03-28	hybrid	GQA	1M	52B	12B
Phi-3	Microsoft	2024-04-22	dense	MHA	4,096	3.8B	3.8B
Llama 3	Llama Team, AI @ Meta	2024-07-23	dense	GQA	131,072	405B	405B
Llama 3.1	Meta (Llama Team, AI @ Meta)	2024-07-31	dense	GQA	131,072	405B	405B
Phi-4	Microsoft Research	2024-12-12	dense	MHA	4,096	14B	14B
DeepSeek-V3	DeepSeek-AI	2024-12-27	sparse-MoE	MLA	4,096	671B	37B
OLMo 2	OLMo Team, Allen Institute for AI (Ai2)	2024-12-31	dense	MHA	4,096	7B	7B
DeepSeek-R1	DeepSeek-AI	2025-01-22	sparse-MoE			671B	37B
xLSTM-7B	NXAI (NX-AI)	2025-03-17	dense	linear/state-space	8,192	6.87B	6.87B
Qwen3	Qwen Team	2025-05-15	dense	GQA	32,768	32B	32B
Qwen3	Qwen Team	2025-05-15	sparse-MoE	GQA	32,768	235B	22B
Kimi K2	Kimi Team (Moonshot AI)	2025-07-28	sparse-MoE	MLA	4,096	1.043T	32.6B
gpt-oss	OpenAI	2025-08-05	sparse-MoE	hybrid	not disclosed	116.83B	5.13B
GLM-4.5 (ARC series)	Zhipu AI & Tsinghua University (GLM-4.5 Team)	2025-08-08	sparse-MoE	GQA	131,072	106B	12B
GLM-4.5 (ARC series)	Zhipu AI & Tsinghua University (GLM-4.5 Team)	2025-08-08	sparse-MoE	GQA	131,072	355B	32B
Kimi Linear	Kimi Team (Moonshot AI)	2025-11-01	sparse-MoE	hybrid	4,096	48B	3B
DeepSeek-V3.2	DeepSeek-AI	2025-12-02	sparse-MoE	MLA	131,072	not disclosed	not disclosed
OLMo 3	Allen Institute for AI (Olmo Team)	2025-12-15	dense	hybrid	8,192	32B	32B
Nemotron 3 Nano	NVIDIA	2025-12-23	sparse-MoE	GQA	524,288	31.6B	3.2B
MiMo-V2	Xiaomi (LLM-Core)	2026-01-06	sparse-MoE	hybrid	262,144	309B	15B
LongCat-Flash	Meituan LongCat Team	2026-01-29	sparse-MoE	not disclosed	131,072	68.5B	2.9B
Nanbeige4.1	Nanbeige LLM Lab, Boss Zhipin	2026-02-13	dense	not disclosed	262,144	3B	3B
GLM-5	Zhipu AI & Tsinghua University (GLM-5 Team)	2026-02-17	sparse-MoE	MLA	200K	744B	40B
Trinity	Arcee AI	2026-02-19	sparse-MoE	hybrid	262,144	400B	13B
Qwen3-Coder-Next	Qwen Team	2026-02-28	hybrid	hybrid	262,144	80B	3B
Tiny Aya	Cohere Labs / Cohere	2026-03-12	dense	hybrid	8,192	3.35B	3.35B
Nemotron 3 Super	NVIDIA	2026-04-03	sparse-MoE	GQA	1,048,576	120.6B	12.7B
Qwen3.5	Qwen Team	2026-04-21	hybrid	hybrid	262,144	not disclosed	not disclosed
DeepSeek-V4	DeepSeek-AI	2026-04-26	sparse-MoE	hybrid	1M	1.6T	49B
ZAYA1	Zyphra	2026-05-06	sparse-MoE	GQA	131,072	8.4B	760M
Mellum 2	JetBrains (with Constructor University, Bremen)	2026-05-29	sparse-MoE	hybrid	131,072	12B	2.5B
Nemotron 3 Ultra (Nemotron 3 family)	NVIDIA	2026-06-09	sparse-MoE	hybrid	1,048,576	550B	55B
Ling 2.6	Ling Team, Inclusion AI	2026-06-13	sparse-MoE	hybrid	262,144	1T	not disclosed
VibeThinker	Weibo	2026-06-15	dense	not disclosed	65,536	3B	3B
Solar Open 2	Upstage (Upstage Solar Team)	2026-07-22	sparse-MoE	hybrid	1,048,576	250B	15B

What varies within this category

attention variant varies — GQA, MHA, MLA, hybrid, linear/state-space, not disclosed. block type varies — dense, hybrid, sparse-MoE. trained context varies — 1000000, 1024, 1048576, 131072, 200000, 262144, 32768, 4096, 524288, 65536, 8192, not disclosed. openness varies — open-weights, open-weights-open-data, undisclosed. scale class varies — frontier, large, medium, not disclosed.