full-attention

taxonomy axis: sequence_mixing

Decision rule: All layers use unrestricted attention over the whole context.

Anchor: Llama 3

Models in this category (18)

Model	Org	Date	Block	Attention	Context	Total	Active
Antares	Cisco Foundation AI	not disclosed	dense	GQA	131,072	1B	1B
GPT-2	OpenAI	not disclosed	dense		1,024	1.542B	1.542B
INTELLECT-3	Prime Intellect (Prime Intellect, Inc.)	not disclosed	sparse-MoE	not disclosed	65,536	106B	12B
Mixtral	Mistral AI	2024-01-08	sparse-MoE	GQA	32,768	47B	13B
Phi-3	Microsoft	2024-04-22	dense	MHA	4,096	3.8B	3.8B
Llama 3	Llama Team, AI @ Meta	2024-07-23	dense	GQA	131,072	405B	405B
Llama 3.1	Meta (Llama Team, AI @ Meta)	2024-07-31	dense	GQA	131,072	405B	405B
Phi-4	Microsoft Research	2024-12-12	dense	MHA	4,096	14B	14B
DeepSeek-V3	DeepSeek-AI	2024-12-27	sparse-MoE	MLA	4,096	671B	37B
OLMo 2	OLMo Team, Allen Institute for AI (Ai2)	2024-12-31	dense	MHA	4,096	7B	7B
DeepSeek-R1	DeepSeek-AI	2025-01-22	sparse-MoE			671B	37B
Qwen3	Qwen Team	2025-05-15	dense	GQA	32,768	32B	32B
Qwen3	Qwen Team	2025-05-15	sparse-MoE	GQA	32,768	235B	22B
Kimi K2	Kimi Team (Moonshot AI)	2025-07-28	sparse-MoE	MLA	4,096	1.043T	32.6B
GLM-4.5 (ARC series)	Zhipu AI & Tsinghua University (GLM-4.5 Team)	2025-08-08	sparse-MoE	GQA	131,072	106B	12B
GLM-4.5 (ARC series)	Zhipu AI & Tsinghua University (GLM-4.5 Team)	2025-08-08	sparse-MoE	GQA	131,072	355B	32B
MiniMax-M2 series	MiniMax	2026-05-26	sparse-MoE	GQA	192K	229.9B	9.8B
VibeThinker	Weibo	2026-06-15	dense	not disclosed	65,536	3B	3B

What varies within this category

attention variant varies — GQA, MHA, MLA, not disclosed. block type varies — dense, sparse-MoE. trained context varies — 1024, 131072, 192000, 32768, 4096, 65536, not disclosed. openness varies — open-weights, open-weights-open-data, undisclosed. scale class varies — frontier, large, medium.