taxonomy node · level 4
global attention
4 methods filed at this node or below it, from the sources of 5 models.
model architecture :: token mixer :: softmax attention :: global attention
Matching aids for the classifier: full-context attention.
In this branch 4
Everything filed at this node or below it, with one collapsible heading per child node.
By model
Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.
| Model | techniques |
|---|---|
| MiMo-V2.5 core | Global attention coreFull attention used— |
| MiniMax-M3 evaluated | Full attention evaluated— |
| Gemma 4 31B core | Unified Keys and Values core— |
| MiMo-V2.5-Pro used | Global attention used— |
| Qwen3.5-397B-A17B used | Multi-Head Attention (MHA) used— |