methods · softmax attention
global attention
5 methods filed at this node or below it, from the sources of 7 models.
In this branch 5
Everything filed at this node or below it, with one collapsible heading per child node.
By model
Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.
| Model | methods |
|---|---|
| Gemma 4 31B core | Unified Keys and Values core— |
| Kolibri-1 core | Full attention without positional encoding core— |
| LFM3-1.7B core | Global attention core— |
| MiMo-V2.5 core | Global attention coreFull attention used— |
| MiMo-V2.5-Pro used | Global attention used— |
| MiniMax-M3 evaluated | Full attention evaluated— |
| Qwen3.5-397B-A17B used | Full attention with multi-head attention for reliability used— |