methods · softmax attention
sliding window attention
11 methods filed at this node or below it, from the sources of 17 models.
In practice
2 features checked against each model’s own code and config.json, apart from its documents. A column’s page has the verdict and the lines.
adopteddiscussednot usedblank: documents silent✓in its code✕not in its code
| Model | Sliding-window attention | Interleaved sliding-window and global attention |
|---|---|---|
| LFM3-1.7B | in its code | in its code |
| DeepSeek-V4.1-Flash-1017 | in its code | in its code |
| Step-5-Preview | in its code | in its code |
| Kolibri-1 | in its code | in its code |
| MiMo-V2.6-Flash | in its code | in its code |
| MiMo-V2.6-Pro | in its code | in its code |
| DeepSeek-V4.1-Flash | in its code | in its code |
| DeepSeek-V4-Flash-Vision-Exp | in its code | in its code |
| Laguna-S-2.1 | in its code | in its code |
| Inkling | in its code | in its code |
| Step-3.7-Flash | in its code | in its code |
| DeepSeek-V4-Flash | in its code | in its code |
| MiMo-V2.5 | in its code | in its code |
| MiMo-V2.5-Pro | in its code | in its code |
| Gemma 4 31B | in its code | in its code |
| gpt-oss-120b | in its code | in its code |
| DeepSeek-V4-Pro-0813 | in its code | not in its code |
| DeepSeek-V4-Flash-0731 | in its code | not in its code |
| DeepSeek-V4-Pro | in its code | code unclear |
| Mistral-Large-4 | not in its code | not in its code |
| MiniMax-M3.1-Flash | not in its code | not in its code |
| NVIDIA-Nemotron-3.5-Super-VL-120B-A12B | not in its code | not in its code |
| Hy4-preview | not in its code | not in its code |
| GLM-5.3-Flash | not in its code | not in its code |
| Qwen3.8-Flash-Next | not in its code | not in its code |
| GLM-5.3 | not in its code | not in its code |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | not in its code | not in its code |
| Xiaomi-Robotics-1 | not in its code | not in its code |
| Kimi K3 | not in its code | not in its code |
| Hy3 | not in its code | not in its code |
| GLM-5.2 | not in its code | not in its code |
| NVIDIA-Nemotron-3-Ultra-550B-A55B | not in its code | not in its code |
| MiniMax-M3 | not in its code | not in its code |
| MolmoAct 2 | not in its code | not in its code |
| Qwen3.6-35B-A3B | not in its code | not in its code |
| Qwen3.5-397B-A17B | not in its code | not in its code |
| DeepSeek-V3.2 | not in its code | not in its code |
| X-VLA | not in its code | not in its code |
| OpenVLA / OpenVLA-OFT | not in its code | not in its code |
In this branch 11
Everything filed at this node or below it, with one collapsible heading per child node.
filed here 11
By model
Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.