Open Models Almanac
Methodsmodel architecturetoken mixerlinear attention & state space

methods · token mixer

linear attention & state space

18 methods filed at this node or below it, from the sources of 10 models.

In practice

3 features checked against each model’s own code and config.json, apart from its documents. A column’s page has the verdict and the lines.

adopteddiscussednot usedblank: documents silent✓in its code✕not in its code
ModelLinear-attention or state-space layers alongside full attentionGated DeltaNet layersMamba-2 layers
GLM-5.3-Flashin its codein its codenot in its code
Qwen3.8-Flash-Nextin its codein its codenot in its code
NVIDIA-Nemotron-3.5-Lightning-30B-A3Bin its codenot in its codein its code
Kimi K3in its codein its codenot in its code
NVIDIA-Nemotron-3-Ultra-550B-A55Bin its codenot in its codein its code
Qwen3.6-35B-A3Bin its codein its codenot in its code
Qwen3.5-397B-A17Bin its codein its codenot in its code
NVIDIA-Nemotron-3.5-Super-VL-120B-A12Bcode unclearcode unclearin its code
Mistral-Large-4not in its codenot in its codenot in its code
LFM3-1.7Bnot in its codenot in its codenot in its code
MiniMax-M3.1-Flashnot in its codenot in its codenot in its code
DeepSeek-V4.1-Flash-1017not in its codenot in its codenot in its code
Step-5-Previewnot in its codenot in its codenot in its code
Kolibri-1not in its codenot in its codenot in its code
MiMo-V2.6-Flashnot in its codenot in its codenot in its code
MiMo-V2.6-Pronot in its codenot in its codenot in its code
DeepSeek-V4.1-Flashnot in its codenot in its codenot in its code
Hy4-previewnot in its codenot in its codenot in its code
DeepSeek-V4-Flash-Vision-Expnot in its codenot in its codenot in its code
GLM-5.3not in its codenot in its codenot in its code
DeepSeek-V4-Pro-0813not in its codenot in its codenot in its code
DeepSeek-V4-Flash-0731not in its codenot in its codenot in its code
Xiaomi-Robotics-1not in its codenot in its codenot in its code
Laguna-S-2.1not in its codenot in its codenot in its code
Inklingnot in its codenot in its codenot in its code
Hy3not in its codenot in its codenot in its code
GLM-5.2not in its codenot in its codenot in its code
MiniMax-M3not in its codenot in its codenot in its code
Step-3.7-Flashnot in its codenot in its codenot in its code
MolmoAct 2not in its codenot in its codenot in its code
DeepSeek-V4-Flashnot in its codenot in its codenot in its code
DeepSeek-V4-Pronot in its codenot in its codenot in its code
MiMo-V2.5not in its codenot in its codenot in its code
MiMo-V2.5-Pronot in its codenot in its codenot in its code
Gemma 4 31Bnot in its codenot in its codenot in its code
DeepSeek-V3.2not in its codenot in its codenot in its code
X-VLAnot in its codenot in its codenot in its code
gpt-oss-120bnot in its codenot in its codenot in its code
OpenVLA / OpenVLA-OFTnot in its codenot in its codenot in its code

In this branch 18

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 2

Linear attention core · 2 sources · 2 quotes
1 only mentioned as not used
Lightning Attention not used · 1 source · 1 quote

Mamba 4

Mamba-2 core · 5 sources · 6 quotes
Mamba core · 2 sources · 2 quotes
2 single mentions
Mamba checkpointing core · 1 source · 1 quote
Mamba-2 SSM cache optional · 1 source · 1 quote

gated delta network 12

Gated DeltaNet core · 12 sources · 12 quotes
Kimi Delta Attention core · 6 sources · 6 quotes
KDA core · 2 sources · 2 quotes
8 single mentions
Gated attention with reduced KV heads core · 1 source · 1 quote
Gated DeltaNet hybrid attention core · 1 source · 1 quote
Hybrid linear attention mechanism core · 1 source · 1 quote
Bounded sigmoid output gate used · 1 source · 1 quote
Input-dependent full-rank output gate used · 1 source · 1 quote
Lower-bounded scaled-sigmoid log-decay used · 1 source · 1 quote
SimpleGDN evaluated · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑