taxonomy node · level 4
gated delta network
11 methods filed at this node or below it, from the sources of 6 models.
model architecture :: token mixer :: linear attention & state space :: gated delta network
Matching aids for the classifier: Gated DeltaNet; Kimi Delta Attention; KDA; SimpleGDN.
In this branch 11
Everything filed at this node or below it, with one collapsible heading per child node.
filed here 11
By model
Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.
| Model | techniques |
|---|---|
| GLM-5.3-Flash core | Kimi Delta Attention core— |
| GLM-5.2 evaluated | Gated DeltaNet evaluatedSimpleGDN evaluated— |
| Kimi K3 core | Hybrid linear attention coreKDA coreKimi Delta Attention coreKimi Delta Attention and Attention Residuals architecture coreGated DeltaNet usedKimi Delta Attention with input-dependent full-rank output gate usedKimi Delta Attention with lower-bounded log-decay used— |
| Qwen3.5-397B-A17B core | Gated DeltaNet coreGated DeltaNet with reduced KV-head configuration coreGated DeltaNet–sparse MoE hybrid core— |
| Qwen3.6-35B-A3B core | Gated DeltaNet core— |
| Qwen3.8-Flash-Next core | Gated DeltaNet coreGated DeltaNet with bounded sigmoid output gate used— |