Open Models Almanac
Methodsmodel architecturetoken mixerlinear attention & state spacegated delta network

single mention · model architecture

Kimi Delta Attention–Attention Residuals architecture

An architecture pairing Kimi Delta Attention with Attention Residuals, rather than naming KDA alone.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See gated delta network for the methods filed beside it.

Filed under model architecture › token mixer › linear attention & state space › gated delta network