KDA

A method labeled KDA and described as hybrid linear attention that replaces standard quadratic attention in some layers; the evidence does not settle whether this is Kimi Delta Attention or a different expansion of the acronym.

Also called Kimi Distributed Attention (KDA).

sources: 2
model: 1
lab adopt it: 1
strongest: core

How sources treat it

One count per evidence span, weakest treatment to strongest.

core 2

Documented in

Kimi K3

Evidence

2 spans quoted from the sources, strongest treatment first.

A hybrid linear attention mechanism that replaces standard quadratic attention in a subset of layers. KDA maintains full expressiveness for critical layers

coremodel architecturein Kimi K3Moonshot AI

Kimi K3 Model Overview: 2.8T Parameters, MXFP4 Quantization, and What the Open Weights Mean for the Community·Hugging Face community blog·2026-07-17

Its architecture uses KDA and Attention Residuals for computational efficiency.

coremodel architecturein Kimi K3Moonshot AI

Kimi K3 - API Pricing & Benchmarks·OpenRouter·2026-07-19

Filed alongside

Other methods under model architecture :: token mixer :: linear attention & state space :: gated delta network.

Gated DeltaNet Kimi Delta Attention Gated DeltaNet–sparse MoE hybrid Gated DeltaNet with bounded sigmoid output gate Gated DeltaNet with reduced KV-head configuration Hybrid linear attention Kimi Delta Attention and Attention Residuals architecture Kimi Delta Attention with input-dependent full-rank output gate Kimi Delta Attention with lower-bounded log-decay SimpleGDN