Kimi K3
Moonshot AIA hybrid linear attention mechanism that replaces standard quadratic attention in a subset of layers. KDA maintains full expressiveness for critical layers
1 more from this model
Its architecture uses KDA and Attention Residuals for computational efficiency.