specific method · filed under model architecture
Gated Multi-Head Latent Attention
A gated variant of Multi-Head Latent Attention, identified as a method for memory-efficient KV-cache management.
Also called Gated MLA.
- sources
- 3
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2core 1
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Kimi K3: Open Frontier Intelligence (Kimi Team, 2026) paper arxiv.org§2.1.2 defines Gated MLA
- Multi-Head Latent Attention (MLA) - LLM Architecture Gallery explainer sebastianraschka.combackground on the base MLA mechanism being gated
Evidence
3 spans quoted from the sources, strongest treatment first.
MLA caches ct and reconstructs the content keys and values through learned up-projections during attention computation.
coremodel architecturein Kimi K3Moonshot AI
Gated MLA: Gated Multi-head Latent Attention for memory-efficient KV-cache management
usedmodel architecturein Kimi K3Moonshot AI
Filed alongside
Other methods under model architecture :: token mixer :: softmax attention :: multi-head latent attention.