specific method · filed under model architecture
Compressed Sparse Attention 2
A CSA variant that shares global key-value state across layers and reuses sparse selections to reduce cache and indexing costs.
Also called Compressed Sparse Attention 2 (CSA2), cross-layer KV-cache and index reuse, cross-layer KV and index reuse, CSA2.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
6 spans quoted from the sources, strongest treatment first.
CSA2 shares global KV across layers to reduce cache storage and reuses sparse selections to reduce indexing work.
cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) and FP4 KV caching reduce its global KV cache footprint
we design Compressed Sparse Attention 2 (CSA2), which applies cross-layer reuse to global KV (including main KV and indexer K) and Top-K indices to substantially reduce KV cache storage.
CSA2 shares main KV and indexer K across layers and allows layers to reuse Top-K indices
DeepSeek-V4.1-Flash uses CSA2, which assigns each attention layer one of three static modes — Full, Reindex, or Reuse — to share main KV and indexer K across layers and reuse Top-K sparse-attention indices.
The remaining 18 encoder layers use CSA2 with a compression rate of 𝑚= 2.
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: sparse attention indexer.