Open Models Almanac
Methodsmodel architecturetoken mixersparse attention

specific method · filed under model architecture

Compressed Sparse Attention 2

Compressed Sparse Attention 2 (CSA2) is an attention sharing scheme that applies cross-layer reuse to global KV, indexer K, and Top-K indices to substantially reduce KV cache storage.1 It runs in three statically assigned modes: Full generates and indexes fresh KV, Reindex reuses KV but reselects indices, and Reuse shares both KV and indices, while every layer keeps its own queries and SWA KV.2

Trade-off Sharing KV and indices cuts duplicated cache storage substantially, enabling much smaller persistent KV caches.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. applies cross-layer reuse to global KV (including main KV and indexer K) and Top-K indices to substantially reduce KV cache storage — DeepSeek V41 Tech Report
  2. Full Mode generates global KV and performs indexing. Reindex Mode reuses the global KV from a preceding layer, and uses its own indexer Q to rescore the shared indexer K and select fresh Top-K indices. Reuse Mode reuses both global KV and the Top-K indices in a preceding layer — DeepSeek V41 Tech Report
  3. Sharing global KV and indexer K reduces duplicated cache storage — DeepSeek V41 Tech Report

Also called Compressed Sparse Attention 2 (CSA2), Cross-layer KV and index reuse, cross-layer reuse, CSA2.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Evidence

3 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
core 3

DeepSeek-V4.1-Flash

DeepSeekadopted2 sources

DeepSeek-V4.1-Flash uses CSA2, which assigns each attention layer one of three static modes — Full, Reindex, or Reuse — to share main KV and indexer K across layers and reuse Top-K sparse-attention indices.

coremodel architecturein DeepSeek-V4.1-FlashDeepSeek
2 more from this model

we design Compressed Sparse Attention 2 (CSA2), which applies cross-layer reuse to global KV (including main KV and indexer K) and Top-K indices

coreunclearin DeepSeek-V4.1-FlashDeepSeek

CSA2 ... applies cross-layer reuse to global KV (including main KV and indexer K) and Top-K indices

coreunclearin DeepSeek-V4.1-FlashDeepSeek
↑