single mention · model architecture
Decoder SWA Bounded Replay
Decoder SWA Bounded Replay bounds the decoder's prefill replay to the last window-size tokens of the prompt when regenerating sliding-window attention KV.1,2 It avoids the expensive full decoder pass over window×layer tokens, since uncached decoder SWA KV must otherwise be recomputed, and the approximate replayed KV is used only for decoding.3,4
Trade-off The replayed SWA KV is only approximate, but experiments show this barely compromises response quality.5
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
only prefills the last𝑛win tokens of the prompt for the SWA computation
— DeepSeek V41 Tech Reportbounds the decoder forward pass to𝑛 win tokens
— DeepSeek V41 Tech Reportexactly reconstructing it requires running the𝐿 2 decoder layers over the last 𝐿 2×𝑛 win prompt tokens, which is expensive
— DeepSeek V41 Tech Reportuse the resulting decoder SWA KV only for decoding, not for prefix caching
— DeepSeek V41 Tech Reportthe replayed prefix state is approximate
— DeepSeek V41 Tech Report
Also called decoder bounded replay during prefill.
Filed under model architecture › token mixer › softmax attention › sliding window attention