specific method · filed under inference & serving
SWA Bounded Replay
Approximates missing sliding-window-attention KV states by replaying only the most recent window-sized segment.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
4 spans quoted from the sources, strongest treatment first.
we introduce SWA Bounded Replay, which approximately reconstructs the required SWA KV states by replaying only the most recent nwin tokens.
SWA Bounded Replay instead replays only the most recent 𝑛win tokens and truncates SWA to the replay segment, accepting approximate states
SWA Bounded Replay further reduces its persistent KV cache footprint (always on SSD or in host memory) to roughly 1/8 of that of DeepSeek-V4-Flash.
SWA Bounded Replay reconstructs missing SWA KV states by replaying only the most recent n_win tokens, avoiding the need to persist SWA KV to SSD and reducing the persistent KV cache footprint to roughly 1/8 of that of DeepSeek-V4-Flash.
Filed alongside
Other methods under inference & serving :: KV cache management.