DeepSeek-V4-Flash · DeepSeek-V4-Pro
DeepSeekThis strategy checkpoints SWA KV entries of the last tokens within every tokens, where is a tunable parameter.
specific method · filed under inference & serving
Periodic cache checkpointing stores the recurrent state only every CC decoding steps instead of each step, so the state is quantized less often.1 To keep the lagging state current, it caches input activations and replays them on-the-fly, reducing sequential quantization steps by a factor of CC.2
Trade-off It saves cache writes and quantization steps but pays extra compute and IO for the activation replay.3
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Instead of storing (thus, quantizing) the state in each decoding step, we store it every𝐶𝐶steps for an integer𝐶𝐶 >1— NVIDIA Nemotron 3 Ultra Technical Report
we cache the input activations and apply activation replay to forward the state on-the-fly. This reduces the number of sequential quantization steps by a factor of𝐶𝐶— NVIDIA Nemotron 3 Ultra Technical Report
can also save time due to saving on cache writes (though there is a tradeoff with the additional compute and IO used for the activation replay— NVIDIA Nemotron 3 Ultra Technical Report
Also called cache checkpointing, Periodic checkpointing of SWA KV entries, Periodic Checkpointing.
3 quotes from 3 sources, one card per model, each opening with its strongest.
This strategy checkpoints SWA KV entries of the last tokens within every tokens, where is a tunable parameter.
We ran initial experiments on Nemotron 3 Super NVFP4, using emulated quantization
we explore the idea of periodic cache checkpointing