specific method · filed under inference & serving
FP4 global KV-cache quantization
FP4 global KV-cache quantization stores the attention key–value cache in a four-bit floating-point format instead of higher-precision storage, reducing its memory footprint during long-context serving.1,2 Quantization-aware training is applied so the cache can use a four-bit format, and cached values are dequantized before attention so no native four-bit matrix-multiplication support is needed.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Long-context agent workloads require large per-request KV caches, increasing serving costs
— DeepSeek V41 Tech ReportWe now extend QAT to the main KV cache, where FP4 reduces storage rather than accelerates matrix multiplication
— DeepSeek V41 Tech ReportDequantizing cached values before attention allows us to use a more accurate format without requiring native matrix-multiplication support for that format
— DeepSeek V41 Tech Report
Also called FP4 global key-value cache, FP4 global KV caches, FP4 main KV cache, FP4 KV caching, FP4 main KV caching, FP4-quantized main key-value cache.