Open Models Almanac
Methodsinference & servinginference quantization

specific method · filed under inference & serving

FP4 global KV-cache quantization

FP4 global KV-cache quantization stores the attention key–value cache in a four-bit floating-point format instead of higher-precision storage, reducing its memory footprint during long-context serving.1,2 Quantization-aware training is applied so the cache can use a four-bit format, and cached values are dequantized before attention so no native four-bit matrix-multiplication support is needed.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Long-context agent workloads require large per-request KV caches, increasing serving costs — DeepSeek V41 Tech Report
  2. We now extend QAT to the main KV cache, where FP4 reduces storage rather than accelerates matrix multiplication — DeepSeek V41 Tech Report
  3. Dequantizing cached values before attention allows us to use a more accurate format without requiring native matrix-multiplication support for that format — DeepSeek V41 Tech Report

Also called FP4 global key-value cache, FP4 global KV caches, FP4 main KV cache, FP4 KV caching, FP4 main KV caching, FP4-quantized main key-value cache.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 1

DeepSeek-V4.1-Flash

DeepSeekadopted2 sources

Combined with FP4 main KV caching (E2M1 format, one E4M3 scale per 16 channels), these designs reduce the global KV cache footprint to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash.

coremodel architecturein DeepSeek-V4.1-FlashDeepSeek
1 more from this model

we use FP4 global KV caches during training with only marginal performance degradation

usedunclearin DeepSeek-V4.1-FlashDeepSeek
↑