single mention · inference & serving
FP8 precision
FP8 precision stores KV cache entries in 8-bit floating point instead of BF16.1 It is applied to all KV dimensions except the RoPE dimensions, which stay in BF16, and this hybrid representation cuts the KV cache size by nearly half.2
Trade-off It halves the KV cache size relative to pure BF16 storage.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
FP8 precision is applied to the remaining dimensions
— DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceBF16 precision is used for the rotary positional embedding (RoPE) dimensions, while FP8 precision is applied to the remaining dimensions. This hybrid representation reduces the KV cache size by nearly half compared with pure BF16 storage
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencereduces the KV cache size by nearly half compared with pure BF16 storage
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Also called FP8 precision for KV storage.
Filed under inference & serving › inference quantization