ambiguous · filed under inference & serving
FP8
The evidence identifies FP8 as a weight format but does not specify a distinct quantization scheme.
Also called FP8 (8-bit floating point), FP8 quantization, --quantization fp8, FP8 model, FP8 quantized instruct model, Hy4 preview-FP8.
- sources
- 14
- models
- 9
- labs adopt it
- 6
- strongest
- default
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
14 spans quoted from the sources, strongest treatment first.
the default zai-org/GLM-5.3 checkpoint is now native FP8
ships native FP8 weights
Hy3-FP8 | FP8 quantized instruct model
Released under Apache 2.0 with BF16, FP8, NVFP4, and GGUF weights on Hugging Face
Hy4 preview-FP8 model weights
KV Cache is quantized to FP8
We recommend using the official FP8 checkpoint Qwen/Qwen3.5-397B-A17B-FP8 for optimal serving efficiency.
Start the FP8 model
--quantization fp8
For FP8 model ... vllm serve ...
Hy4 preview-FP8 | FP8 quantized instruct model
A separate Hy3-FP8 checkpoint is also released. FP8 lowers the memory footprint for cheaper serving.
Qwen's official FP8 checkpoint
--quantization fp8
Filed alongside
Other methods under inference & serving :: inference quantization.