Open Models Almanac
Methodsinference & servinginference quantization

specific method · filed under inference & serving

ModelOpt NVFP4 re-quantization

ModelOpt NVFP4 re-quantization re-quantizes a model's MLP linears to NVFP4 (W4A16) to shrink the weights.1 It keeps attention linears and the KV cache in FP8, so the smaller checkpoint fits a single GPU and vLLM auto-detects the quantization format.2

Trade-off NVIDIA reports near-lossless accuracy versus the FP8 baseline.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. MLP linears drop to NVFP4 (W4A16 — Qwen/Qwen3.6-27B — 27B · DENSE · 256K ctx
  2. the attention linears and KV cache stay FP8, so the ~22 GB weights fit a single Blackwell GPU — Qwen/Qwen3.6-27B — 27B · DENSE · 256K ctx
  3. NVIDIA reports near-lossless accuracy versus the FP8 baseline — Qwen/Qwen3.6-27B — 27B · DENSE · 256K ctx

Also called NVFP4 ModelOpt re-quantization (W4A16), ModelOpt re-quantization, NVFP4 quantization with modelopt, NVFP4 model.

sources
2
models
2
labs adopt it
2
strongest
used

Documented in

Alibaba (Qwen)

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
optional 1used 1
↑