Step-3.7-Flash
StepFunFor NVFP4 model ... --quantization modelopt
specific method · filed under inference & serving
ModelOpt NVFP4 re-quantization re-quantizes a model's MLP linears to NVFP4 (W4A16) to shrink the weights.1 It keeps attention linears and the KV cache in FP8, so the smaller checkpoint fits a single GPU and vLLM auto-detects the quantization format.2
Trade-off NVIDIA reports near-lossless accuracy versus the FP8 baseline.3
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
MLP linears drop to NVFP4 (W4A16— Qwen/Qwen3.6-27B — 27B · DENSE · 256K ctx
the attention linears and KV cache stay FP8, so the ~22 GB weights fit a single Blackwell GPU— Qwen/Qwen3.6-27B — 27B · DENSE · 256K ctx
NVIDIA reports near-lossless accuracy versus the FP8 baseline— Qwen/Qwen3.6-27B — 27B · DENSE · 256K ctx
Also called NVFP4 ModelOpt re-quantization (W4A16), ModelOpt re-quantization, NVFP4 quantization with modelopt, NVFP4 model.
2 quotes from 2 sources, one card per model, each opening with its strongest.
For NVFP4 model ... --quantization modelopt
NVIDIA's ModelOpt re-quantization: MLP linears drop to NVFP4 (W4A16) while the attention linears and KV cache stay FP8