FP8 per-tensor GEMMs for shared experts and Mamba linear layers; static max-calibrated per-tensor scales.
specific method · filed under inference & serving
Per-tensor FP8 quantization
FP8 quantization using a single scale per tensor, distinct from block-scaled FP8 approaches.
The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See inference quantization for the methods filed beside it.
Also called Per-tensor-scale FP8 quantization, per-tensor FP8.
Documented in
Evidence
2 quotes from 2 sources, one card per model, each opening with its strongest.
How sources treat it
evaluated 1core 1
corepost trainingin Nemotron 3 UltraNVIDIA
aggregation benchmarks can identify important candidates in FP8, where even per-tensor FP8 precision can be acceptable over 100 tasks at sixteen repetitions.
evaluatedoptimizationin Nemotron 3.5 Super and LightningNVIDIA