single mention · inference & serving
Restricted-search MSE weight-scale selection
MSE 4-over-6 is a weight-scale selection scheme for quantization that picks each block scale to minimize quantization error by sweeping over FP8 scale values.1 It differs from plain MSE search by restricting the scale search to a narrower space.2
Trade-off Evaluation found it recovers accuracy worse than unrestricted MSE scale selection.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
MSE: Each block scale is selected to minimize quantization error by sweeping over FP8 scale values
— NVIDIA Nemotron 3.5 Super and Lightning Technical ReportMSE 4-over-6: MSE-based scale selection restricted to a narrower search space
— NVIDIA Nemotron 3.5 Super and Lightning Technical Reportthe MSE algorithm is superior in terms of accuracy recovery compared to the Max, and MSE 4-over-6
— NVIDIA Nemotron 3.5 Super and Lightning Technical Report
Also called MSE 4-over-6.
Filed under inference & serving › inference quantization