FP16 with stochastic rounding preserves FP32-cache accuracy
3 more from this model
enable-mamba-cache-stochastic-rounding
stochastic rounding on gradients
--enable-mamba-cache-stochastic-rounding
implementation detail · filed under optimization
Stochastic Rounding is used for FP16 SSM cache storage.1 It mitigates the recurrent rounding bias observed with naive FP16 round-to-nearest cache storage and preserves FP32-cache accuracy and verbosity.2,3
Trade-off It reduces the number of sequential quantization steps and saves time on cache writes, but involves additional compute and IO for activation replay.4
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
For the current release, we use FP16 SSM cache storage with stochastic rounding— NVIDIA Nemotron 3 Ultra Technical Report
mitigating the recurrent rounding bias observed with naive FP16 round-to-nearest cache storage— NVIDIA Nemotron 3 Ultra Technical Report
FP16 with stochastic rounding preserves FP32-cache accuracy and verbosity— NVIDIA Nemotron 3 Ultra Technical Report
reduces the number of sequential quantization steps by a factor of𝐶𝐶 and can also save time due to saving on cache writes (though there is a tradeoff with the additional compute and IO used for the activation replay— NVIDIA Nemotron 3 Ultra Technical Report
Also called Stochastic rounding for Mamba cache, mamba-cache-stochastic-rounding, stochastic rounding on gradients.
6 quotes from 5 sources, one card per model, each opening with its strongest.
FP16 with stochastic rounding preserves FP32-cache accuracy
enable-mamba-cache-stochastic-rounding
stochastic rounding on gradients
--enable-mamba-cache-stochastic-rounding
such as randomness generation, stochastic rounding, or scaling calculations.
stochastic rounding on gradients
such as randomness generation, stochastic rounding, or scaling calculations.
Picked by hand, not extracted: where to read more, not evidence for anything on this page.