Open Models Almanac
Methodsoptimizationquantization-aware training

implementation detail · filed under optimization

Stochastic Rounding

Stochastic Rounding is used for FP16 SSM cache storage.1 It mitigates the recurrent rounding bias observed with naive FP16 round-to-nearest cache storage and preserves FP32-cache accuracy and verbosity.2,3

Trade-off It reduces the number of sequential quantization steps and saves time on cache writes, but involves additional compute and IO for activation replay.4

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. For the current release, we use FP16 SSM cache storage with stochastic rounding — NVIDIA Nemotron 3 Ultra Technical Report
  2. mitigating the recurrent rounding bias observed with naive FP16 round-to-nearest cache storage — NVIDIA Nemotron 3 Ultra Technical Report
  3. FP16 with stochastic rounding preserves FP32-cache accuracy and verbosity — NVIDIA Nemotron 3 Ultra Technical Report
  4. reduces the number of sequential quantization steps by a factor of𝐶𝐶 and can also save time due to saving on cache writes (though there is a tradeoff with the additional compute and IO used for the activation replay — NVIDIA Nemotron 3 Ultra Technical Report

Also called Stochastic rounding for Mamba cache, mamba-cache-stochastic-rounding, stochastic rounding on gradients.

sources
5
models
3
lab adopt it
1
strongest
used

Documented in

Evidence

6 quotes from 5 sources, one card per model, each opening with its strongest.

How sources treat it
optional 1used 5

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted3 sources

FP16 with stochastic rounding preserves FP32-cache accuracy

usedpost trainingin Nemotron 3 UltraNVIDIA
3 more from this model

enable-mamba-cache-stochastic-rounding

usedinference servingin Nemotron 3 UltraNVIDIA

stochastic rounding on gradients

usedoptimizationin Nemotron 3 UltraNVIDIA

--enable-mamba-cache-stochastic-rounding

optionalinference servingin Nemotron 3 UltraNVIDIA

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑