Open Models Almanac
Methodsoptimization

specific method · filed under optimization

Exponential moving average of model weights

Maintains a smoothed model by exponentially averaging network weights, with one mention specifying a weight of 0.999.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See optimization for the methods filed beside it.

Also called exponential moving average of weights.

sources
2
models
2
lab adopt it
1
strongest
used

Documented in

Physical Intelligence
Toyota Research Institute

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1unclear 1
↑