Laguna-S-2.1
Poolsidewe apply an exponential moving average (EMA) over the 10 most recent checkpoints
usedoptimizationin Laguna XS.2Poolside
implementation detail · filed under optimization
Exponential moving average (EMA) computes an average over recent checkpoints to produce smoother weights.1 It is used to reduce variation in the final checkpoint, with the EMA weights serving as the final base checkpoint.2
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we apply an exponential moving average (EMA) over the 10 most recent checkpoints— laguna m1 xs2 technical report
To reduce variation in the final base checkpoint, we apply an exponential moving average (EMA) over the 10 most recent checkpoints— laguna m1 xs2 technical report
Also called disabling EMA.
2 quotes from 2 sources, one card per model, each opening with its strongest.
we apply an exponential moving average (EMA) over the 10 most recent checkpoints
you may want to consider disabling EMA