Kolibri-1
Aleph AlphaIn addition to this per-token bound, we clip the global gradient norm.
1 more from this model
a maximum gradient norm of 1.0.
specific method · filed under optimization
Clips gradients by their global norm; the cited implementations use a maximum norm of 1.0 or magnitude 1.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training stability for the methods filed beside it.
Also called Global gradient-norm clipping, we clip the global gradient norm, gradient clipping at a global norm of 1.0, Gradient clipping, Gradient clipping at magnitude 1, clip gradient magnitude to 1.
7 quotes from 6 sources, one card per model, each opening with its strongest.
In addition to this per-token bound, we clip the global gradient norm.
a maximum gradient norm of 1.0.
gradient clipping at a global norm of 1.0.
gradient clipping at a global norm of 1.0.
Gradient clipping with a maximum norm of 1.0 is applied to stabilize optimization
Gradient clipping is applied with a maximum norm of 1.0
clip gradient magnitude to 1