Open Models Almanac
Methodsoptimizationtraining stability

specific method · filed under optimization

Global-norm gradient clipping

Clips gradients by their global norm; the cited implementations use a maximum norm of 1.0 or magnitude 1.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training stability for the methods filed beside it.

Also called Global gradient-norm clipping, we clip the global gradient norm, gradient clipping at a global norm of 1.0, Gradient clipping, Gradient clipping at magnitude 1, clip gradient magnitude to 1.

sources
6
models
6
labs adopt it
5
strongest
used

Documented in

Evidence

7 quotes from 6 sources, one card per model, each opening with its strongest.

How sources treat it
used 7
↑