Open Models Almanac
Methodsoptimizationoptimizer

single source · optimization

Newton–Schulz orthogonalization

Newton–Schulz iterations are used for orthogonalization in the Muon update, with evidence mentioning an eight-step setting and other iteration choices.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See optimizer for the methods filed beside it.

Also called Eight-step Newton–Schulz orthogonalization, Newton–Schulz iteration to 8 steps, the number of iteration steps.

Filed under optimization › optimizer