specific method · filed under optimization
Muon
A matrix-based optimizer used primarily for matrix parameters, with orthogonalization as a distinguishing feature in the supplied evidence.
Also called Muon optimizer, Muon matrix-based optimizer.
- sources
- 11
- models
- 7
- labs adopt it
- 5
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Muon: An optimizer for hidden layers in neural networks (Keller Jordan) explainer kellerjordan.github.io
- KellerJordan/Muon code github.com
Evidence
13 spans quoted from the sources, strongest treatment first.
We employ the Muon optimizer for the majority of modules in DeepSeek-V4 series
We use Muon as the main optimizer
We use the Muon optimizer [39], specifically the Moonlight variant of Liu et al. [49], across all training stages
Muon Optimizer: employs the Muon optimizer for faster convergence and improved training stability.
Following Kimi K2, Kimi K3 adopts Muon [53] as the optimizer for its matrix parameters.
Muon is applied to the two-dimensional weights that act as linear maps
We employ the Muon optimizer (Jordan et al., 2024; Liu et al., 2025) for the parameters of linear transformations
the Muon optimizer
employ the Muon optimizer for faster convergence
In our experiments with Muon optimizer, we find that MLA with a 576-dimension latent KV-cache cannot match the performance of GQA-8.
we follow the setting of GLM-4.5, including the Muon optimizer
Muon Optimizer: We employ the Muon optimizer for faster convergence and greater training stability.
Filed alongside
Other methods under optimization :: optimizer.