Open Models Almanac
Methodsoptimizationoptimizer

specific method · filed under optimization

AdamW

AdamW is used as an optimizer, including for non-matrix parameters; the evidence includes a configuration with weight decay disabled.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See optimizer for the methods filed beside it.

Also called AdamW optimizer, AdamW optimization without weight decay, we use AdamW for the router.

sources
7
models
7
labs adopt it
6
strongest
core

Documented in

Alibaba (Qwen)
Aleph Alpha
Mistral AI
Physical Intelligence
Xiaomi

Evidence

10 quotes from 7 sources, one card per model, each opening with its strongest.

How sources treat it
used 9core 1

Qwen3.8-Flash-Next

Alibaba (Qwen)adopted1 source

Muon exacerbates early-training fluctuations and destabilizes the router. ... Therefore, we use AdamW for the router.

coreoptimizationin Qwen3.8-NextQwen Team
3 more from this model

For the output gates (the attention output gate ... and the GDN projection), our ablations found AdamW on par with or slightly better than Muon.

usedoptimizationin Qwen3.8-NextQwen Team

The two low-rank projections of GR (§2.2) likewise perform better with AdamW.

usedoptimizationin Qwen3.8-NextQwen Team

The input embeddings and the output head stay on AdamW.

usedoptimizationin Qwen3.8-NextQwen Team

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑