Open Models Almanac
Methodsoptimizationoptimizer

implementation detail · filed under optimization

Category-specific optimizer assignment

Muon and Adam-family optimizers are assigned to different parameter categories.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See optimizer for the methods filed beside it.

Also called Tailored Training Recipe, division of labour between Muon and AdamW.

sources
5
models
2
labs adopt it
2
strongest
core

Documented in

Alibaba (Qwen)
Aleph Alpha

Evidence

6 quotes from 5 sources, one card per model, each opening with its strongest.

How sources treat it
used 4default 1core 1

Qwen3.8-Flash-Next

Alibaba (Qwen)adopted4 sources

The Muon and AdamW optimizers are applied to specific weight categories to maximize efficiency.

coreunclearin Qwen3.8-Flash-NextQwen Team
4 more from this model

the input and n-gram embeddings, output head, MoE router and the low-rank projections of GR stay on AdamW

defaultunclearin Qwen3.8-Flash-NextQwen Team

The Muon and AdamW optimizers are applied to specific weight categories

usedunclearin Qwen3.8-Flash-NextQwen Team

the division of labour between Muon and AdamW

usedoptimizationin Qwen3.8-Flash-NextQwen Team

Muon and AdamW applied to specific weight categories

usedoptimizationin Qwen3.8-Flash-NextQwen Team

Kolibri-1

Aleph Alphaadopted1 source

it uses the optimiser groups of pre-training (Table 6), i.e., Muon (Jordan et al. 2024) for the attention and expert weights and Adam (Kingma and Ba 2015) for the embedding, norm scales, LM head and router.

usedoptimizationin Kolibri RL training runAleph Alpha
↑