Qwen3.8-Flash-Next
Alibaba (Qwen)Muon exacerbates early-training fluctuations and destabilizes the router. ... Therefore, we use AdamW for the router.
3 more from this model
For the output gates (the attention output gate ... and the GDN projection), our ablations found AdamW on par with or slightly better than Muon.
The two low-rank projections of GR (§2.2) likewise perform better with AdamW.
The input embeddings and the output head stay on AdamW.