Open Models Almanac
Methodsoptimization

general family · filed under optimization

Hyperparameter scaling-law fitting

Develops or refits a scaling law to determine training hyperparameters such as learning rate and batch size.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See optimization for the methods filed beside it.

Also called refit the scaling law, dedicated scaling-law studies.

sources
3
models
2
labs adopt it
2
strongest
core

Documented in

Alibaba (Qwen)
Moonshot AI

Evidence

4 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
used 3core 1

Qwen3.8-Flash-Next

Alibaba (Qwen)adopted2 sources

The new architecture and optimizer also shift the optimal hyperparameters, so we refit the scaling law used for the Qwen3.5 series.

usedunclearin Qwen3.8-Flash-NextQwen Team
2 more from this model

Therefore, we develop an updated hyperparameter scaling law.

coreoptimizationin Qwen3.8-NextQwen Team

with the scaling law refitted for the new architecture

usedoptimizationin Qwen3.8-Flash-NextQwen Team

Kimi K3

Moonshot AIadopted1 source

we conduct dedicated scaling-law studies to retune key hyperparameters, including the batch size, learning rate, tokens-per-parameter ratio (TPP) and the model shape.

usedoptimizationin Kimi K3Moonshot AI
↑