Open Models Almanac
Methodsoptimizationoptimizer

implementation detail · filed under optimization

Splitting fused parameters before orthogonalization

Fused parameters or gradients are divided into sub-matrices before orthogonalization, then their results are recombined or applied in the original layout.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See optimizer for the methods filed beside it.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Alibaba (Qwen)

Evidence

3 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2core 1

Qwen3.8-Flash-Next

Alibaba (Qwen)adopted2 sources

We therefore split the fused gradient before orthogonalization, run NS on each sub-matrix independently, and gather the results back into the original layout before applying the update.

coreoptimizationin Qwen3.8-NextQwen Team
2 more from this model

Fused parameters are split before orthogonalization

usedunclearin Qwen3.8-Flash-NextQwen Team

the splitting of fused parameters

usedoptimizationin Qwen3.8-Flash-NextQwen Team
↑