Open Models Almanac
Methodsmodel architecturechannel mixermixture of expertsexpert load balancing

single mention · model architecture

Sign-based expert-load correction-bias updates

Adjusts correction bias according to the sign of each expert's load deviation from the mean.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See expert load balancing for the methods filed beside it.

Filed under model architecture › channel mixer › mixture of experts › expert load balancing