single mention · model architecture
Persistent load balancing
A load-balancing procedure described as operating in the backward pass, with no further mechanism specified.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See expert load balancing for the methods filed beside it.
Filed under model architecture › channel mixer › mixture of experts › expert load balancing