Kimi K3
Moonshot AIwe introduce Quantile Balancing (QB), which sets each expert bias from the router-score quantile that matches its target load [111].
4 more from this model
Quantile Balancing derives expert allocation directly from router-score quantiles, eliminating heuristic updates and a sensitive balancing hyperparameter
Both updates are thus the same quantile along the token and expert axes, respectively, which gives the method its name.
while adopting QB (§ 2.3.3) for MoE load balancing
Quantile Balancing for load management