single mention · model architecture
MaxVio
MaxVio is a metric that quantifies expert load imbalance in Mixture-of-Experts layers.1 It computes the peak token load on any single expert divided by the mean number of tokens per expert, so higher values indicate more severe token skew.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
To quantify expert imbalance, we measure the MaxVio metric
— NVIDIA Nemotron 3 Ultra Technical Reportcalculates the peak load on any single expert compared to the perfectly balanced mean
— NVIDIA Nemotron 3 Ultra Technical Report
Also called MaxVio metric.
Filed under model architecture › channel mixer › mixture of experts › expert load balancing