Open Models Almanac
Methodsmodel architecturechannel mixermixture of expertsexpert load balancing

single mention · model architecture

MaxVio

MaxVio is a metric that quantifies expert load imbalance in Mixture-of-Experts layers.1 It computes the peak token load on any single expert divided by the mean number of tokens per expert, so higher values indicate more severe token skew.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. To quantify expert imbalance, we measure the MaxVio metric — NVIDIA Nemotron 3 Ultra Technical Report
  2. calculates the peak load on any single expert compared to the perfectly balanced mean — NVIDIA Nemotron 3 Ultra Technical Report

Also called MaxVio metric.

Filed under model architecture › channel mixer › mixture of experts › expert load balancing