single mention · model architecture
Uniform-bin histogram quantile estimation
Uniform-bin histogram quantile estimation approximates a quantile of router scores by partitioning their range into B uniform bins and counting values per bin.1 Each rank accumulates local counts into a histogram, an all-reduce pools them, and the quantile is read from the first bin whose cumulative count reaches the target, interpolating within it.2
Trade-off It is accurate to within the bin width and cheap, needing only one integer all-reduce per layer per step instead of exchanging raw margins.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
We partition this interval into B uniform bins
— k3 tech reporta single all-reduce sums the local counts into the global histogram, and every rank recovers the quantile from the same pooled counts
— k3 tech reportthe only communication is one integer all-reduce of nB values per layer per step
— k3 tech report
Also called B uniform bins.
Filed under model architecture › channel mixer › mixture of experts › expert load balancing