single mention · inference & serving
Dynamic load balancing
Dynamic load balancing is an inference-kernel technique that evens out per-tile work, which varies by orders of magnitude under variable-length sequences and data-dependent sparsity.1 The kernel runs as a persistent grid where CTAs claim work through a global atomic counter, and tiles are split into sub-tiles whose count scales with per-tile query count, with a minimum granularity to amortize overhead.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Per-tile work varies by orders of magnitude under variable-length sequences and data-dependent sparsity
— MiniMax Sparse AttentionThe kernel runs as a persistent grid in which CTAs claim work through a global atomic counter; each tile is partitioned along its gathered-query dimension into sub-tiles whose count scales with the per-tile query count, subject to a minimum sub-tile granularity that amortizes per-sub-tile overhead
— MiniMax Sparse Attention
Filed under inference & serving › inference kernel