single mention · not yet filed
LSE fusion
LSE fusion computes log-sum-exp values needed for the sparse KL loss inside the main kernel's pass, emitting them to global memory as it runs.1 The backward kernel then loads these saved scalars directly into the softmax, eliminating the redundant forward computation.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
emitting these LSE values directly to global memory during the main pass, allowing us to skip the KL loss forward pass entirely
— MiniMax Sparse AttentionThe backward kernel then loads these scalars directly into the softmax, eliminating the redundant forward computation
— MiniMax Sparse Attention