single mention · inference & serving
Two-phase forward
Two-phase forward splits attention into two kernels separated by HBM buffers holding locally normalized partial outputs and per-partial logsumexps.1 The attention kernel writes each query's partials to preassigned slots, and the combine kernel reads the valid slots, merges the partials, and forms the final normalized output.2
Trade-off Splitting the forward across kernels is needed because the KV-outer split forbids inline softmax normalization, since each query's partials come from different CTAs.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
The forward is therefore split into two kernels separated by HBM buffers (locally normalized partial outputs) and (per-partial logsumexps
— MiniMax Sparse AttentionThe attention kernel runs the worklist above and writes each partial to its preassigned slot. The combine kernel reads the valid slots of each query, computes and , then forms normalized split-K weights
— MiniMax Sparse AttentionThe KV-outer split forbids inline softmax normalization since each query’s partials are produced by different CTAs
— MiniMax Sparse Attention
Filed under inference & serving › inference kernel