Open Models Almanac
Methodsinference & servinginference kernel

single mention · inference & serving

Two-phase forward

Two-phase forward splits attention into two kernels separated by HBM buffers holding locally normalized partial outputs and per-partial logsumexps.1 The attention kernel writes each query's partials to preassigned slots, and the combine kernel reads the valid slots, merges the partials, and forms the final normalized output.2

Trade-off Splitting the forward across kernels is needed because the KV-outer split forbids inline softmax normalization, since each query's partials come from different CTAs.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The forward is therefore split into two kernels separated by HBM buffers (locally normalized partial outputs) and (per-partial logsumexps — MiniMax Sparse Attention
  2. The attention kernel runs the worklist above and writes each partial to its preassigned slot. The combine kernel reads the valid slots of each query, computes and , then forms normalized split-K weights — MiniMax Sparse Attention
  3. The KV-outer split forbids inline softmax normalization since each query’s partials are produced by different CTAs — MiniMax Sparse Attention

Filed under inference & serving › inference kernel