Open Models Almanac
Methodsinference & servinginference kernel

single mention · inference & serving

WarpDecode token-centric routed-expert decoding

WarpDecode token-centric routed-expert decoding is a MoE decoding kernel design built around tokens rather than tiles.1 Each warp handles one output neuron and streams its associated weights directly from memory, since small-batch group GEMMs reduce to memory-bound weight streaming.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. the MoE decoding kernel upon the token-centric design of WarpDecode [12 — k3 tech report
  2. each warp is responsible for one output neuron and streams the associated weights directly from memory — k3 tech report

Also called token-centric design of WarpDecode.

Filed under inference & serving › inference kernel