single mention · inference & serving
WarpDecode token-centric routed-expert decoding
WarpDecode token-centric routed-expert decoding is a MoE decoding kernel design built around tokens rather than tiles.1 Each warp handles one output neuron and streams its associated weights directly from memory, since small-batch group GEMMs reduce to memory-bound weight streaming.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
the MoE decoding kernel upon the token-centric design of WarpDecode [12
— k3 tech reporteach warp is responsible for one output neuron and streams the associated weights directly from memory
— k3 tech report
Also called token-centric design of WarpDecode.
Filed under inference & serving › inference kernel