implementation detail · filed under inference & serving
Fused AttnRes merge and RMSNorm kernel
Fuses the AttnRes output merge and partial-sum update with subsequent RMSNorm into the preceding tensor-parallel all-reduce.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
core 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
the merging of the AttnRes output with its partial-sum update, together with the subsequent RMSNorm, is fused into the preceding TP all-reduce
coreinference servingin Kimi K3
Filed alongside
Other methods under inference & serving :: inference kernel.
Batch-invariant deterministic kernelsCUDA GraphCUDA Graph capture size reductionFlashAttention 3Kernel fusionMoE-side chunkingSynchronization-free static-shape MoE executionTensorRT-LLM multi-head attention backendAvoiding split-KDeepGEMM-based batch-invariant matrix multiplicationDistributed shared memory for cross-SM attention data exchangeDual-kernel batch-invariant attention decodingDynamic load balancingExp-free TopK kernelExpert-optimized Triton kernelsFA4 sheared-bias attention kernelFlashAttentionFlashAttention 4FlashKDAFP8 GEMMFused latent down-projection and MoE-router GEMMFused mHC kernelsFused QSA kernelFused RoPE-attention-RoPE-cast kernel