specific method · filed under inference & serving
Synchronization-free static-shape MoE execution
Uses static shapes to eliminate per-layer host synchronization in MoE execution.
Also called sync-free MoE execution with static shapes, sync-free execution with static shapes.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1core 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
This eliminates the per-layer MoE host synchronization and alleviates the host-side kernel-launch overhead.
coresoftware implementationin Kimi K3Moonshot AI
In conventional MoE implementations, the per-expert token counts vary across steps and layers, and the host must synchronize with the device at every layer to obtain the actual computation
usedunclearin Kimi K3Moonshot AI
Filed alongside
Other methods under inference & serving :: inference kernel.
Batch-invariant deterministic kernelsCUDA GraphCUDA Graph capture size reductionFlashAttention 3Kernel fusionMoE-side chunkingTensorRT-LLM multi-head attention backendAvoiding split-KDeepGEMM-based batch-invariant matrix multiplicationDistributed shared memory for cross-SM attention data exchangeDual-kernel batch-invariant attention decodingDynamic load balancingExp-free TopK kernelExpert-optimized Triton kernelsFA4 sheared-bias attention kernelFlashAttentionFlashAttention 4FlashKDAFP8 GEMMFused AttnRes merge and RMSNorm kernelFused latent down-projection and MoE-router GEMMFused mHC kernelsFused QSA kernelFused RoPE-attention-RoPE-cast kernel