specific method · filed under software implementation
MegaMoE
MegaMoE, in DeepSeek's DeepGEMM library, runs an MoE layer's dispatch, both expert matrix products, the activation and the combine as one pipelined kernel that overlaps the exchange of tokens between GPUs with the expert arithmetic. It needs expert parallelism over NVLink with symmetric memory on every GPU, and each variant supports specific number formats.
Written for the explainer Kernels and graph compilation: where the cost model meets the GPU.
Also called MegaMoE expert-parallel serving backend, MegaMoE MoE backend, MegaMoE MoE execution backend.