Qwen3.8-Flash-Next
Alibaba (Qwen)Use TEP8 with the Triton MoE backend on Hopper
specific method · filed under optimization
TEP keeps attention tensor-parallel and places the routed experts whole on the GPUs. Memory per GPU, the copied single-head cache and the traffic stay those of tensor parallelism; the expert matrix multiplications run on whole experts instead of slices, which keeps quantization blocks intact.
Written for the explainer Parallelism for serving large MoE models: what each layout copies, and what it sends.
Also called Combined tensor and expert parallelism, TEP8.
2 quotes from 2 sources, one card per model, each opening with its strongest.
Use TEP8 with the Triton MoE backend on Hopper
For TEP8 or DEP8, retain `--enable-expert-parallel` and remove `--moe-backend deep_gemm_mega_moe`