specific method · filed under inference & serving
Tensor parallelism for MoE layers
Adds tensor-parallel support to MoE layers so larger models can run across GPUs.
Also called tensor parallelism in MoE.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
with a small addition of supporting tensor parallelism in MoE so that the larger model can run with this code (e.g., on 4xH100 or 2xH200)
usedsoftware implementationin gpt-oss reference implementationsOpenAI
Filed alongside
Other methods under inference & serving :: serving parallelism.
Attention data parallelismDeepEPPrefill-decode disaggregationEncoder-Prefill-Decode disaggregationTensor parallelism (degree 4)Tensor parallelism (degree 8)Topology-aware NVLink domain placementZero-copy fused token permutation and unpermutationData-parallel vision encodingExpert parallelismFused reduce-scatter/all-gather collectivesIdentical cache-layout pinning across prefill and decode poolsLow-precision MoE combineRound-robin routing for prefill-decode disaggregationTensorRT-LLM all-reduce backendToken migration for balanced expert placement