single mention · inference & serving
Tensor parallelism for MoE layers
Adds tensor parallelism support specifically to MoE layers so the model can run across GPUs.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See serving parallelism for the methods filed beside it.
Also called tensor parallelism in MoE.
Filed under inference & serving › serving parallelism