Open Models Almanac
Methodsinference & servingserving parallelism

single mention · inference & serving

Tensor parallelism for MoE layers

Adds tensor parallelism support specifically to MoE layers so the model can run across GPUs.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See serving parallelism for the methods filed beside it.

Also called tensor parallelism in MoE.

Filed under inference & serving › serving parallelism