tensor parallelism (TP, which shards the weight matrices of each linear layer across ranks)
3 more from this model
wide TP the practical choice for low-latency serving
8-way tensor parallelism
--tensor-parallel-size 4
general family · filed under optimization
Tensor parallelism partitions tensor computations across ranks; the cited degrees are configuration details.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training parallelism for the methods filed beside it.
7 quotes from 6 sources, one card per model, each opening with its strongest.
tensor parallelism (TP, which shards the weight matrices of each linear layer across ranks)
wide TP the practical choice for low-latency serving
8-way tensor parallelism
--tensor-parallel-size 4
--tensor-parallel-size 2
Choose one of the following parallel configurations for a single node with four B300 GPUs:
Picked by hand, not extracted: where to read more, not evidence for anything on this page.