implementation detail · filed under inference & serving
Tensor parallelism (degree 8)
Uses tensor parallelism with degree eight for serving.
Also called tensor-parallel serving on 8 GPUs, tensor parallel on 8 GPUs, Tensor parallelism with size 8, tensor-parallel-size 8.
- sources
- 2
- models
- 2
- labs adopt it
- 2
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
optional 1used 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
--tensor-parallel-size 8
usedinference servingin Hy4-previewvLLM
using tensor parallel on 8 GPUs
optionalinference servingin Qwen3.5-397B-A17BQwen
Filed alongside
Other methods under inference & serving :: serving parallelism.
Attention data parallelismDeepEPPrefill-decode disaggregationEncoder-Prefill-Decode disaggregationTensor parallelism (degree 4)Topology-aware NVLink domain placementZero-copy fused token permutation and unpermutationData-parallel vision encodingExpert parallelismFused reduce-scatter/all-gather collectivesIdentical cache-layout pinning across prefill and decode poolsLow-precision MoE combineRound-robin routing for prefill-decode disaggregationTensor parallelism for MoE layersTensorRT-LLM all-reduce backendToken migration for balanced expert placement