implementation detail · filed under inference & serving
Topology-aware NVLink domain placement
Assigns ranks with hardware topology in mind, including co-locating GPUs in an expert-parallel group on the same rack.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
The fix was to make rank assignment domain-aware and ensure all GPUs within an EP group are co-located on the same rack.
usedsoftware implementationin NeMo-RLNVIDIA
The fix was to make rank assignment domain-aware and ensure all GPUs within an EP group are co-located on the same rack.
usedsoftware implementationin Nemotron 3 UltraNVIDIA
Filed alongside
Other methods under inference & serving :: serving parallelism.
Attention data parallelismDeepEPPrefill-decode disaggregationEncoder-Prefill-Decode disaggregationTensor parallelism (degree 4)Tensor parallelism (degree 8)Zero-copy fused token permutation and unpermutationData-parallel vision encodingExpert parallelismFused reduce-scatter/all-gather collectivesIdentical cache-layout pinning across prefill and decode poolsLow-precision MoE combineRound-robin routing for prefill-decode disaggregationTensor parallelism for MoE layersTensorRT-LLM all-reduce backendToken migration for balanced expert placement