Open Models Almanac
Methodsinference & servingserving parallelism

implementation detail · filed under inference & serving

Topology-aware NVLink domain placement

Topology-aware NVLink domain placement is a worker and rank placement method for GPU clusters that uses NVLink domain membership to co-locate GPUs of a group on the same rack.1 Without it, Expert Parallelism groups can silently span racks, forcing MoE all-to-all communication over InfiniBand instead of NVLink.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. make rank assignment domain-aware and ensure all GPUs within an EP group are co-located on the same rack — NVIDIA Nemotron 3 Ultra Technical Report
  2. EP) groups can span multiple racks, forcing MoE all-to-all communication over InfiniBand instead of NVLink — NVIDIA Nemotron 3 Ultra Technical Report
sources
2
model
1
lab adopt it
1
strongest
used

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted2 sources

The fix was to make rank assignment domain-aware and ensure all GPUs within an EP group are co-located on the same rack.

usedsoftware implementationin NeMo-RLNVIDIA
1 more from this model

The fix was to make rank assignment domain-aware and ensure all GPUs within an EP group are co-located on the same rack.

usedsoftware implementationin Nemotron 3 UltraNVIDIA
↑