implementation detail · filed under inference & serving
Topology-aware NVLink domain placement
Topology-aware NVLink domain placement is a worker and rank placement method for GPU clusters that uses NVLink domain membership to co-locate GPUs of a group on the same rack.1 Without it, Expert Parallelism groups can silently span racks, forcing MoE all-to-all communication over InfiniBand instead of NVLink.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
make rank assignment domain-aware and ensure all GPUs within an EP group are co-located on the same rack
— NVIDIA Nemotron 3 Ultra Technical ReportEP) groups can span multiple racks, forcing MoE all-to-all communication over InfiniBand instead of NVLink
— NVIDIA Nemotron 3 Ultra Technical Report