The fix was to explicitly bind policy and vLLM worker processes to the CPU socket local to their assigned GPUs.
1 more from this model
The fix was to explicitly bind policy and vLLM worker processes to the CPU socket local to their assigned GPUs.
specific method · filed under inference & serving
NUMA binding pins worker processes to the CPU socket local to their assigned GPUs.1 It prevents cross-socket placement, which would force memory traffic over the inter-socket coherence link and degrade GPU memory bandwidth.2
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
explicitly bind policy and vLLM worker processes to the CPU socket local to their assigned GPUs— NVIDIA Nemotron 3 Ultra Technical Report
cross- socket placement forces memory traffic to traverse the inter-socket coherence link before reaching NVLink-C2C, degrading GPU memory bandwidth— NVIDIA Nemotron 3 Ultra Technical Report
Also called NUMA Binding for CPU-GPU affinity, NUMA Binding.
2 quotes from 2 sources, one card per model, each opening with its strongest.
The fix was to explicitly bind policy and vLLM worker processes to the CPU socket local to their assigned GPUs.
The fix was to explicitly bind policy and vLLM worker processes to the CPU socket local to their assigned GPUs.