Open Models Almanac
Methodsoptimizationtraining runtime

specific method · filed under optimization

GPU-to-inference weight sync over GPUDirect RDMA

Transfers updated trainer weights to inference replicas using NCCL point-to-point communication over GPUDirect RDMA, without host-memory staging.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training runtime for the methods filed beside it.

Also called GPU-to-GPU weight transfer, RDMA weight transfer.

sources
2
models
2
labs adopt it
2
strongest
used

Documented in

Aleph Alpha
Poolside

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2

Kolibri-1

Aleph Alphaadopted1 source

After every optimiser step, the trainer pushes updated weights to the inference replicas, as shown in Figure 37.

usedsoftware implementationin Kolibri RL training runAleph Alpha

Laguna-S-2.1

Poolsideadopted1 source

The weight sync uses NCCL point-to-point primitives over GPUDirect RDMA with no intermediate offload to host memory or remote object storage.

usedinference servingin LAGUNA M.1/XS.2Poolside
↑