Kolibri-1
Aleph AlphaAfter every optimiser step, the trainer pushes updated weights to the inference replicas, as shown in Figure 37.
specific method · filed under optimization
Transfers updated trainer weights to inference replicas using NCCL point-to-point communication over GPUDirect RDMA, without host-memory staging.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training runtime for the methods filed beside it.
Also called GPU-to-GPU weight transfer, RDMA weight transfer.
2 quotes from 2 sources, one card per model, each opening with its strongest.
After every optimiser step, the trainer pushes updated weights to the inference replicas, as shown in Figure 37.
The weight sync uses NCCL point-to-point primitives over GPUDirect RDMA with no intermediate offload to host memory or remote object storage.