Kolibri-1
Aleph Alphawe compute the group-relative advantage of each segment as
specific method · filed under post-training
Group-relative advantage estimation computes the advantage of a segment by subtracting the group mean reward from its training reward.1 This method calculates the mean reward across a group of sequences to provide a relative advantage for each segment.2
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Given a groupG of sequences yi, we compute the group-relative advantage of each segment as Ai,e = Ri,e− ¯R— tech report
The mean reward ¯R is computed on a sequence level, over the Ej segments of each sequence yj, to prevent segment splits from overweighting individual sequences— tech report
Also called group-relative advantages, Group-relative reward advantages.
2 quotes from 2 sources, one card per model, each opening with its strongest.
we compute the group-relative advantage of each segment as
A rollout’s advantage is its reward minus the group mean, and advantages are standardized across the batch.