Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

Group-relative length bonus

A group-relative length bonus augments the task reward during RL to reduce verbosity and inference costs while preserving task performance.1 For each prompt the policy samples a group of rollouts, and only correct rollouts receive a length-based bonus, computed relative to the shortest correct rollout in the group.2,3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. To reduce verbosity and inference costs while preserving task performance, we augment the task reward with a group-relative length bonus for correct rollouts — technical report
  2. the policy samples a group ofG rollouts{y1,...,y G}, each with correctness scoreci and lengthLi — technical report
  3. For nonempty C, let Lmin = mini∈CLi — technical report

Filed under post-training › reward modelling