Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

Quantized group-relative rollout-length bonus

A length bonus that rewards correct rollouts for being shorter, added to the task reward before computing the group-normalized advantage.1 It quantizes the lengths of correct rollouts into bins anchored at the shortest correct rollout, assigns larger bonuses to shorter bins, and scales everything relative to the group, so no per-task length target is needed.2,3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The bonus is added to the task reward before computing the group-normalized advantage — technical report
  2. we quantize rollout lengths into bins of width ∆ = 0.1Lmin, anchored atLmin — technical report
  3. Because both quantization and normalization are relative to the correct rollouts within each group, this scheme requires no per-task length target — technical report

Filed under post-training › reward modelling