Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

Multiplicative aggregation of format signals

A post-training reward scheme in which per-output format signals are combined with the environment task reward.1

Trade-off The aggregation lets a single format signal significantly affect the final reward while keeping environment reward the primary training signal.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Format signals act multiplicatively on environment reward — tech report
  2. allows a single format signal to have a significant impact on the final reward, while ensuring that environment reward remains the primary training signal — tech report

Also called act multiplicatively on environment reward.

Filed under post-training › reward modelling