Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

p-norm aggregation of format signals

P-norm aggregation of format signals combines multiple format-quality signals into a single multiplier on the environment reward.1 Each signal is rescaled to a configurable strength, and the norm p controls how quickly simultaneous format defects compound in reducing the reward.2

Trade-off It lets a single format defect significantly lower the reward while guaranteeing the environment reward remains the primary training signal.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The final reward Ri of environment i is computed by scaling the environment reward Renv i > 0 with a p-norm aggregation of format signals — tech report
  2. the aggregation norm p controls how quickly simultaneous defects compound — tech report
  3. allows a single format signal to have a significant impact on the final reward, while ensuring that environment reward remains the primary training signal — tech report

Filed under post-training › reward modelling