Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

Offline trajectory auditing for reward hacking

Offline trajectory auditing for reward hacking regularly inspects agent trajectories produced during training for reward hacking.1 It catches shortcuts the evolving policy finds that adversarial screening missed, and the observations guide environment cleanup and access restrictions.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Weregularlyauditagenttrajectoriesofflineforrewardhackingthrough- outtraining — MiMo V2 6 technical report
  2. Asthepolicyevolves, itmaydiscovershortcutsthatwerenotfoundduringadversar- ial screening. We use these observations to identify weaknesses in the environments and guide further cleanup and access restrictions — MiMo V2 6 technical report

Also called Training-Time Auditing.

Filed under post-training › reward modelling