single mention · post-training
Offline trajectory auditing for reward hacking
Offline trajectory auditing for reward hacking regularly inspects agent trajectories produced during training for reward hacking.1 It catches shortcuts the evolving policy finds that adversarial screening missed, and the observations guide environment cleanup and access restrictions.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Weregularlyauditagenttrajectoriesofflineforrewardhackingthrough- outtraining
— MiMo V2 6 technical reportAsthepolicyevolves, itmaydiscovershortcutsthatwerenotfoundduringadversar- ial screening. We use these observations to identify weaknesses in the environments and guide further cleanup and access restrictions
— MiMo V2 6 technical report
Also called Training-Time Auditing.
Filed under post-training › reward modelling