Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

Safety reward

A safety reward is a reward signal used during post-training that penalizes unsafe or physically implausible motion trajectories.1 It complements reasoning and consistency rewards, which improve interpretability but do not explicitly constrain the model to produce safe trajectories.2

Trade-off It reduces the close encounter rate and stabilizes trajectory generation without compromising reasoning quality.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we introduce a safety reward that penalizes unsafe or physically implausible trajectories during post-training — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
  2. reasoning and consistency rewards improve interpretability and causal grounding, they do not explicitly constrain the model to produce safe motion trajectories — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
  3. adding the safety reward further reduces the close encounter rate and stabilizes trajectory generation without compromising reasoning quality — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail

Filed under post-training › reward modelling