Open Models Almanac
Methodspost-trainingreward modelling

single mention · post-training

Hybrid reward system

A hybrid reward system combines multiple types of reward signals — rule-based functions, outcome reward models, and generative reward models — to supervise RL training across diverse objectives.1 It blends them because each type has distinct strengths and weaknesses, balancing precision, efficiency, and robustness so no single component's weaknesses dominate.2

Trade-off Rule-based rewards are precise but narrow, ORMs are low-variance but hackable, and GRMs are robust but high-variance; blending them trades these off against each other.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we build a hybrid reward system that integrates three complementary types of reward signals: rule-based reward functions, outcome reward models (ORMs), and generative reward models (GRMs — GLM-5: from Vibe Coding to Agentic Engineering
  2. By blending these three signal types, we obtain a reward system that balances precision, efficiency, and robustness, mitigating the weaknesses of any single component — GLM-5: from Vibe Coding to Agentic Engineering
  3. Rule-based rewards provide precise and interpretable signals, but are limited to aspects expressible as deterministic rules. ORMs offer low-variance signals and high training efficiency, but are more susceptible to reward hacking — GLM-5: from Vibe Coding to Agentic Engineering

Filed under post-training › reward modelling