Generative Reward Models (GenRM) (Wang2025HelpSteer3Preference) with reasoning capabilities help mitigate such reward hacking behaviors
2 more from this model
Nemotron 3 Ultra 550B-A55B GenRM: GenRM used for RLHF
GenRM used for RLHF
general family · filed under post-training
A Generative Reward Model uses the actor network to function as both the generator and the judge, scoring its own outputs against structured rubrics.1 It addresses hard-to-verify tasks by using structured rubrics to evaluate policy trajectories, fusing evaluation into the generative process.2
Trade-off The method achieves superior performance while requiring only a minimal set of diverse human annotations.3
Summarised by glm-5.3-flash from 2 sources; every sentence is checked against the quote it cites.
The actor network functions as both the generator and the judge, scoring its own outputs against structured rubrics— DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)
Instead, to address hard-to-verify tasks, we curate rubric-guided RL data and employ a Generative Reward Model (GRM) to evaluate policy trajectories— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
achieving "superior performance with only a minimal set of diverse human annotations— DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)
Also called GenRM, Generative Reward Model (GRM), rubric-conditioned generative reward model.
7 quotes from 6 sources, one card per model, each opening with its strongest.
Generative Reward Models (GenRM) (Wang2025HelpSteer3Preference) with reasoning capabilities help mitigate such reward hacking behaviors
Nemotron 3 Ultra 550B-A55B GenRM: GenRM used for RLHF
GenRM used for RLHF
we curate rubric-guided RL data and employ a Generative Reward Model (GRM) to evaluate policy trajectories.
Generative Reward Model replaces scalar reward models
we curate rubric-guided RL data and employ a Generative Reward Model (GRM) to evaluate policy trajectories.
We then develop detailed evaluation rubrics across multiple quality dimensions and employ a generative reward model to score responses based on these rubrics.
Picked by hand, not extracted: where to read more, not evidence for anything on this page.