Open Models Almanac
Methodspost-trainingreward modelling

methods · post-training

reward modelling

68 methods filed at this node or below it, from the sources of 19 models.

In this branch 68

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 68

Groupwise Agentic Grading core · 3 sources · 4 quotes
Groupwise Reward Synthesis core · 3 sources · 4 quotes
Generative Reward Model used · 6 sources · 7 quotes
Adversarial-agent reward-hacking audit used · 4 sources · 4 quotes
Length penalty used · 4 sources · 4 quotes
Language-consistency reward used · 3 sources · 3 quotes
Reasoning-length penalty used · 2 sources · 2 quotes
Verifier cross-checking used · 2 sources · 2 quotes
59 single mentions
Binary task verifier core · 1 source · 1 quote
Deterministic reward-checking chain core · 1 source · 1 quote
p-norm aggregation of format signals core · 1 source · 1 quote
Per-source composite reward functions core · 1 source · 1 quote
Role-centered reward baselines core · 1 source · 1 quote
Role-conditioned Merlin–Arthur rewards core · 1 source · 1 quote
Sparse terminal-verifier reward core · 1 source · 1 quote
Conservative failure attribution default · 1 source · 1 quote
Abstention-aware factual QA reward used · 1 source · 1 quote
Agentic Generative Reward Model used · 1 source · 1 quote
Auxiliary tool-use reward used · 1 source · 1 quote
Behavior rubrics used · 1 source · 1 quote
Binary answer-correctness reward used · 1 source · 1 quote
Continuous tracking-quality reward used · 1 source · 1 quote
Deterministic task-verifier rewards used · 1 source · 2 quotes
Fail-to-pass test verifier construction used · 1 source · 1 quote
Group-relative length bonus used · 1 source · 1 quote
Hybrid reward system used · 1 source · 1 quote
Matched think-block format validation used · 1 source · 2 quotes
Monitoring-only penalty strategy used · 1 source · 1 quote
Multi-level reward formulation used · 1 source · 1 quote
Multiplicative p-norm format reward used · 1 source · 1 quote
Outcome Reward Model used · 1 source · 1 quote
Pairwise rewards with groupwise penalties used · 1 source · 1 quote
Per-token tool-error reward shaping used · 1 source · 1 quote
Public and hidden verifier pairing used · 1 source · 1 quote
Reasoning reward from LRM feedback used · 1 source · 1 quote
Reasoning-action consistency reward used · 1 source · 2 quotes
Reasoning-to-action verification used · 1 source · 1 quote
Regression-adjusted test-pass reward used · 1 source · 1 quote
Reward-scale calibration used · 1 source · 1 quote
Rule-based outcome reward used · 1 source · 1 quote
Rule-based verifiers used · 1 source · 1 quote
Safety reward used · 1 source · 1 quote
Scientific-tool answer cross-checking used · 1 source · 1 quote
Segment-level behavioral penalties used · 1 source · 1 quote
Solution-quality rubrics used · 1 source · 1 quote
Test-case pass-fraction reward used · 1 source · 1 quote
Test-difficulty-driven code reward used · 1 source · 1 quote
Trajectory-quality reward used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
DeepSeek-V4.1-Flash usedReasoning-length penalty used—
MiMo-V2.6-Flash coreGroupwise Agentic Grading coreGroupwise Reward Synthesis coreAdversarial-agent reward-hacking audit usedBehavior rubrics usedFive-dimension comparative grading of passing patches usedMonitoring-only penalty strategy usedMultiplicative reward synthesis from test and rubric scores usedNegative rubric checks for unintended side effects usedOffline trajectory auditing for reward hacking usedRule-based or model-judged trajectory error detection usedSegment-level behavioral penalties usedSolution-quality rubrics usedVerifier cross-checking used—
NVIDIA-Nemotron-3-Ultra-550B-A55B usedGenerative Reward Model usedLength penalty usedPrinciple-conditioned generative reward model used—
DeepSeek-V4-Flash usedDirect RL optimization of a generative reward model usedGenerative Reward Model used—
Alpamayo 1.5 / 2 Super usedReasoning reward from LRM feedback usedReasoning-action consistency reward usedSafety reward usedTrajectory-quality reward used—
DeepSeek-V3.2 usedGenerative Reward Model usedLanguage-consistency reward usedLength penalty usedRule-based outcome reward used—
DeepSeek-V4-Pro usedDirect RL optimization of a generative reward model usedGenerative Reward Model used—
GLM-5.2 usedHybrid reward system usedMulti-level reward formulation used—
Inkling usedAbstention-aware factual QA reward used—
Kimi K3 usedAgentic Generative Reward Model usedPublic and hidden verifier pairing usedWeb-development reward with deterministic checks and model judging used—
Kolibri-1 coreEnvironment-specific output-format reward signals coreMultiplicative aggregation of format signals corep-norm aggregation of format signals coreReinforcement Learning with Verifiable Rewards coreRole-centered reward baselines coreRole-conditioned Merlin–Arthur rewards coreFormat reward conditioned on positive environment reward usedHallucination mitigation through abstention and context-sensitive RL usedKillable, time-limited verifier process pool usedLanguage-consistency reward usedMerlin–Arthur correctness and abstention reward usedMerlin–Arthur hallucination and abstention example construction usedMultiplicative p-norm format reward usedReasoning-to-action verification usedRegression-adjusted test-pass reward usedTest-case pass-fraction reward used—
LFM3-1.7B usedLength penalty used—
Laguna-S-2.1 coreBinary task verifier coreDeterministic reward-checking chain coreSparse terminal-verifier reward corePer-token tool-error reward shaping used—
MiMo-V2.5 usedOutcome Reward Model usedRule-based verifiers usedTest-difficulty-driven code reward used—
MiMo-V2.5-Pro usedRule-based verifiers usedTest-difficulty-driven code reward used—
MiMo-V2.6-Pro coreGroupwise Agentic Grading coreAdversarial-agent reward-hacking audit usedBehavior rubrics usedFive-dimension comparative grading of passing patches usedGroupwise Reward Synthesis usedMonitoring-only penalty strategy usedMultiplicative reward synthesis from test and rubric scores usedNegative rubric checks for unintended side effects usedOffline trajectory auditing for reward hacking usedRule-based or model-judged trajectory error detection usedSegment-level behavioral penalties usedSolution-quality rubrics usedVerifier cross-checking used—
Mistral-Large-4 corePer-source composite reward functions coreConservative failure attribution defaultAdversarial-agent reward-hacking audit usedComposable multi-method reward verification usedExploit-neutralization and regression-free patch reward usedFail-to-pass test verifier construction usedGroup-relative length bonus usedPairwise rewards with groupwise penalties usedQuantized group-relative rollout-length bonus usedReward-scale calibration usedScientific-tool answer cross-checking usedReinforcement Learning with Verifiable Rewards optional—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B usedAuxiliary tool-use reward usedBinary answer-correctness reward usedContinuous tracking-quality reward usedCorrect-response-only soft length preference usedDeterministic task-verifier rewards usedMatched think-block format validation usedReasoning-length penalty usedReinforcement Learning with Verifiable Rewards used—
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B usedAuxiliary tool-use reward usedBinary answer-correctness reward usedContinuous tracking-quality reward usedCorrect-response-only soft length preference usedDeterministic task-verifier rewards usedMatched think-block format validation usedReasoning-length penalty usedReinforcement Learning with Verifiable Rewards used—
↑