taxonomy node · level 2

reinforcement learning algorithm

62 methods filed at this node or below it, from the sources of 18 models.

post-training :: reinforcement learning algorithm

Matching aids for the classifier: GRPO; group relative policy optimization; MLAPO; IcePop; pivot RL; direct double-sided importance sampling; rollout routing replay.

By model

Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Model	techniques
DeepSeek-V4.1-Flash core	Effort-Conditioned Reinforcement Learning coreConcatenated Routing Replay usedDerived-Latency Penalty usedEffort-Dependent Length Penalty usedExponentially Decaying Token-Penalty Coefficient usedFailure Replay and Targeted Reinforcement Learning usedGroup-Relative Advantage Normalization Within Effort-Level Subgroups usedProgressive Scaling of RL Data, Tasks, and Rollouts usedRL Reward with Collaboration Bonus and Derived-Latency Penalty usedReinforcement Learning Post-Training mentioned—
NVIDIA-Nemotron-3-Ultra-550B-A55B used	Abstention Training usedDynamic Abstention-Reward Calibration usedGroup Relative Policy Optimization usedIcePop usedNegative-Advantage Penalties for Malformed Reasoning and Tool Calls usedPivot Reinforcement Learning usedPPO-Style Policy-Ratio Clipping usedReinforcement Learning usedReinforcement Learning from Verifiable Rewards used—
MiMo-V2.6-Flash core	Asynchronous Group Relative Policy Optimization coreGroupwise Advantage Redistribution corePenalty Module coreDropping All-Zero-Advantage Groups defaultAdvantage Shaping usedDomain-Specific GRPO Training usedDynamic Sampling usedFlagged-Token Masking and Penalty usedFreezing the MoE Router During Reinforcement Learning usedGroup-Relative Length Penalty usedHierarchical Penalty Escalation usedKeep Candidate-Set Replay usedQuality-Weighted Advantage Redistribution usedRollout Routing Replay usedSingle-Harness Reinforcement Learning usedZero-Reward Correction for Leaked or External Answers used—
DeepSeek-V4-Flash used	Group Relative Policy Optimization used—
MiMo-V2.5 used	Group Relative Policy Optimization usedReinforcement Learning usedRollout Routing Replay used—
GLM-5.3 used	SAO with Compaction used—
Hy3 used	Group Relative Policy Optimization usedReinforcement Learning usedReinforcement Learning Post-Training used—
GLM-5.2 used	Direct Double-Sided Importance Sampling usedGroup Relative Policy Optimization usedGroup-Wise Policy Optimization usedIcePop used—
DeepSeek-V3.2 core	Group Relative Policy Optimization coreKeep Routing coreMixed Reinforcement Learning coreKeep Sampling Mask usedOff-Policy Sequence Masking usedReinforcement Learning Training usedUnbiased KL Estimate used—
DeepSeek-V4-Pro used	Group Relative Policy Optimization usedReinforcement Learning used—
Inkling used	Large-Scale Reinforcement Learning usedReinforcement Learning for Calibration with Proper Scoring Rules usedRL with Rubric and Claims Graders used—
Kimi K3 core	Multi-Domain, Multi-Effort Reinforcement Learning coreDomain-Specialized RL Experts used—
Laguna-S-2.1 core	CISPO with Length-Weighted Leave-One-Out Group-Relative Advantages coreAgentic RL Task Mix usedPer-Token Tool-Error Step Penalty usedReinforcement Learning from Code Execution Feedback usedMoonlight Scaling optional—
MiMo-V2.5-Pro used	Reinforcement Learning used—
MiMo-V2.6-Pro core	Penalty Module coreDropping All-Zero-Advantage Groups defaultAdvantage Shaping usedAsynchronous Group Relative Policy Optimization usedDomain-Specific GRPO Training usedDynamic Sampling usedFlagged-Token Masking and Penalty usedFreezing the MoE Router During Reinforcement Learning usedGroup-Relative Length Penalty usedGroupwise Advantage Redistribution usedHierarchical Penalty Escalation usedKeep Candidate-Set Replay usedQuality-Weighted Advantage Redistribution usedRollout Routing Replay usedSingle Mixed Reinforcement-Learning Run usedSingle-Harness Reinforcement Learning usedZero-Reward Correction for Leaked or External Answers used—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B used	Group Relative Policy Optimization usedGRPO with Masked Importance Sampling used—
Qwen3.5-397B-A17B used	GSPO usedInteraction-Aligned Reinforcement Learning usedProgressively Complex Task Distributions used—
gpt-oss-120b core	Chain-of-Thought Reinforcement Learning coreHelpful-Only Adversarial Reinforcement Learning usedIncremental Reinforcement Learning used—

reinforcement learning algorithm

In this branch 62

filed here 62

By model