Open Models Almanac
Methodspost-trainingreinforcement learning algorithm

general family · filed under post-training

Reinforcement learning

Reinforcement learning is a post-training method that optimizes a model against explicit reward signals on its own rollouts rather than on fixed demonstrations.1 Unlike supervised fine-tuning, it gives targeted feedback on test-time inference errors, improving robustness and generalization.2

Trade-off It yields larger robustness and generalization gains than SFT for the same compute budget.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. RL provides explicit inference feedback on the model’s own rollouts, aligning the optimization objective with how the system is actually deployed — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
  2. Unlike SFT, which optimizes the likelihood of expert demonstrations under teacher forcing without feedback on the test-time inference errors — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
  3. yields disproportionately larger gains in robustness and generalization for the same compute budget — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail

Also called RL, Reinforcement Learning (RL), advanced reinforcement learning techniques, reinforcement learning (RL) training, strengthened reinforcement learning, RL training.

sources
20
models
11
labs adopt it
6
strongest
used

Documented in

Evidence

23 quotes from 20 sources, one card per model, each opening with its strongest.

How sources treat it
mentioned 1optional 1used 21

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted5 sources

Post-trained with enhanced pipeline involving Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD) for improved model accuracy.

usedpost trainingin Nemotron 3 UltraNVIDIA
5 more from this model

post-trained using Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation

usedpost trainingin Nemotron 3 UltraNVIDIA

advanced reinforcement learning techniques with concurrent multi-environment post-training at scale

usedpost trainingin Nemotron 3 familyNVIDIA

post-trained using SFT, RL, and MOPD

usedpost trainingin Nemotron 3 UltraNVIDIA

multi-environment reinforcement learning using asynchronous GRPO

usedtraining objectivein Nemotron 3 UltraNVIDIA

post-trained using ... Reinforcement Learning (RL)

usedpost trainingin Nemotron 3 UltraNVIDIA

Hy3

Tencent (Hunyuan)adopted3 sources

through strengthened reinforcement learning and enhanced data quality and diversity

usedtraining objectivein Hy3Tencent
3 more from this model

Through joint optimization of SFT and RL

usedpost trainingin Hy3Tencent Hy Team

scaling up RL training

usedpost trainingin Hy3Tencent Hy Team

RL Post-training

usedpost trainingin Hy3Tencent Hy Team

Mistral-Large-4

Mistral AIadopted2 sources

we've put that feedback to work while continuing to scale reinforcement learning, driving massive improvements in coding and defensive cybersecurity.

usedunclearin Mistral Large 4Mistral AI
1 more from this model

We use Reinforcement Learning (RL) because it adapts as the model does: we train on the outcomes of the model's own attempts

usedpost trainingin Mistral Large 4Mistral AI

then post-trained with roughly 3.8 million samples and reinforcement learning across dozens of environments.

usedpost trainingin Nemotron 3.5 Super VLNVIDIA
1 more from this model

Reinforcement learning (RL) improves performance in agentic environments that reflect organizational workflows.

optionalpost trainingin Nemotron 3.5 Super VLNVIDIA

Alpamayo 1.5 / 2 Super

NVIDIAadopted2 sources

we apply RL-based post-training to simultaneously improve reasoning quality, reasoning-action consistency, and trajectory quality

usedunclearin Alpamayo-R1NVIDIA
2 more from this model

employs reinforcement learning (RL) to boost the reasoning quality, reasoning-action consistency and trajectory quality

usedunclearin Alpamayo-R1NVIDIA

is RL post-trained

usedpost trainingin Alpamayo 1.5NVIDIA

DeepSeek-V3.2

DeepSeekadopted1 source

These performance gains can be attributed to the increased computational resources allocated to RL training.

usedoptimizationin DeepSeek-V3.2DeepSeek

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑