Open Models Almanac
Methodspost-trainingreinforcement learning algorithm

specific method · filed under post-training

Dropping zero-advantage rollout groups

Removes rollout groups whose sequences all have the same reward and therefore provide no advantage signal.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See reinforcement learning algorithm for the methods filed beside it.

sources
2
models
3
labs adopt it
2
strongest
default

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1default 1

Mistral-Large-4

Mistral AIadopted1 source

When every rollout in a group gets the same reward, all its advantages are zero and it carries no learning signal. We drop such groups before they reach the trainer

usedunclearin Mistral Large 4Mistral AI
↑