Open Models Almanac
Methodspost-trainingreinforcement learning algorithm

specific method · filed under post-training

Dynamic sampling of informative rollout groups

Dynamic sampling drops rollout groups in which all sequences receive the same environment reward, since these give no learning signal.1 Dropped groups are resampled from the same environment so that environments with high drop rates are not under-represented in training.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. drop groups in which all sequences obtain the same environment reward — tech report
  2. We re-sample dropped groups two additional times from the same environment to keep environments with high drop rates from being under-represented in training — tech report

Also called dynamic sampling.

sources
2
models
3
labs adopt it
2
strongest
used

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2

Kolibri-1

Aleph Alphaadopted1 source

we generally drop groups in which all sequences obtain the same environment reward.

usedoptimizationin KolibriAleph Alpha
↑