Open Models Almanac
Methodspost-trainingreinforcement learning algorithm

implementation detail · filed under post-training

IcePop

IcePop is a reinforcement-learning technique that mitigates the training-inference mismatch between the policy used for sampling trajectories and the policy used for gradient updates.1 It works by suppressing samples whose training-inference mismatch ratio deviates excessively.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. incorporates the IcePop technique [61] to mitigate the training-inference mismatch, i.e., the discrepancy between the inference distribution and the training distribution during RL optimization — GLM-5: from Vibe Coding to Agentic Engineering
  2. The operator suppresses samples whose mismatch ratio deviates excessively — GLM-5: from Vibe Coding to Agentic Engineering

Also called IcePop technique, token-level masking using IcePop strategy.

sources
2
models
2
labs adopt it
2
strongest
used

Documented in

Evidence

3 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 3

where represents token-level masking using IcePop strategy (ring1t)

usedoptimizationin Nemotron 3 UltraNVIDIA
1 more from this model

icePop strategy (ring1t)

usedoptimizationin Nemotron 3 UltraNVIDIA
↑