specific method · filed under post-training
Off-Policy Sequence Masking
Masking negative sequences that exhibit significant policy divergence, measured by KL divergence from the current policy.
Also called Off-policy sequence masking for GRPO.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
3 spans quoted from the sources, strongest treatment first.
we mask negative sequences that introduce significant policy divergence, as measured by the KL divergence between the data-sampling policy πold and the current policy πθ.
Off-Policy Sequence Masking To improve the efficiency of RL systems, we typically generate a large batch of rollout data, which is subsequently split into multiple mini-batches for several gradient update steps.
we mask negative sequences that introduce significant policy divergence, as measured by the KL divergence between the data-sampling policy and the current policy.
Filed alongside
Other methods under post-training :: reinforcement learning algorithm.