specific method · filed under post-training
Unbiased KL Estimate
An unbiased KL estimate obtained by correcting the K3 estimator with the importance-sampling ratio between current and old policies.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
3 spans quoted from the sources, strongest treatment first.
we correct the K3 estimator (Schulman, 2020) to obtain an unbiased KL estimate using the importance-sampling ratio between the current policy πθ and the old policy πold.
we correct the K3 estimator (Schulman, 2020) to obtain an unbiased KL estimate using the importance-sampling ratio between the current policy πθ and the old policy πold.
we correct the K3 estimator (Schulman, 2020) to obtain an unbiased KL estimate using the importance-sampling ratio between the current policy and the old policy.
Filed alongside
Other methods under post-training :: reinforcement learning algorithm.