single source · post-training
Direct Double-sided Importance Sampling
Direct Double-sided Importance Sampling is a token-level importance sampling method for asynchronous RL that reuses rollout log-probabilities as a direct behavior proxy, avoiding separate old-policy inference.1 It replaces PPO's asymmetric clipping with a double-sided trust region, masking tokens outside the interval from gradient computation to prevent instability from extreme policy divergence.2
Trade-off It accepts a controlled degree of off-policy bias in exchange for avoiding historical policy tracking while boosting training stability.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
reuses the log-probabilities generated during rollout as a direct behavior proxy
— GLM-5: from Vibe Coding to Agentic EngineeringInstead of the asymmetric clipping used in standard PPO, we restrict the trust region to
— GLM-5: from Vibe Coding to Agentic Engineeringreusing rollout log-probabilities accepts a controlled degree of off-policy bias to circumvent the need for historical policy tracking while boosting training stability
— GLM-5: from Vibe Coding to Agentic Engineering
Filed under post-training › reinforcement learning algorithm