single mention · optimization
Off-policy sample filtering
Off-policy sample filtering discards training samples that are too far behind the current policy or produced by environment failures.1 A sample is dropped when its oldest rollout model version is stale beyond a threshold, and samples failing due to environment collapse are excluded, with incomplete groups padded or dropped.2,3,4
Trade-off It removes spurious reward noise and improves training stability, at the cost of discarding some trajectories.5,6
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
To filter out these severely off-policy samples, we log the policy weight version used by the rollout engine at generation time
— GLM-5: from Vibe Coding to Agentic EngineeringWe discard a sample if its oldest rollout version is too stale
— GLM-5: from Vibe Coding to Agentic Engineeringexclude samples that fail due to environment collapse
— GLM-5: from Vibe Coding to Agentic Engineeringwe pad the group by repeating valid samples if the number of valid samples exceeds half of the group size; otherwise, we drop the entire group
— GLM-5: from Vibe Coding to Agentic EngineeringThis procedure reduces spurious reward noise and improves training stability
— GLM-5: from Vibe Coding to Agentic EngineeringThis removes trajectories that lag too far behind the current policy
— GLM-5: from Vibe Coding to Agentic Engineering
Also called Dropping off-policy and noisy samples.
Filed under optimization › training stability