DeepSeek-V3.2
DeepSeekFor DeepSeek-V3.2, we still adopt Group Relative Policy Optimization (GRPO) ... as the RL training algorithm.
implementation detail · filed under post-training
Group Relative Policy Optimization is a reinforcement learning algorithm that optimizes the policy model by maximizing a clipped objective on a group of responses sampled from the old policy for each question.1 It estimates each output's advantage by normalizing the outcome reward within the group, subtracting the group's average reward from that output's reward, and adds a KL penalty against a reference policy.2
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
GRPO optimizes the policy model by maximizing the following objective on a group of responses sampled from the old policy given each question— DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
The advantage of is calculated by subtracting the average reward of the group from the reward of output— DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
Also called GRPO, GRPO (Group Relative Policy Optimization), GRPO algorithm, GRPO reinforcement learning training, Group Relative Policy Optimization (GRPO), RL with GRPO.
18 quotes from 16 sources, one card per model, each opening with its strongest.
For DeepSeek-V3.2, we still adopt Group Relative Policy Optimization (GRPO) ... as the RL training algorithm.
Our training procedure largely follows the asynchronous GRPO algorithm with the stability optimizations proposed in NVIDIA (2026)
The model underwent multi-environment reinforcement learning using asynchronous GRPO (Group Relative Policy Optimization)
asynchronous GRPO algorithm with the stability optimizations proposed in nvidia2026nemotron3superopen
asynchronous GRPO (Group Relative Policy Optimization)
For the RL stage, we implemented the Group Relative Policy Optimization (GRPO) algorithm
Reinforcement Learning (RL) is applied using Group Relative Policy Optimization (GRPO)
independent cultivation of domain-specific experts (through SFT and RL with GRPO)
Reinforcement Learning using Group Relative Policy Optimization (GRPO)
Hy3 supports GRPO reinforcement learning training with verl
Hy3 supports GRPO reinforcement learning training with verl
For the RL stage, we implemented the Group Relative Policy Optimization (GRPO) algorithm
Reinforcement Learning (RL) is applied using Group Relative Policy Optimization (GRPO)
independent cultivation of domain-specific experts (through SFT and RL with GRPO)
The model underwent multi-environment reinforcement learning using GRPO (Group Relative Policy Optimization) across math, code, science, instruction following, multi-step tool use, multi-turn conversations, and structured output environments.
Our RL algorithm builds upon GRPO [40] and incorporates the IcePop technique [61] to mitigate the training-inference mismatch.
the group size in the GRPO algorithm is configured to 1
GRPO
We adopt GRPO (Shao et al., 2024) as our alignment algorithm.
Picked by hand, not extracted: where to read more, not evidence for anything on this page.