Open Models Almanac
Methodspost-trainingreward modelling

specific method · filed under post-training

Groupwise Reward Synthesis

Groupwise Reward Synthesis builds task-specific rubrics by comparing multiple offline rollouts of a task, to add quality scoring beyond binary test rewards.1 During training the rubrics are reused to score individual rollouts, and these quality scores are combined with test rewards.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Itcomparesmultipleofflinerolloutstoconstruct task-specific rubrics — MiMo V2 6 technical report
  2. which are reused during training to score individual rollouts and combine their quality scores with test rewards — MiMo V2 6 technical report

Also called Groupwise Reward Synthesis (GRS).

sources
3
models
2
lab adopt it
1
strongest
core

Documented in

Evidence

4 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
used 3core 1

MiMo-V2.6-Flash

Xiaomiadopted2 sources

Groupwise Reward Synthesis (GRS) builds task-specific rubrics offline from contrasting rollouts and fuses rubric quality with test outcomes

coretraining objectivein MiMo-V2.6-Flash-RLXiaomi
2 more from this model

Through our proposed Groupwise Reward Synthesis (GRS) and Groupwise Advantage Redistribution (GAR), these fine-grained distinctions are converted into more informative learning signals

usedtraining objectivein MiMo-V2.6 RL trainingXiaomi

It compares multiple offline rollouts to construct task-specific rubrics, which are reused during training to score individual rollouts and combine their quality scores with test rewards.

usedpost trainingin MiMo-V2.6-ProXiaomi

MiMo-V2.6-Pro

Xiaomiadopted2 sources

Through our proposed Groupwise Reward Synthesis (GRS) and Groupwise Advantage Redistribution (GAR), these fine-grained distinctions are converted into more informative learning signals

usedtraining objectivein MiMo-V2.6 RL trainingXiaomi
2 more from this model

It compares multiple offline rollouts to construct task-specific rubrics, which are reused during training to score individual rollouts and combine their quality scores with test rewards.

usedpost trainingin MiMo-V2.6-ProXiaomi

Groupwise Reward Synthesis (GRS) builds task-specific rubrics offline from contrasting rollouts and fuses rubric quality with test outcomes

usedtraining objectivein MiMo-V2.6-Pro-RLXiaomi
↑