implementation detail · filed under post-training
Budget-based verbosity control
A reward-hacking mitigation that penalizes a candidate in binary comparison when its output exceeds a length budget estimated from the cold-start model.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
To mitigate reward hacking toward increasingly verbose outputs, we apply a budget-based verbosity control analogous to the reasoning-effort control above: given an initial verbosity ℓ0 estimated from the cold-start model and a multiplier σ, a candidate whose output length exceeds σ·ℓ0 automatically loses the binary comparison.
usedunclearin Kimi K3Moonshot AI
Filed alongside
Other methods under post-training :: preference optimization.