specific method · filed under training objective
Multi-Token Prediction Boosting
A post-training procedure that tunes the MTP head to match the backbone’s next-token distribution under inference conditions, including noise.
Also called MTP Boosting, MTP-boosting phase, dedicated MTP-boosting phase.
- sources
- 3
- models
- 2
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning paper arxiv.org§3.4 MTP Boosting
Evidence
5 spans quoted from the sources, strongest treatment first.
After training, a dedicated MTP-boosting phase further improved MTP accuracy.
Head-only KL distillation aligns MTP drafts with backbone logits for faster speculative decoding.
The goal of MTP Boosting is to make the MTP head match the backbone’s next-token distribution under the input conditions or noise it encounters at inference.
iterative MOPD optimization and MTP Boosting procedures
The goal of MTP Boosting is to make the MTP head match the backbone’s next-token distribution under the input conditions or noise it encounters at inference.
Filed alongside
Other methods under training objective :: multi-token prediction objective.