specific method · filed under training objective
MTP Boosting
MTP Boosting is a dedicated post-training phase that updates only the multi-token-prediction head's parameters while the backbone stays frozen.1 It trains the MTP head with a knowledge-distillation loss against the frozen backbone and continues hidden-state mixing, reducing train-inference mismatch and improving draft acceptance.2,3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
the backbone remains frozen and only the MTP head parameters are updated
— NVIDIA Nemotron 3.5 Super and Lightning Technical Reportthese runs use a knowledge- distillation loss with the frozen backbone as the teacher
— NVIDIA Nemotron 3.5 Super and Lightning Technical Reportreducing the train–inference mismatch and improving acceptance at longer draft lengths
— NVIDIA Nemotron 3.5 Super and Lightning Technical Report
Also called MTP-boosting training phase, MTP-boosting phase, dedicated MTP-boosting phase, Multi-Token Prediction Boosting, post-training MTP boosting.