single mention · model architecture
Multi-Token Prediction module for residual codebooks
Uses MTP to model residual speech-codebook tokens for fine-grained acoustic modeling and control.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See prediction head for the methods filed beside it.
Also called multi-token prediction (MTP) module.
Filed under model architecture › prediction head
Explainer · P1 · 34 minMulti-token prediction: a training signal and a built-in drafterAn MTP module takes one or two percent of a checkpoint's parameters and has two uses: a training signal whose measured gains are modest, and a drafter that loses value at each further token and can fall behind the backbone as training goes onExplainer · X2 · 32 minHearing, speaking and keeping time: audio, omni models and multimodal positionsA separate speaking model keeps a large model's steps and context for text at the price of a second stack; one backbone is a single model whose every speech frame takes a step and a position; and time can travel in position ids, which cost no tokens, or in text, which is read like any other token