Open Models Almanac
Methodstraining objectivemulti-token prediction objective

methods · training objective

multi-token prediction objective

7 methods filed at this node or below it, from the sources of 7 models.

In this branch 7

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 7

MTP Boosting used · 6 sources · 7 quotes
Multi-Token Prediction used · 5 sources · 7 quotes
5 single mentions
Continued pretraining for MTP layers used · 1 source · 1 quote
Frozen-backbone MTP-head fine-tuning used · 1 source · 1 quote
LK loss used · 1 source · 1 quote
MTP warm-up used · 1 source · 1 quote
Shared-weight MTP objective used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑