Open Models Almanac
Methodstraining objectivelanguage modelling objective

methods · training objective

language modelling objective

15 methods filed at this node or below it, from the sources of 16 models.

In this branch 15

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 15

Autoregressive discrete-action prediction core · 3 sources · 3 quotes
Action-token-only cross-entropy loss core · 2 sources · 3 quotes
Causal next-token prediction core · 2 sources · 2 quotes
12 single mentions
Chain-of-thought labeling used · 1 source · 1 quote
Cross-entropy objective used · 1 source · 1 quote
Discrete-tokenized pretraining used · 1 source · 1 quote
High-level subtask prediction used · 1 source · 1 quote
Sampled-token objective used · 1 source · 1 quote
Stricter token constraints during training used · 1 source · 1 quote
Text pre-training used · 1 source · 1 quote
Fill-in-the-middle completion optional · 1 source · 1 quote
Implicit high-level inference from training data evaluated · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑