Open Models Almanac
Methodstraining objectivelanguage modelling objective

general family · filed under training objective

Causal next-token prediction

Trains a causal sequence model to predict the next token; the evidence includes language and vision-language data, as well as robot trajectories.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See language modelling objective for the methods filed beside it.

Also called causal next-token-prediction objective, next-token prediction objective.

sources
2
models
2
labs adopt it
2
strongest
core

Documented in

Aleph Alpha
Allen Institute for AI

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
core 2

MolmoAct 2

Allen Institute for AIadopted1 source

adds robot-specific state, setup/control, and action-output tokens so that robot trajectories can be trained with the same next-token objective as vision-language data.

coreunclearin MolmoAct2-PretrainAllen Institute for Artificial Intelligence (AI2)
↑