Open Models Almanac
Methodstraining objectivelanguage modelling objective

implementation detail · filed under training objective

Action-token-only cross-entropy loss

Computes cross-entropy supervision only on predicted action tokens, rather than on all sequence tokens.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See language modelling objective for the methods filed beside it.

Also called Action-token-only next-token training loss, loss is computed only on the action tokens, standard next-token prediction objective.

sources
2
models
2
labs adopt it
2
strongest
core

Documented in

Allen Institute for AI
OpenVLA project (Stanford et al.)

Evidence

3 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
core 3

OpenVLA / OpenVLA-OFT

OpenVLA project (Stanford et al.)adopted1 source

OpenVLA is trained with a standard next-token prediction objective, evaluating the cross-entropy loss on the predicted action tokens only

coretraining objectivein OpenVLAOpenVLA team
1 more from this model

evaluating the cross-entropy loss on the predicted action tokens only

coretraining objectivein OpenVLAOpenVLA team
↑