single mention · training objective
Attention-mask isolation of packed examples
Separates packed training examples so tokens from one example cannot condition on another.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See auxiliary loss for the methods filed beside it.
Also called training attention mask.
Filed under training objective › auxiliary loss