Open Models Almanac
Methodsdata curationsequence packing

single mention · data curation

Bin packing of document chunks

Document chunks are packed into sequences to avoid padding tokens and unnecessary document cuts.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sequence packing for the methods filed beside it.

Filed under data curation › sequence packing