Open Models Almanac
Methodsdata curationsequence packing

methods · data curation

sequence packing

27 methods filed at this node or below it, from the sources of 13 models.

In this branch 27

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 27

Length-aware best-fit packing used · 2 sources · 2 quotes
Packed-sequence training for long contexts used · 2 sources · 2 quotes
Sequence packing used · 2 sources · 2 quotes
24 single mentions
Cost-aware longest-first bin packing core · 1 source · 2 quotes
Next-k-Fit core · 1 source · 1 quote
Bin packing of document chunks used · 1 source · 1 quote
Cross-source document packing used · 1 source · 1 quote
Exact-length sequence bin packing used · 1 source · 2 quotes
Fixed-cap action-chunk packing used · 1 source · 1 quote
Group-local sequence packing used · 1 source · 1 quote
Multi-stage robotics data preprocessing used · 1 source · 1 quote
Oldest-chunk anchoring used · 1 source · 1 quote
Quantized sequence packing used · 1 source · 1 quote
Random Sequential Document Buffer (RSDB) used · 1 source · 2 quotes
RSDB document chunking used · 1 source · 1 quote
Smallest-overfill fallback with truncation used · 1 source · 1 quote
Subset-sum sequence filling used · 1 source · 1 quote
Vision-language token sequence packing used · 1 source · 1 quote
Configurable sequence padding optional · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
MiMo-V2.6-Flash usedGroup-local sequence packing used—
NVIDIA-Nemotron-3-Ultra-550B-A55B usedLength-aware best-fit packing used—
DeepSeek-V4-Flash usedCross-source document packing used—
DeepSeek-V4-Pro usedCross-source document packing used—
Kolibri-1 coreCost-aware longest-first bin packing coreNext-k-Fit coreBin packing of document chunks usedExact-length sequence bin packing usedFixed-length conversation packing with document-masked attention usedOldest-chunk anchoring usedPer-rank buffering and ordering by attention cost usedRandom Sequential Document Buffer (RSDB) usedRSDB document chunking usedSmallest-overfill fallback with truncation usedSubset-sum sequence filling usedUniform random buffered-document chunk selection used—
Laguna-S-2.1 usedSequence packing used—
MiMo-V2.6-Pro usedGroup-local sequence packing used—
Mistral-Large-4 usedSequence packing used—
MolmoAct 2 usedDynamic-programming selection for sequence packing usedFixed-cap action-chunk packing usedOn-the-fly sequence packing with attention masking usedQuantized sequence packing usedRobot-example packing with per-example VLM context used—
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B usedPacked-sequence training for long contexts used—
SmolVLA usedFixed-shape batching with excess-frame truncation used—
VLA Foundry (Foundry-VLA-1.7B) corePer-modality preprocessing into WebDataset tar shards coreMulti-stage robotics data preprocessing usedConfigurable sequence padding optional—
Xiaomi-Robotics-1 usedVision-language token sequence packing used—
↑