Methodsdata curation
methods · pipeline stage 1 of 6
data curation
532 methods filed at this node or below it, from the sources of 38 models.
Explainer · 01 · 35 minThe anatomy of a training recipeLabs agree on the rough order of stages and a few habits inside them. Each stage spends tokens on what the last one could not teach, and each can lose some of what came beforeExplainer · D1 · 30 minFiltering and deduplication: from a crawl to a corpusA good filter raises average quality and leaves less unique text to fill a budget. Reports count what each step removes, and rarely what removing it boughtExplainer · D4 · 25 minMultimodal training data: when pixels and sound enter the recipeImages from the first step train both modalities together, at a cost in text tokens; images added later keep a finished text model, which then has to be protected
In this branch 532
Everything filed at this node or below it, with one collapsible heading per child node.
filed here 11
data sourcing 37
34 single mentions
data filtering 145
129 single mentions
deduplication 15
13 single mentions
synthetic data 138
128 single mentions
data mixture & curriculum 117
109 from one source, or only mentioned as not used
sequence packing 27
24 single mentions
tokenization 42
38 single mentions
By model
Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.
Proposed children
Paths the corpus wanted and the taxonomy does not have. A human promotes them into the outline; the extraction cannot.
robotics action and state normalization