Open Models Almanac
Methodsdata curationdata sourcing

single mention · data curation

Conversion to WebDataset tar shards

Uses custom scripts to convert raw datasets into WebDataset tar shards.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See data sourcing for the methods filed beside it.

Filed under data curation › data sourcing