implementation detail · filed under data curation
Common Crawl
A web crawl dataset sourced from the Common Crawl Foundation.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
The English Common Crawl data was downloaded from the Common Crawl Foundation
useddata curationin Nemotron 3 UltraNVIDIA
Filed alongside
Other methods under data curation :: data sourcing.
Expert co-created training dataGitHub CrawlAgentic knowledge graph constructionEssentialWebFinePDFsHigh-recall web-data curationImage-code pairs and computer-use trajectoriesLong-document curationNemotron-3-Ultra corpusNemotron-CCNemotron-Post-Training-v3Star-threshold-filtered GitHub repositoriesStreaming training-data ingestionVideo frame extraction at 1 FPSVideo sampling parametersWeb knowledge graph construction and question generation