ambiguous · filed under data curation
Deduplication
A general process for removing duplicate data; the evidence does not specify a particular deduplication mechanism.
- sources
- 2
- models
- 2
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
Deduplication was done in the same way as for Nemotron-CC.
useddata curationin Nemotron 3.5 LightningNVIDIA
The data was subsequently deduplicated and filtered in various ways
useddata curationin Nemotron 3 UltraNVIDIA
Filed alongside
Other methods under data curation :: deduplication.