specific method · filed under data curation
Training data deduplication and filtering
Removing duplicates and filtering out junk or low-quality training data, potentially to support safety or other objectives.
Also called deduplication and filtering.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
may include deduplication and filtering to remove junk or other low-quality data, or to advance safety or other objectives
useddata curationin InklingThinking Machines Lab
may include deduplication and filtering to remove junk or other low-quality data, or to advance safety or other objectives
useddata curationin InklingThinking Machines Lab
Filed alongside
Other methods under data curation.