taxonomy node · level 2
deduplication
5 methods filed at this node or below it, from the sources of 4 models.
data curation :: deduplication
Matching aids for the classifier: in-pack deduplication; corpus dedup.
In this branch 5
Everything filed at this node or below it, with one collapsible heading per child node.
By model
Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.
| Model | techniques |
|---|---|
| DeepSeek-V4.1-Flash used | Joint prefetching and assignment of training samples usedSemantic image deduplication used— |
| NVIDIA-Nemotron-3-Ultra-550B-A55B used | Deduplication usedIn-pack deduplication used— |
| Laguna-S-2.1 used | Snapshot-level fuzzy deduplication used— |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B used | Deduplication used— |