Kolibri-1
Aleph Alphawe remove from the training corpus every document that contains text from our evaluation suite.
5 more from this model
Documents in our mid-training and long-context extension datasets were scored for contamination by n-gram overlap against our evaluation suite, and contaminated documents were dropped in full.
we therefore decontaminate the mid-training pool, from which long-context extension also draws, and remove every document that contains text from the 226 tasks of our evaluation suite, with all their splits.
To avoid this, we scan the SFT data pool against our benchmarks and drop the rows that reuse benchmark text.
All samples undergo the same processing pipeline, which removes overlap with our evaluation benchmarks
We therefore decontaminate the mid- and post-training pool