implementation detail · filed under data curation
Deterministic pre-splitting of ultra-long documents
Pre-splits ultra-long documents before mixing to distribute training tokens uniformly across shards and steps.
Also called deterministically pre-split.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
Ultra-long documents are deterministically pre-split before mixing to ensure a uniform distribution of training tokens across data shards and training steps.
usedunclearin DeepSeek-V4.1-FlashDeepSeek
Ultra-long documents are deterministically pre-split before mixing to ensure a uniform distribution of training tokens across data shards and training steps
usedunclearin DeepSeek-V4.1-FlashDeepSeek
Filed alongside
Other methods under data curation :: data mixture & curriculum.
AutoMixerVulnerability-discovery data inclusionAgent-centric multimodal data mixtureAttribution, hedging, and refusal data subsetsBlind pairwise bucket-boundary calibrationData SchedulerDomain-specific multimodal datasetsDynamic sampling for mixed-task reinforcement learningFiltering, deduplication, and difficulty calibrationGaussian-based data mixture and curriculum constructionInterleaved dataInterleaved multimodal trainingKL-regularized data-mixture optimizationLong-context data upsamplingPass-rate-based RL task samplingPrior-constrained Dirichlet mixture explorationProtocolQA-aligned RL training datasetsSample MixerScaling-ladder-guided data and model planningSmooth weighted round-robin source schedulingText-only pre-training corpusThree-stage data mixture strategyThree-stage pre-training curriculumTwo-phase curriculum