Kolibri, Aleph Alpha’s 78B-parameter model, reads 20T tokens in pre-training, and about 4.8T of those were written by other language models. German shows why. Open German datasets came to 390B tokens after deduplication and filtering (its blog), and the run wanted at least 4T German tokens seen. So Aleph Alpha curated its own German crawl, then reused about 1.1T unique tokens of web pages that a model had rewritten as encyclopedia entries, dialogues and lessons for an earlier project.
DeepSeek V4.1 filters “model-generated content with limited information gain” out of its pre-training text, as “implicit duplication”, without saying whether it means text found on the web or made in-house. The same report’s post-training section credits synthesized data and environments with “essentially all of the observed gains”, an attribution with no number or ablation behind it.
Both statements fit one report because the two stages differ. In pre-training, generated text sits beside trillions of organic tokens, and only another model reading the source can tell you that a rewrite got a fact wrong, with blind spots of its own. In post-training, nobody publishes millions of worked reasoning traces or coding tasks with a grader, and for some of that data a test can catch a wrong answer.
The sections run from where organic text is most plentiful to where there is none: rephrasing, translation, long-context data, answers for fine-tuning, then tasks. At each step, ask what could catch a generator’s mistake.
Background
The primer covers the stages of a run. Pre-training is the long first stage on broad text, followed by shorter base-model stages, mid-training and long-context extension. Post-training means supervised fine-tuning (SFT), which trains on conversations, and reinforcement learning (RL), which trains on tasks with a reward. Tokens come in kinds: seen tokens count repeats, a pool is what a source holds, unique tokens are the distinct ones read, packed tokens fill the final sequences, and loss tokens are those the loss has targets on. A proxy is a small model standing in for a run nobody can repeat, and an ablation changes one piece of a recipe to see what it did.
A generator or teacher is a model whose output becomes training data. (Labs also say “teacher” for a model that supervises a student as it generates, which is a different operation.) Rephrasing rewrites a given document in another form, and other pipelines generate text from a prompt or a short seed with no document behind it. A reasoning trace is the text a model writes before it answers. A verifier is the code that decides whether a task was solved, and its verdict often becomes the reward.
Modeling code records none of this, so we read technical reports first, then cards and official blogs, and name outside papers where we use them. We count labs out of the 16 on this site, and a blank means we searched and found nothing, which doesn’t mean a lab doesn’t do it. Most reports give a recipe without a size or an effect, so we mention a missing number only where it changes how to read a claim. Reports describe what shipped, and none says that a synthetic pipeline it kept added nothing overall.
Kolibri's stages in order. Stage lengths are not to scale.
- Pre-training 20T seen
keep about 24% synthetic (4.8T)
- Mid-training 3.44T seen
remove German rephrased and translated web text
keep synthetic instruction and QA stays
- Long-context extension 200B seen
remove synthetic samples from the naturally long pool, one third of the stage's mix
- SFT 268B packed per run
keep 63.8% of pool tokens written or rewritten by Aleph Alpha, and the rest open data, often model-written elsewhere
- RL 38 environments
keep tasks partly agent-written
Pre-training rephrasing
Labs give two reasons for rewriting. A run needs more tokens than good text supplies without repeats: on M.1, Poolside’s earlier model, it found “excessive repetition in high-value subsets”, and for Laguna XS.2 it “expanded the amount of usable training signal through targeted synthetic rephrasing at scale”. Moonshot’s K3 follows a recipe from Kimi K2, whose report (outside our corpus) says one epoch “is insufficient for comprehensive knowledge absorption”, and more bring “diminishing returns”. The other reason is scarcity in a language, as with Kolibri’s German.
Nemotron-CC (Su et al., 2024) set the pattern most labs cite. A prompt turns each seed document into a new form, with no instruction to add knowledge: the approach “does not utilize the language model as a knowledge bank”. It has five forms, a Wikipedia-style rewrite, question-answer pairs appended to the document, a distillation, an extraction of knowledge and a knowledge list. Kolibri’s English data uses a “Nemotron-CC-style recipe” with Gemma-4-26B-A4B, and its German data uses three styles from Mistral-Nemo-Instruct-2407. Moonshot adds “fidelity verification against the source documents”, which it doesn’t describe further.
Invented example
Seed document
The Ostrel runs 1,233 km from the Talen hills to the sea. Three cities stand on its banks: Morvane, Lisk and Haldt. Morvane, the largest, grew up around the lower ford.
Wikipedia-style rewrite
The Ostrel is a river 1,323 km changed fact long, rising in the Talen hills and flowing to the sea. Its cities are Morvane, Lisk and Haldt, of which Morvane, which formed at the lower ford, is the largest.
Question and answer pairs appended to the seed
Q: How long is the Ostrel? A: 1,233 km.
Q: Which cities stand on it? A: Morvane, Lisk and Haldt.
Q: Which is the largest? A: Morvane, by the lower ford.
Knowledge list
- Length: 1,233 km
- Course: Talen hills to the sea
- Cities: Morvane, Lisk, Haldt
- Largest city: Morvane, at the lower ford
3 new texts, 0 new facts, 1 new error
Generators here tend to be small, because the volume is enormous: 12B in Nemotron-CC, 26B at Kolibri. That may do. Outside studies find little or no benefit from a larger generator above 1B (Niklaus et al., 2026) or 3B (BeyondWeb, 2025), and Niklaus et al. find “the choice of rephrasing prompt is the dominant factor”, which is the part labs print least of. Kolibri names its templates, Ultra states no recipe for syn-crawl (NVIDIA’s label for rephrased crawl), and Moonshot gives adjectives.
Five of the 16 labs state a share or a recipe for generated text in the broad pre-training mix. The numbers don’t compare, since each is a share of something else.
| Model | Stated share | Share of what |
|---|---|---|
| Kolibri | about 24% | 20T seen, both languages, rephrases and translations |
| Nemotron 3 Ultra | syn-crawl 32.7% then 26.2%; with “synthetic SFT-style data”, at least 47.4% then 43.3% (computed from Fig. 4) | each phase’s sampling mix, 15T then 5T |
| Laguna XS.2 | about 13% | the mix across all stages, from about 4.4T generated |
| MiMo-V2-Flash | about 5%, reasoning data | a 4T stage it calls mid-training, inside the 27T its abstract calls pre-training; unit unstated |
| Kimi K3 | no share printed | rephrases knowledge and mathematics |
No lab rejects synthetic pre-training text outright. Three state limits:
- DeepSeek’s V4.1 filter covers “outputs from less capable models and low-quality machine-translated text”, and the same paragraph explores “model-in-the-loop data iteration approaches as a foundation for future large-scale synthetic data”. A separate report, V4, removes “batched auto-generated and templated content” from web data.
- Z.ai’s GLM-5 report says pipelines “strictly avoid the use of synthetic, AI-generated, or template-based data”, in its paragraph on math and science. It builds synthetic data for later stages.
- MiniMax’s M3 blog says “interleaved data scales more easily than synthetic data”. From its heading, interleaved seems to mean mixed image and text documents. That compares ease of scaling, and doesn’t say synthetic data was dropped.
Of the other eight, Mistral states generated data only in mid-training, and Thinking Machines’ card says “synthetically generated or augmented” with no stage. The other six say nothing on broad text.
What a rephrase can add
DeepSeek says the low-gain text it filters “largely reformulates existing information”. Poolside says of rephrasing that it expands “usable training signal”. They aren’t speaking of quite the same text, and each claim can be made precise (our derivation, in no report).
Write for how much knowing tells you about . Let be the world the text describes, a document, the generator, which was trained on text about and so depends on it too, and a rephrase. Once you hold and , the world affects through nothing else, so is a Markov chain, and what the rephrase adds to the document is bounded by what the generator knows beyond it:
A verbatim repeat adds zero. If the generator knows nothing more about than the document does, a rephrase adds zero too, and that’s the sense in which it’s duplication. With the whole corpus in place of , a generator trained on the same web has little left to add. Mutual information doesn’t separate true from false, so this bound says nothing about errors. It also describes an ideal learner, one that already knows how wordings relate.
Training doesn’t start out knowing that, which is where Poolside’s claim has room. Take a linear toy with one feature for a fact and one for each wording, being the strength of wording features against the fact feature. Trained from zero weights to zero loss on wordings of a fact, the model’s margin on a wording it hasn’t seen is , against for verbatim repeats, and more steps don’t close that gap. A model that memorises strings has to see several wordings before it picks up the fact.
Errors change the answer, and how they correlate matters more than their rate. Keep the original among the texts. If each other wording is wrong independently with probability , rephrasing stays ahead of repeating while , at any . If the generator gets this fact wrong every time, the margin is , and each further rephrase makes it worse. We found no report that measures how correlated a generator’s errors are across rephrases.
Margin on an unseen wording, from −1 to 1. The original text is always correct.
Threshold σ²/(2(1+σ²)) at σ² = 4: ε below 40.0%
At ε = 10.0%, independent errors stay above repeats
What’s measured
One report on a shipped model isolates pre-training rephrasing, and it’s narrow. On an early checkpoint of Kimi K2, SimpleQA scores 23.76 on raw wiki-text at 10 epochs, 27.39 with one rephrasing, and 28.94 with ten rephrasings at one epoch each. The recipe it shipped rephrases each corpus at most twice, and it calls synthetic data for scaling “an active area of investigation”.
Su et al.’s Table 10 tests the recipe most labs copy, on four 8B models at 1T tokens, each a single run.
| Blend | ARC-E | MMLU | The paper’s average |
|---|---|---|---|
| Low-quality documents | 67.7 | 48.2 | 52.5 |
| Low-quality documents, rephrased | 71.3 | 47.1 | 54.0 |
| High-quality documents, eight passes | 74.2 | 53.4 | 55.8 |
| Four of those passes swapped for rephrases | 76.7 | 53.6 | 56.7 |
The second pair is the alternative a lab faces when it rephrases to avoid repeats, and rephrasing gains 0.9 points (computed). The first pair gains 1.5, and MMLU, a test of knowledge, falls 1.1. The authors write that such drops “may indicate potential misinformation introduced by data synthesis”, and that they “did not verify the factual accuracy or fidelity to the original contents”.
Kang et al. (2025) train over 1,000 models of up to 3B on up to 200B tokens, against fresh raw web text, so they test something else: their high-quality rephrasing aims to be, in their words, “akin to an aggressive data filtering”. A mix of one third rephrased text reaches the same validation loss up to 5 to 10 times sooner at larger budgets, though rephrased text alone isn’t faster. Niklaus et al. put the best mix at 60% to 80% synthetic, against Kang’s roughly 30%, on different metrics.
Text generated from seeds or prompts, with no document to rewrite, looks different. Kang’s two-thirds textbook mix underperforms, while one third overtakes raw web after about 20B tokens. Phi-4’s 13B ablation models on synthetic data alone score 14.8 points below phi-3-medium on TriviaQA, and an equal share of web rewrites halves that to 7.7, while the other seven benchmarks hold or rise. NVIDIA’s Ultra tests benchmark-seeded questions in a 100B ablation on an unnamed Nemotron checkpoint: GPQA moves from 30.8 to 41.9, MMLU-Pro from 64.8 to 66.6, and math holds. Those are seeded from benchmarks’ training splits, so the gain is hard to read.
In short, Su’s one pair and K2’s one test favour rephrasing over repeats, by 0.9 points and by 3.6 to 5.2 points (computed), with no repeated runs, on models up to K2’s early checkpoint. Kang’s favours a rephrased mix over raw web, on a loss. Labs running at larger scale are extrapolating from these (our reading).
Model collapse
The studies behind the phrase “model collapse” train a model on its predecessor’s output, repeatedly. Rare events disappear from the output first. Shumailov et al. (2024) find “only early signs” in a fine-tuned OPT-125m, and Gerstgrasser et al. (2024) find, on a 9M GPT-2, that replacing data hurts where accumulating it doesn’t. None of these tests rephrasing by a different model family across several generations at scale.
The paper DeepSeek V4 cites, Zhu et al. (2024), tests one generation: small models trained from scratch on a mix of human text and text generated from prompts. Perplexity on human text rises about 5% at a quarter synthetic and about 2.3 times at all synthetic (computed over its 22 domains). Any text at share dilutes the target, so an ideal learner’s perplexity can rise by up to , and the measured rises up to three quarters stay below that (our derivation). Its task averages barely move, about 41 on GPT-2 and 33 on OLMo-237M, though that may be a floor, since guessing gets about 33 on OLMo’s six tasks and about 35 on GPT-2’s seven (our estimate from option counts). The authors conclude that “relying solely on synthetic data leads to model collapse” and propose editing human text at the token level. Kang’s rephrased mixes point another way, on another kind of text, and no head-to-head at scale exists.
What it costs
A rewrite is built to differ at the string level, so it passes exact, substring and fuzzy deduplication, and repetition caps count it as new (our reading; we found no report whose deduplication compares a rephrase with its source). That is the practical form of “implicit duplication”, and it’s why a count of unique tokens can flatter a pool.
Generated text can carry benchmark content, too, since generators learned from benchmarks and some are seeded with them. In Kolibri’s curated mid-training pool, “synthetic instruction and question-answering data holds 32 % of the inspected documents but 67 % of the removals”, at about 4.3 times the rate of everything else (computed). The method finds benchmark-shaped text most easily, so that gap may partly reflect what it can see. The filtering article covers the method, and why a null at proxy scale doesn’t carry to a full run. For SFT, Kolibri adds that “paraphrasing can still get past this filter”.
Translation
Translation is rephrasing into another language, used mostly where that language is short of text, and its output can be scored against the source. NVIDIA’s Indic pipeline has four stages for twelve priority languages: translate, score from 0 to 100 and list errors, rewrite what scores below 75 against those errors, score again, and keep at 75 or above. Gemma-4-31B-it does all four. Translation alone has a median cost of about 0.335 GPU-hours per million input tokens on a benchmark, after speed work, so the 1,358B of native-script input in those languages would come to some 455k GPU-hours (computed, rough: the report says its input counts aren’t total processing, and doesn’t say the whole corpus ran at this rate).
One model does all four boxes for the twelve priority languages, 88.4% of the output (computed). Ten further languages use another translator, Bodhan-MT.
The report names the problem: “using the same model for translation and judging may favor its own translations”. There’s a precise worry behind that. With the writer’s error rate on an item and the judge’s miss rate, the chance that an item is wrong and passes is , if the two err independently on a given item. A model judging itself catches its slips and shares its blind spots, so the covariance stays. A second model lowers it without removing it, since hard items are hard for everyone (Eckhardt and Lee, 1985, on multi-version software). Human review is “sampled”, and could estimate the miss rate on kept documents, but the report prints none.
The gains on five Indic benchmarks, 4.98 to 9.82 points against the checkpoint before that 100B stage, belong to the stage, since “they do not separate the contributions of synthetic and natural Indic text”. Filters can miss translated text: NVIDIA’s Nemotron 3 Nano (outside our corpus) found after training began that its quality classifiers rated some uninformative translated documents highly.
A judge checks whether a translation matches its source, and it can’t supply what the source never held. Aleph Alpha writes: “A corpus translated from English therefore remains, in many respects, an English-language view of the world expressed in German.” It excludes rephrased and translated German web text from mid-training because it “read less naturally”. A later proxy sweep neither confirms nor refutes that: runs on an extended pool score 2.4 to 3.1 points lower on German HellaSwag (a commonsense test), against at most 0.9 between curated runs, but no pair test after SFT clears the noise, and that pool also adds low-quality web tiers. Kolibri doesn’t translate reasoning traces at all, since doing so “leaks artifacts of the English trace”. It translates prompts, and a teacher writes a new German trace.
Translation as training data is older than these pipelines: back-translation gained 2.8 to 3.7 BLEU in machine translation (Sennrich et al., 2016). There “only the source side” is synthetic. In a language model every token is a target, so a translation error is a wrong label.
Long-context data
Long documents exist. What doesn’t exist, as K3’s report puts it, is range for free: “Length alone, however, does not confer long-range capability.” A question about the last page can be answered from the last page (our gloss). Labs respond differently by stage.
In base stages, the data trains attention at length. Moonshot permutes and concatenates documents and sub-tasks, so that tasks “can be solved only by attending to information scattered across the full 1M-token context”. Z.ai’s GLM-5 uses natural and synthetic data, with similar texts packed together and “MRCR-like data” at 200K.
In SFT, the data supplies questions that span a document. Kolibri’s recipe takes about 78k documents of 128k to 224k tokens. Qwen3.8-27B reads a document with an index of about 40 sections and writes four simulated requests, each tied to the sections it needs, and in 77.4% of conversations the requests draw on at least three of the document’s five parts. A separate call usually answers from just those sections, so the answering call needn’t handle the length, while the student reads the whole document. NVIDIA’s Nemotron 3 Super concatenates related documents up to 512K and has an LLM write questions needing “at least 4 to 7 distinct retrieval or reasoning steps”, then eight sampled traces vote on the answer.
These data can resemble what benchmarks test. RULER (Hsieh et al., 2024) “consists solely of synthetic tasks” with fixed templates: find the magic number, track a variable. AA-LCR, by contrast, has 100 human-written questions over real documents. So Aleph Alpha removes synthetic samples from the naturally long pool, one third of its base stage’s mix, “to avoid artificially favourable results on synthetic long-context benchmarks such as RULER”. NVIDIA’s Ultra says of its long-context phase, “We did not use any RULER-style data in the blend”. Both are precautions, and we found no paper that measures the inflation. A measurement would compare the gain on a templated test with the gain on a test of the same skill that the data can’t resemble.
Whiskers are the printed ± intervals. * p below 0.05. LongBench Pro's +3.2 is the report's; the cells give 3.3 by rounding.
The one significant gain is on RULER, though this data isn’t RULER-style. It doesn’t show inflation: variable tracking minus AA-LCR is +7.2, with an unpaired half-width of about 17 (computed), so this table can’t tell either way. AA-LCR’s +8 is a failure to detect, at the power of about 100 questions, and not evidence of no effect.
Answers for fine-tuning
Web crawls don’t hold reasoning traces or tool-use conversations, and people can’t write millions. Eight labs state that an SFT or equivalent stage uses data written by models, Qwen under another name for the stage, and Z.ai is a partial ninth. NVIDIA writes that “SFT in turn consists mostly of synthetic data”, and adds that “very long SFT stages can reduce how much the model later gains from RL”. About 30% of Poolside’s SFT tokens are single-turn trajectories (records of one attempt) from “an open-source teacher model”, and about 45% are a corpus of the same size with reasoning traces added.
Three operations recur, and only one has a check on the output itself.
Regeneration keeps a dataset’s prompts and replaces its answers using a better teacher. Aleph Alpha found “many completions of mixed quality, including wrong answers to science problems, which led to regressed benchmark performance if included in training”. This is pseudo-labelling on a large scale. Its check, where there is one, is a judge or a match against a known answer, and NVIDIA uses a length filter, removing reasoning “shorter than 5,000 tokens”.
A backfill adds reasoning to a tool-use conversation that has none, written to lead to a call already recorded. Kolibri has a test: “The reasoning is kept only if a second LLM call derives the same tool call from it alone.” That shows the reasoning determines the call. It doesn’t show that the prompt leads there, or that the call was right, and since the teacher wrote it knowing the answer, a trace that simply states the call would pass.
A simulated user writes the later turns of a conversation. Kolibri’s reason: “Open data holds few long conversations, so we generate them.” Two labs state this for SFT data, Aleph Alpha and NVIDIA, which trains only on the final assistant response, since earlier turns can be poor. MiniMax has a simulator it uses in “training and evaluation” without saying how, and Mistral uses one inside RL environments.
Regenerated and in-house: 63.8% of 335.8B pool tokens.
Open data as released isn't human-written: the report says most open datasets are generated by other models.
Both totals: 63.8% of pool tokens (31.7 + 32.1, the table's "All" row), 42.5% of loss tokens (computed). The loss view assumes each domain's loss tokens split among its segments like its tokens. Without that, the total lies between 26% and 56%.
Does a stronger teacher or a filter help?
NVIDIA’s 3.5 report says that with its older teacher, “downstream reasoning performance largely plateaued after a single pass”, and with data regenerated by Kimi K3 “performance continued to improve over multiple passes”. Its hypothesis is that K3’s “more concise and information-dense reasoning” needs “additional exposure”. The contrast crosses runs and prints no numbers: Ultra’s report says of the earlier teacher’s data that “the student has not directly seen” it, since that data trained a separate teacher.
The largest public test is OpenThoughts (Guha et al., 2025), with SFT on a 7B Qwen student from 31,600 examples per run. A small gap between teachers can be a large gap between students. QwQ-32B and DeepSeek-R1 are about level on the paper’s three math benchmarks, 89.2 and 88.1 (computed from its Table 29), yet students on their math data reach 71.6 and 62.5 to 64.8, and Phi-4-reasoning-plus scores 85.4 with a student at 49.0, so on math the order holds and the gaps widen. On filtering, “we experimented with numerous verification and answer filtering methods, and none gave significant performance improvements”. Kolibri cites this to skip a filter on German chat replies, since “it adds compute without a known benefit”. These are small students, and no lab publishes the same test at scale.
Xiaomi gives the only before-and-after we found with token counts for one teacher’s data going into a student from another family. It fine-tuned Qwen3.5-9B on 77.4B tokens from MiMo, 27.2B of them loss tokens. SWE-bench Verified barely moves, 60.0 to 61.1, and SWE-bench Pro goes from 32.0 to 44.6. The baseline is the released student, with no control on other SFT data.
Keeping a teacher’s reasoning in German
A language can be a teacher’s blind spot. Asked to reason in German, a teacher often slides into English. Aleph Alpha measured more than 5,000 completions each: with an instruction in the system prompt, 67% of traces stay German throughout, 22% switch partway and 11% start in English. Prefilling, which here means writing the start of the model’s reasoning for it (a German opener such as “Der Nutzer möchte”), gets 97%. Despite the name, reasoning-prefill distillation is offline data generation. It intervenes where a filter would select: dropping English traces at 67% would also drop whichever prompts push the teacher out of German.
Those figures describe the teacher, and no run tests the prefill on a student. Table 51 tests all German SFT data on a German math benchmark, by raising its share. From none to 26.5% of the mix, the overall German score moves −4.4 (not significant) while answers both correct and in German rise 23.5 points. With no German SFT data, about 60% of the correct answers were in German (42.3 of 70.2), and with any share all are (computed, reading the overall score as counting any language), so the dip compares different behaviour. These are single runs per share.
Tasks and verifiers
A web crawl holds no graded tasks, so here generation is close to unavoidable, and a test can check the result. Here’s the shape of one coding task.
One coding task, from a merged pull request
The whole task: always or never solved
Environment
repository before the merge
When generated: set up by an agent
Prompt
the issue text
When generated: written by a model
Agent
takes both and makes an attempt
Verifier
the PR's tests, hidden from the agent
When generated: written by a model
returns the reward
Reference
the human's diff
When generated: written by a model, or absent
A typical shape; details vary by lab.
Eight labs describe a method for writing tasks, StepFun does so in a blog outside our corpus, and Thinking Machines gives a clause. The rewards article covers the rewards. Here we ask how a task gets built.
“Synthetic” covers a range. Laguna’s section “Synthetic Code Environments” mines commits, and it keeps “on the order of 30–60k tasks from a raw pool of ∼236k commits”. SWE-rebench V2 (Badertdinov et al., 2026), which Kolibri builds on, keeps 32,079 of 805,598 candidate pull requests, about 4% (computed), with human tests. DeepSeek V3.2 has 4,417 tasks with both environment and prompt synthesized, out of 85,267 (computed), and MiMo-V2-Flash 50K of 320K. So counts of environments don’t compare: the 409k of Laguna S 2.1 cover agentic and non-agentic work and include purchases from vendors.
What a verifier is checked against depends first on what exists. Where a repository has tests, labs mine them. Laguna requires “the gold solution to pass the tests and an empty solution to fail them” (its check), and Mistral keeps “tests that fail before the reference solution and pass afterwards”. Mined tasks exist only where a repository has usable tests, which favours some kinds of code.
Where nothing exists, a verifier has to be written, and two designs appear, with no comparison published. In DeepSeek V3.2’s general agent pipeline, “the agent will modify the solution or verification functions until the solution’s output passes the verification”. The two agree by construction, so only a later filter tests them: “retain only instances with non-zero pass@100”, meaning a model passed at least once in a hundred tries. GLM-5.3’s verifiers “are synthesized without access to the reference solution”, for a subset of tasks, and must pass “oracle, no-op, and unsolved-state checks” (we read these as: a known answer passes, doing nothing fails, the unsolved state fails). Those pipelines “still require a meaningful amount of human-in-the-loop work”.
A third route makes tasks from tests that exist. Kolibri has GLM-5.3 inject a bug: “A bug is kept only if at least one test fails and at least half of the tests still pass, so that it is detectable but local.” That’s SFT data, from a teacher fixing the bug.
Why does verifier error matter? A verifier that errs at random, on any attempt alike, slows training without changing what’s best. With and its false-positive and false-negative rates, expected reward is . Errors tied to how the task was attempted do change it: the policy moves to whatever passes most often, so an exploit only has to pass more often than honest work, and hard tasks are the most exposed. RL searches for such errors, where an SFT dataset only contains the errors already in it. Six labs report observing reward hacking, passing the check without doing the task, during training: DeepSeek, Moonshot, Z.ai, NVIDIA, Mistral and Xiaomi. Some of these cases involve reward models and not written verifiers. Poolside reports it in evaluations run during post-training.
Labs use different checks against different flaws.
| Flaw | Check | Example |
|---|---|---|
| Reference fails, or empty solution passes | Run both | Laguna; GLM-5.3 |
| Flaky tests or setup | Rerun | SWE-rebench three times, Xiaomi V2.6 eight |
| No signal | Band of pass rates | Poolside drops tasks “always solved or never solved”; Mistral calibrates difficulty by “solver pass rates” |
| Rejects valid work, accepts wrong work | Solvers of varying strength, an audit | Mistral, to “distinguish model failures from flaws in the specification or tests”; Xiaomi attempts each task four times |
| Solution left behind | Scrub the environment | Xiaomi calls leakage “a recurring failure mode”; DeepSeek V4.1’s setup agent “removes any traces” |
| Shortcut | An attacker | Mistral’s “adversarial agent”; Xiaomi’s hack agent, run “until the hack agent could no longer find a successful exploit” |
Rates are scarce. Xiaomi gives a confirmed-hack share “below 2%” of trajectories after that cleanup, a floor since it counts what was confirmed. Poolside’s judge flagged over 50% of trajectories on SWE-bench evaluations as post-training went on, and below 2% after a prompt told the model not to use solutions found online. SWE-rebench’s LLM judge finds some leakage in 23.0% of 509 generated problem statements, and 2.4% explicit.
Each check also selects. What’s kept follows , with the chance of passing, and nothing reweights it afterwards. A test that flips 5% of the time still passes three reruns 86% of the time (computed). A pass@100 filter keeps a task that a model solves 3% of the time with probability 95%, and keeps it just as readily if the model passes by an exploit. A bug filter keeps bugs the suite already catches, unlike many that ship. Pass rates also go stale as the student improves, and “some model solved it” removes what no model can solve yet. We found no report that prints what its filter did to difficulty. All this costs compute as well: eight traces, four attempts or a hundred rollouts per task.
Generators have habits in this job too. StepFun writes that agents “repeatedly converge on familiar task patterns” and “tend to construct easier tasks”. DeepSeek V4.1 trains its model to construct tasks, using “difficulty and correctness as reward signals”, and says “this capability remains far from perfect”. Five labs also build questions that take several search steps, from graphs or existing sets, and Z.ai and Aleph Alpha drop what a model answers without searching.
Who generates for whom
Most named external teachers are other open-weight models, and several labs on this site are each other’s sources.
Poolside and Liquid use open models they don't name. Xiaomi also trained a Qwen student on MiMo data.
Datasets carry this further than models do. Kolibri regenerates answers on NVIDIA’s released Nemotron datasets and translates Toucan, a tool-use set built with Kimi-K2, GPT-OSS and Qwen3. Generator choice also follows the job. Aleph Alpha writes: “smaller models are generally faster for long-context tasks, but larger models are more suitable for generating strong reasoning traces. They also have different values and biases, which we consciously select or avoid”. Google’s and OpenAI’s own documents don’t mention that Gemma and gpt-oss write data for others.
Labs filter for two traits they expect from teachers, identity and politics. “Most open datasets are generated by other models, and responses often carry their identity”, so Kolibri drops conversations in which the assistant claims to be another model. It adds that “several of our teacher models were built in China, so they carry known political biases on sensitive topics”, and has gpt-oss-120b judge open-ended conversations for that. NVIDIA’s cards state a keyword filter for teachers’ “nationalistic narratives”.
On licences, NVIDIA’s cards say that for math, code and science they “use open source permissive models such as GPT-OSS-120B”, and Aleph Alpha’s card says “permissively-licensed LLMs”. We found no document discussing a specific teacher’s terms. Open weights don’t settle whether a model’s outputs may train another. Laguna says a pre-training token “can be traced back through deduplication, filtering, and synthesis to its source document”, which covers one lab. A teacher’s own training data is usually unknown, and a student can later become a teacher, so an error could pass through several models with nobody able to trace it (our reading; no one has measured it).
Most of this has older names. Learning from a teacher’s samples at temperature 1 has the same expected objective as distillation with soft targets at temperature 1 (Hinton et al., 2015, raise it above 1), with more variance, and at lower temperatures the target sharpens, which is a bias. Training on invented inputs labelled by a model is model compression (Buciluǎ et al., 2006). A bug filter borrows mutation testing’s mechanism (DeMillo et al., 1978), a pass-rate band is goals of intermediate difficulty (Florensa et al., 2018), and a model trained to write tasks is unsupervised environment design (Dennis et al., 2020). The scale is new, and so is generated text being the target itself.
The methods compared
The table runs from weakest check to strongest.
| Method | Organic alternative | What the check can catch | What it costs | Best evidence, and its scope |
|---|---|---|---|---|
| Generation from seeds | Little in that form | Rarely stated | Knowledge, hallucination | Phi-4: TriviaQA −14.8, other seven level or up, 13B ablation models |
| Rephrasing | Repeats of good text | Junk output; wrong facts only if checked against the source (Moonshot, no detail) | Errors, no new facts unless the generator adds them, GPUs | Su: +0.9 over repeats, 8B at 1T. Kang: sooner to a loss than raw web, up to 3B |
| Simulated users | Few long conversations | Little | Generator’s idea of a user | None isolated |
| Regeneration | Weak open answers | A judge or a known answer | Teacher errors, identity, politics | OpenThoughts: filters null at 7B; near-level teachers, students far apart |
| Translation | Scarce native text | A score, often from the same model | Shared blind spots, a translated view of the world | NVIDIA: stage gains, synthetic and natural together |
| Long-context requests | Long documents, few spanning questions | Spread across sections | Compute, few loss tokens, resemblance to benchmarks | Kolibri: four of five rise, one significant, single runs on a proxy |
| Backfill | None | That the trace determines the call | Reasoning written in hindsight | No student test |
| Written tasks | Mined tasks, where tests exist | Solvable, detectable, stable, partly unhackable | Rollouts per task, wrong rewards, a shifted set | Yields and two hack rates; no comparison of verifier designs |
Where the models in the map sit
Sizes aren’t like for like.
| Lab | Pre-training text | Translation | Long-context | Teacher data | Named generators | Tasks and verifiers |
|---|---|---|---|---|---|---|
| Aleph Alpha | About 24% of 20T seen | Curated set; traces never | Requests over sections; removed from one pool | 63.8% of pool tokens own or rewritten | GLM-5.2, 5.3, Qwen3.8-27B, Gemma, others | Bug injection, multi-step search |
| NVIDIA | 32.7%, 26.2% syn-crawl; floor 47%, 43% | Indic, 2.83T generated | Concatenation, graphs, chunked translation | Yes; shares not printed | K3, V4 Pro, GLM-5.2, gpt-oss-120b, others | Wikidata, terminal, execution checks |
| Poolside | About 13% of XS.2’s mix | About 30% and 45% of SFT tokens | Unnamed open model | Mined tests, two-sided check | ||
| Moonshot | Rephrasing, no share | Permute and concatenate | Yes | Own | Graph-guided search; hidden verifiers | |
| Xiaomi | About 5% of one stage | Yes; MiMo into a Qwen student | Own | Audit, 8 reruns, hack agent | ||
| DeepSeek | Filters limited-gain text | Filters low-quality | Generator not stated | Own V3.2 for pipelines | Co-written verifiers, inspection agent | |
| Z.ai | Avoids in math and science | Natural and synthetic | Partial | Unnamed | Blind verifiers, knowledge graph | |
| MiniMax | Interleaved scales more easily | Simulator only | ||||
| Mistral | Trajectories in mid-training only | Fail-to-pass, solvers, attacker | ||||
| Liquid | Vision and audio only | LFM2-VL | Yes, including text turned into audio | Unnamed open models | ||
| Qwen | In-house teachers | Own | ||||
| Thinking Machines | Card, no stage | Small bootstrap SFT | Kimi K2.5 | One clause | ||
| StepFun | Blog outside corpus | |||||
| Tencent, Google, OpenAI |
A blank means searched and not found. Eleven labs state generated data in post-training in our corpus, StepFun in a blog outside it, and MiniMax a simulator (computed from this table).
Easy to get wrong
- “DeepSeek rejects synthetic data.” It filters limited-gain model text in pre-training and explores synthesis in the same paragraph.
- “Model collapse” as a verdict on rephrasing. The studies test recursive training on small models, or one generation of text generated from prompts.
- Kolibri as “synthetic for pre-training, organic later.” The stated exclusion is German rephrased and translated web text. Synthetic instruction and QA data is the curated pool’s largest class by documents, at 32%, and the report doesn’t say whether English rephrases stay.
- 2.83T as Super’s Indic data. It’s the corpus. Super’s stage has at most 10B of Indic text, natural included.
- 63.8% as the synthetic share of Kolibri’s SFT. It’s what Aleph Alpha wrote or rewrote, as a share of pool tokens, and much of the remainder is model-written elsewhere.
- “Synthetic environments” as written by agents. Many are mined from commits, with only the setup generated.
Check yourself
-
A lab has 500B of good text and wants 2T seen tokens from it. How many passes is that? If it swaps two of the passes for two different rephrases, about how many tokens of new strings does it get, and how many new facts, if the generator knows nothing beyond the documents?
Answer
Four passes. About 1T of new strings, if each rephrase is about as long as its source. New facts: none, by the assumption, and the bound on says that assumption is what matters. Some statements may also be wrong. Whether a model learns more from these than from repeats is what Su et al. test. -
Kolibri’s long-context domain has 49.8B tokens and 0.9B loss tokens. What share of its tokens does the loss have targets on? Why do its share of the pool and its share of loss tokens tell different stories?
Answer
. So it’s 14.8% of pool tokens and 0.6% of loss tokens. The pool share tracks the compute of reading those tokens, and the loss share tracks how many targets they supply. -
A verifier has false-positive rate 0.2 and false-negative rate 0.2, at random across attempts. What’s the expected reward for a policy that solves a task half the time, and for one that never does? Then suppose an exploit passes 70% of the time and honest work passes 60%. What does training favour?
Answer
, and . The ordering is preserved, with a smaller gap, so training slows. In the second case the policy moves to the exploit, since the verifier’s error depends on the attempt. -
Why does V3.2’s general agent pipeline need a pass@100 filter when its solution and verifier already agree? What does the filter still miss?
Answer
They agree by construction, since the agent edits either until they match, so agreement can’t show that the verifier is right. Some independent passes give partial evidence. A task passed once by an exploit is kept as well. -
A lab has plenty of organic English and no graded coding tasks. Where would you expect it to rely on generated data, and what could catch the generator’s mistakes in each place?
Answer
Tasks, where tests can catch a wrong patch, though they need checking themselves. If it rephrases English, only a model check against the source can catch a changed fact, and repeats or fresh lower-tier text are the alternatives. That’s where the trade-off in the callout is sharpest.
In the map
Methods
- Template-based synthetic data generation 1 source
- Knowledge and mathematics corpus rephrasing with fidelity verification 1 source
- Filtering batched auto-generated and templated web content 2 sources
- Four-stage Indic synthetic-data generation pipeline 1 source
- Section-grounded simulated requests for long-context training 1 source
- Synthetic long-context data via permutation and concatenation of multimodal documents 1 source
- Regenerate training-data completions 1 source
- Backfilling missing reasoning between tool calls 1 source
- User-simulated conversational continuation 1 source
- Knowledge distillation for synthetic data generation 3 sources
- Large-scale agentic task synthesis pipeline 3 sources
- Verifier synthesis without reference solution 1 source
- Verifiable terminal-task generation by injecting repository bugs 1 source
- Fail-to-pass test verifier construction 1 source
- Knowledge-graph-guided task synthesis 1 source
- Reasoning-Prefill Distillation 1 source
Taxonomy nodes
- synthetic data 138 methods
Further reading
- Kolibri report, §2.3.2, §2.3.3.1, §3.1.1, §3.1.2, D.2.5, H and I: https://aleph-alpha.com/downloads/tech-report.pdf. Every SFT recipe with its gates, the fullest public account.
- Nemotron 3.5 report §2.3, §3.1, §3.12 and Appendix C: https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Super-and-Lightning-Technical-Report.pdf. The Indic pipeline and post-training decontamination.
- DeepSeek V4.1 report §4.1, §5.1: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf. DeepSeek V3.2 §3.2.3, and V4 §4.1 for the collapse citation.
- Kimi K3 report §3.1, §3.4, §4.2, and Kimi K2 report (arXiv:2507.20534) §2.2. Laguna report §3.2.2, §4.3. Mistral Large 4 report §4.1. MiMo V2.6 report §4.2, §7.
- Su et al., Nemotron-CC, arXiv:2412.02595. The templates in Appendix H, and Table 10.
- Kang et al., arXiv:2510.01631, and Niklaus et al., arXiv:2604.13977. The most extensive tests of rephrasing by number of runs, which disagree on the best share.
- Zhu et al., arXiv:2412.14689; Shumailov et al., Nature 631 (2024); Gerstgrasser et al., arXiv:2404.01413. What collapse studies do and don’t show.
- Guha et al., OpenThoughts, arXiv:2506.04178. A thousand teacher and filter ablations on a small student.
- Badertdinov et al., SWE-rebench V2, arXiv:2602.23866. A mined-task funnel. Xu et al., TOUCAN, arXiv:2510.01179. A published tool-use pipeline.
- Hsieh et al., RULER, arXiv:2404.06654. The benchmark’s templates, to judge resemblance for yourself.
