Synthetic Data for Training: Generate It, Then Judge It

Synthetic data for training works when a strong model generates it and an independent judge filters it. Skip that filter and you get model collapse.

Synthetic data for training works when three conditions hold: a strong teacher model generates it, an independent judge (not the generator) filters it, and you check the result against real data for distributional coverage. Skip that third step and the failure mode is not vague underperformance. It is model collapse, where the rare cases and long tails of your real distribution quietly disappear over a handful of training generations.

Here is a pattern I keep seeing. A team generates tens of thousands of synthetic support-ticket examples with a frontier model, fine-tunes on them, and ships a classifier that scores well in eval and stumbles on the first genuinely odd ticket a real customer sends in. The synthetic set never had an odd ticket in it, because the model that wrote the examples did not know what it did not know. Nobody checked the set against a real, held-out sample before training on it.

This is one of the deep dives under my complete guide to fine-tuning, because the data you train on decides more of the outcome than the technique you fine-tune with. Get the data wrong and no adapter method fixes it after the fact.

Key takeaways

• Synthetic data for training is model-generated text, code, or reasoning traces used as training examples — distinct from scraped web data or human-labeled data.
• The generator should never grade its own output. Filtering needs an independent judge: a separate model, a reward model, or a held-out real dataset.
• Model collapse is real and measured, not hypothetical. Recursive training on unfiltered generated data erodes the tails of the original distribution within a few generations.
• Curated synthetic data can substitute for scale. Microsoft's phi-1 hit 50.6% pass@1 on HumanEval at 1.3B parameters, largely on synthetic "textbook" data.
• How much synthetic data belongs in your mix depends on the stage. Pretraining wants it as a supplement; alignment and RLHF-style tuning can run almost entirely on it, if the filtering is disciplined.

What Counts as Synthetic Data for Training in 2026

Synthetic data for training is text, code, or reasoning traces generated by a model and then used as training examples for another model, rather than scraped from the web or hand-labeled by a person. That includes instruction-response pairs written by a teacher model, code generated and then filtered for correctness, and step-by-step reasoning traces produced and checked against a verifier. What it excludes is anything a human wrote from scratch and anything pulled from a live corpus without generation in the loop.

The category has grown because the economics flipped. Generation used to be expensive relative to scraping data you already had. By 2026, it is cheap enough that the harder problem is not producing synthetic data. It is deciding which of it is worth keeping.

Why Teams Generate Synthetic Training Data Instead of Collecting It

Four reasons show up in almost every team I talk to about this.

Cost. Human annotation for a few thousand high-quality examples can run into the tens of thousands of dollars once you account for guidelines, review, and rework. A teacher model produces a first draft of the same volume for a fraction of that, leaving humans to spot-check rather than write from scratch.

Coverage of rare edge cases. Real data follows whatever distribution your users happen to produce, which underrepresents the cases you most need the model to handle correctly. A teacher model can be prompted to generate deliberately for the tail: the ambiguous ticket, the adversarial input, the malformed request.

Privacy. Real user data carries real user information, and training on it means carrying that liability through every downstream model. Synthetic data sidesteps this when generation does not condition on real user records, which matters anywhere compliance reviews the training pipeline.

Speed to fine-tune. Collecting real labeled data is a project with a calendar. Generating and filtering a synthetic set is a pipeline you rerun next week when the task definition changes. That iteration speed is often the real reason synthetic data wins the argument internally.

The Generation Techniques That Produce Usable Synthetic Data

Four techniques cover most of what teams ship in production, and they are not interchangeable. Pick based on what you are trying to teach the model, not on which one is trending.

Teacher-student distillation. A larger, stronger model generates the training examples; a smaller model trains on them. This is the workhorse behind most small, cheap, specialized models that outperform general models on a narrow task. The student never sees the teacher's weights, only its outputs, so quality is entirely a function of what survives the filter.

Self-instruct and seed expansion. You start with a small seed set of human-written examples, often a few hundred, and prompt a model to generate variations and new instructions from that seed. This turns a small investment in human writing into a large training set, but it also amplifies whatever bias sits in the seed, so seed quality matters more than expansion volume.

Persona-based rewriting. The same underlying content gets rewritten from different personas or voices, which teaches a model tone and register rather than new facts. It works well for behavior-shaping fine-tunes where the task is consistent but the acceptable range of phrasing is wide.

Verifier-driven reasoning traces. For reasoning-heavy tasks, a model generates a full chain of reasoning, and a separate verifier checks whether the final answer is correct, keeping only traces that reach it. This is the mechanism behind RLVR-style training for reasoning models. The verifier is often a symbolic checker (does the code pass, does the math check out), which is far harder to game than an LLM judge scoring prose.

How to Filter Synthetic Data Before It Touches a Training Run

Generation is the easy half. Filtering is where quality gets decided, and it is the step teams skip when they are in a hurry. A working pipeline has four stages, and none are optional if you want the training run to hold up.

LLM-as-judge scoring. A separate model, not the generator, scores each example against a rubric: correctness, relevance, format compliance. Independence matters more than the rubric's sophistication. A generator judging its own output rates its own failure modes as fine, because those failure modes are exactly what it does not know it is doing wrong.

Reward-model scoring. Where you have one, a reward model trained on real human preference data gives a second signal that does not share the generator's blind spots. This is the pattern behind large-scale alignment pipelines: generate broadly, then let a preference-trained model do the narrowing.

Deduplication and diversity checks. Generated data clusters tightly around whatever the teacher model finds easy to produce, so embedding-based deduplication is not optional cleanup. It is the step that prevents your "40,000 examples" from actually being 4,000 examples repeated ten times with cosmetic variation.

Held-out real-data comparison. Before training, compare the synthetic set's distribution against a held-out sample of real data you did not touch during generation. This is the step from the opening story that got skipped, and it is the only check that catches coverage gaps the judge and reward model will not see, because they were never shown what real data looks like.

Model Collapse: The Failure Mode Everyone Skips Past

Model collapse is what happens when a generative model trains recursively on its own output, or on another model's output, without a real, independent signal correcting the drift. The rare cases in the original data distribution shrink a little each generation, and the model's outputs converge toward the safe, average center of what it already believes. Shumailov et al. demonstrated this across variational autoencoders, Gaussian mixture models, and large language models in "The Curse of Recursion: Training on Generated Data Makes Models Forget" (arXiv:2305.17493).

The mechanism is not exotic. Every generation step samples from the model's current distribution, and finite sampling always underrepresents the tails relative to the true distribution. Train the next model on those samples, and its estimate of the tails is worse than the last model's. Repeat that loop three or four times without a real-data anchor, and the tails are functionally gone. Your model still answers fluently; it has just quietly forgotten how to handle anything unusual.

By the time collapse shows up in your eval numbers, you have usually already baked the drift into two or three training runs, and the compute and headcount for each one already hit the P&L. This is the exact argument I work through in more depth in Synthetic Data, Carefully, because the fix is not "use less synthetic data." It is never training on synthetic output that has not been filtered against a real, independent signal.

A Worked Example: What Phi-1 Proved About Synthetic Data

Microsoft's phi-1 is the clearest public proof that curated synthetic data can substitute for raw scale. At 1.3B parameters, phi-1 trained on roughly six billion tokens of curated web data plus about one billion tokens of synthetically generated "textbook quality" content and coding exercises, produced with GPT-3.5. It reached 50.6% pass@1 on HumanEval and 55.5% on MBPP, outperforming models many times its size. The paper is "Textbooks Are All You Need" (Gunasekar et al., arXiv:2306.11644).

The lesson is not "generate more synthetic data." It is "generate it with a narrow, deliberate quality bar, and throw away everything that misses it." Phi-1's synthetic set was small relative to typical pretraining corpora and heavily curated for one specific quality: clear, textbook-style explanations, not raw scraped code. Volume was not the lever. Curation was.

How Much of Your Training Mix Should Be Synthetic

The right synthetic share depends on the stage of training, not on a fixed rule. Here is the shape I see hold up across pretraining, fine-tuning, and alignment work.

The alignment row is not theoretical. NVIDIA reports that more than 98% of the data used in Nemotron-4 340B's alignment process was synthetically generated, produced with Llama 3.1 405B and filtered through a Nemotron-4 340B reward model plus LLM-as-judge scoring and semantic deduplication via NeMo-Curator (NVIDIA, Nemotron-4 340B Technical Report, arXiv:2406.11704; pipeline detail also at the NVIDIA Developer blog). What made that 98% figure work is not the volume. It is that every stage of the pipeline had an independent filter attached before the data reached the run.

Where this sits inside your fine-tuning decision depends on the method you pick. I cover the upstream question of when fine-tuning is the right call at all separately, and once you have made that call, LoRA versus full fine-tuning shapes how much synthetic volume you need, since parameter-efficient methods tolerate a smaller, more curated set better than a full fine-tune does.

A Practical Checklist Before You Train on Synthetic Data

• Name the teacher. Which model generated this data, and is it strong enough at the task you are training for?
• Confirm the judge is independent. The model scoring the data should not be the model that generated it, and ideally not the same family.
• Deduplicate by embedding, not by exact match. Cosmetic variation on the same underlying example still counts as a duplicate.
• Compare against a held-out real sample. If you have not run this comparison, you do not know what your synthetic set is missing.
• Check the tails, not just the average. Rare-case coverage is exactly what erodes first in model collapse; measure it directly.
• Version the pipeline, not just the data. If the teacher model or the judge changes, the synthetic set's quality profile changes with it.

Frequently asked questions

Is synthetic data actually good enough to train a production model on?

Yes, when it is filtered correctly. Phi-1 and Nemotron-4 340B are both production-grade proof points, and both relied on synthetic data that passed through an independent filter, not raw generated output. The failure mode is not synthetic data itself. It is skipping the filter.

What is model collapse and how do I know if it's happening to my model?

Model collapse is the gradual disappearance of rare cases and distributional tails when a model trains recursively on generated data without a real-data anchor. You catch it early by tracking performance on a fixed, held-out set of rare or edge-case examples across training generations, rather than only watching the aggregate eval score, which can look stable while the tails quietly erode.

How much of my fine-tuning dataset should be synthetic vs. real?

There is no fixed ratio. Start with as much real, task-representative data as you can get, use synthetic data to fill specific coverage gaps you can name, and validate the mix against a held-out real sample before you train. If you cannot say which gap the synthetic data is filling, you probably do not need it yet.

How do I check synthetic data quality before I train on it?

Run it through an independent judge or reward model, deduplicate by embedding similarity, and compare its distribution against a held-out real dataset you never used in generation. If the synthetic set is missing entire categories the real data has, or is suspiciously narrow, that is the coverage gap that turns into model collapse later.

If you are building the pipeline that generates, filters, and versions training data for a real fine-tuning run, that is precisely what hiring a ViitorCloud ML engineer to build your synthetic data pipeline is for: someone who treats the judge as a separate system from the generator, from day one. Generate the data. Then judge it, with something that never wrote a word of it.