- Distillation lets a small, cheap model learn to imitate a big, expensive one, capturing much of its skill at a tiny fraction of the cost.
- It is why a frontier model that cost hundreds of millions can be partly copied for thousands, which quietly erodes the moat around the leaders.
- It is also a legal and security flashpoint, with OpenAI, Google, and Anthropic accusing rivals of distilling their models against the rules.
The Setup
How does a small lab ship a model that rivals a giant's, for a rounding-error budget? Often the answer is distillation. It is one of the most important and least understood ideas in AI right now, and it explains both why open models keep catching up and why the biggest labs are suddenly fighting each other in court.What Distillation Actually Is
Distillation is teaching a small model to imitate a big one. You take a large, capable teacher model, have it answer a huge number of questions, and use those answers to train a smaller student model to produce the same outputs. The student does not learn from scratch, it learns from the teacher's behavior and copies its patterns. Done well, the small model ends up with much of the big model's skill, and it is far cheaper and faster to run.Why It Is So Cheap
The cost gap is the whole point. Training a frontier model from scratch can cost hundreds of millions in compute. Distilling its behavior into a small open model can cost a tiny fraction of that. DeepSeek showed this at scale, reportedly fine-tuning a strong distilled model for around ten thousand dollars, and distilled models can run at 5 to 20 times lower cost per token. You skip the expensive part, the original training, and inherit the result.Why This Scares the Big Labs
This is where the moat cracks. A frontier lab spends a fortune to build a lead, then a competitor can copy a lot of that capability just by studying its outputs, with no giant training run of their own. It even works around chip sanctions, since you do not need a massive cluster to distill, only access to a good model's answers. The reward for being first gets shorter, which is exactly what keeps closed-model executives up at night.The Legal and Security Fight
So it has turned into a battle. OpenAI accused DeepSeek of distilling its models against its terms of service, harvesting outputs through disguised accounts, and US authorities have been probing possible trade-secret violations. Anthropic said it caught industrial-scale distillation aimed at Claude, more than 16 million exchanges run through roughly 24,000 fraudulent accounts by three labs. Most major providers now ban using their outputs to train competitors. Enforcing that across borders is the hard part.What It Means For Investors
Distillation reshapes the moat question. If capability leaks through outputs, a pure model lead is hard to defend, and value shifts toward things that do not distill easily: proprietary data, distribution, brand, and the infrastructure to serve at scale. It also pressures pricing, since a cheap distilled clone caps what anyone can charge for being slightly ahead. For the open-source side, it is rocket fuel. For the leaders, it is a leak they cannot fully plug.FAQ
Is distillation the same as just copying the model?No. You do not get the original weights, you train a new model to behave like it by learning from its answers. The student is its own model, it just inherited the teacher's style and skill. That distinction is also why the legal questions are messy.
If it is so easy, why are frontier models still ahead?
Because distillation copies what a model already does. It cannot copy the ability to push the frontier forward, and the teacher still has to exist first. Distillation narrows the gap behind the leader, it does not move the leader.
What should I watch as an investor?
Watch what cannot be distilled. Unique data, distribution, and serving infrastructure hold value when raw model capability leaks. And watch the legal fights, because how courts treat training on a rival's outputs will shape who can copy whom.