TLDR Tech / AI
A brief history of distillation in AI
Distillation started as a way to compress large models into smaller, cheaper ones, but has evolved into a core post-training technique for transferring instruction-following and reasoning abilities from frontier models. Methods used by DeepSeek, Qwen, and GLM now sit at the center of both open model development and ongoing disputes over whether training on proprietary model outputs constitutes unauthorized copying.
Read the full story