TLDR Tech / AI
Embarrassingly Simple Self-Distillation Improves Code Generation
Simple self-distillation (SSD) is a process where models are fine-tuned on samples of their output with standard supervised fine-tuning. It offers a complementary post-training direction for improving LLM code generation. SSD can improve code generation in models using only raw outputs. This study looks at why such a simple method can work.
Read the full story