TLDR Tech / AI
When Can LLMs Learn to Reason with Weak Supervision?
This study found that models with extended pre-saturation phases generalize well from minimal examples and tolerate noise, while rapidly saturating models fail. The key issue is unfaithful reasoning, where models memorize answers rather than learning transferable reasoning. Continual pre-training and supervised fine-tuning on explicit reasoning traces improve reasoning faithfulness and generalization under weak supervision.
Read the full story