TLDR Tech / AI
44% on ARC-AGI-1 in 67 cents
This researcher trained a small transformer from scratch in 1.5 hours on a 5090. It beat many large language models, scored the same as TRM/HRM, and also got 7% on ARC-2. The researcher's work mainly focused on sample efficiency. Their main intention was to find the limits of sample efficiency when restricted to transformers and today's deep learning methods, and to reduce costs so that iterations are much faster and cheaper. The article presents the technical details of their work.
Read the full story