TLDR Tech / Dev
Review: Measuring AI Ability to Complete Long Software Tasks
The "time horizon" metric measures AI progress by the time a human expert needs to complete a task that an LLM can now solve, with success rates currently doubling every seven months. While this suggests AIs may soon tackle month-long expert tasks, these benchmarks often overlook the complex, non-automated realities of real software development.
Read the full story