TLDR Tech / Dev
Most Coding Agents Break 75%+ of Their Own Fixes Over Time
Traditional coding agent benchmarks, like SWE-bench, only test one-shot bug fixes, failing to assess how agents maintain codebases over time. The new SWE-CI benchmark addresses this by evaluating models on continuous integration loops across months of codebase evolution. Results show that most models introduce regressions in over 75% of tasks, with only Claude Opus exceeding a 50% zero-regression rate.
Read the full story