RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades

2026-06-30

RoadmapBench evaluates long-horizon coding tasks, spanning multiple files and languages, grounded in real version upgrades across 17 repositories. The benchmark tests 115 tasks, requiring agents to implement functionalities with a median modification of 3,700 lines across 51 files.