TLDR Tech / Practical
AI2 switches GPU scheduling from priority to time budgets
The AI Infrastructure team at Ai2 has replaced its priority-based GPU cluster scheduler with a new system. This system incorporates GPU time budgets, hierarchical fair-share allocation, and time-slicing contracts to prevent resource monopolization and priority inflation. Following the changes, teams obtained 98% of their allocated GPU hours, experienced shorter waits for debugging, and saw a 74% reduction in manual repairs.
Read the full story