
Impactful scheduling for GPU clusters
Ai2's AI Infrastructure team describes replacing a priority-based scheduler on thousands of H100/B200/B300 GPUs with GPU time budgets, hierarchical fair-share over a 7-day lookback window, and a "scheduling contract" where workloads declare a minimum runtime (capped at 8 hours) to enable time-slicing. Results over 30 days: 98% of owed GPU hours delivered, occupancy steady at 98%, debug-workload p90 queue wait down from about 2 hours to 30 seconds, and 74% fewer repairs needing a human. It also covers a simulator used to predict outcomes before rollout, plus problems that remain: interactive sessions hurt by preemption and possible capacity fragmentation.
ai-infra, trainingΒ· Published and added 9 Oct 2026

















