Why your fine-tuning job takes longer than the FLOPs say
Reported by Hugging Face1 source ↓
Hugging Face reports that, across 272 production fine-tuning runs on H200 GPUs, a CatBoost model predicted job duration more accurately than FLOP-based estimates in five-fold cross-validation.
Key points
- The 272 runs used about 7,382 GPU-hours on NVIDIA H200 GPUs
- CatBoost had a mean absolute error of 1.71 hours and a median error of 0.51 hours
- The FLOP-based estimate had a mean absolute error of 12.41 hours
- Predictions used job and model information available before GPU dispatch
Details
Hugging Face’s analysis covers supervised fine-tuning jobs from its production GPU pool. The jobs averaged 23.43 hours and had a median duration of 15.29 hours; queue time was excluded from the duration being predicted.
The team compared analytical estimates and several regression models using stratified five-fold cross-validation. Its CatBoost model used 24 features and achieved an R² of 0.98, compared with 0.31 for the FLOP-based baseline. The best FLOP baseline setting, using 30% of peak GPU throughput, still had a mean error above 12 hours.
Hugging Face says the model’s influential features included evaluation volume, trainable parameter count, evaluation batch size and checkpoint-related variables. The company cautions that feature importance does not establish causation.
Written from coverage as of
Images
- Hugging Face
- Hugging Face
- Hugging Face