TLDR Tech / AI
Final training runs account for a minority of R&D compute spending
The final training run for a model is only the last step in a long, expensive process. Before the run, companies burn compute on running experiments at various scales, generating synthetic data, testing ideas, and training unreleased models. The full cost of developing a model is much higher than the cost of the final training run of a frontier model. Most of the spend is on exploration rather than execution. Companies that learn from their competition can replicate their results for a fraction of the original cost.
Read the full story