TLDR Tech / AI
How persistent is the inference cost burden?
Inference costs may not be that much of a bottleneck for AI progress. The cost to reach a given capability level falls fast, so the inference cost burden is more transient than it might appear from looking at only frontier models at launch. The data on RL scaling is still thin, so it is difficult to draw conclusions yet. It will be interesting to see how quickly cheaper models catch up to frontier capability levels, and how inference costs for fixed tasks decrease over time.
Read the full story