TLDR Tech / DevOps
DigitalOcean Inference Router, Now Cache-Aware: Why the Cheapest Model Isn't Always the Best Deal
DigitalOcean has enhanced its Inference Router with cache-aware routing. This feature allows the system to consider the cost of invalidating a warm prompt cache when deciding to switch models mid-session. The update introduces two new controls: an explicit model affinity header to link requests to a session and a routing budget parameter that limits the additional cost a model switch can incur.
Read the full story