TLDR Tech / DevOps
Keeping the KV cache warm: Measuring prompt cache eviction across Anthropic, OpenAI, and Google
Prompt caching can dramatically reduce latency and cost, but cache lifetime differs significantly across providers and is often undocumented. Measurements found Anthropic's default cache remained warm for roughly five minutes, OpenAI's automatic prompt cache typically expired after 5–10 minutes of inactivity, and Google's implicit Gemini cache showed more variable eviction behavior, highlighting that production systems with long pauses between requests may need explicit cache-warming or provider-specific caching strategies to maintain high cache hit rates.
Read the full story