TLDR Tech / AI
Prompt Caching In Agents
Prompt caching makes agents more economic, but this is quite fragile. A changed tool definition, a model switch, or a provider routing decision can turn a cheap incremental request into a full replay of the context. Cache behavior affects latency, cost, tool design, session design, and even which product features should be made available. This post looks at what the cache is, how it works, and common reasons for worse cache performance.
Read the full story