Accelerating LLM Inference with Prompt Caching for Open‑Source Models on Databricks

2026-05-25

Databricks rolled out automatic prompt caching for open-source LLMs including Llama, Mistral, and DBRX models, reducing redundant processing of repeated prompts to cut costs and latency without requiring any customer configuration.