TLDR Tech / IT
Lightbits targets the KV-cache bottleneck in AI inference
Lightbits unveiled Inferra, a KV-cache orchestration engine designed to improve GPU utilization for long-context and multi-session inference workloads. The company says the storage layer can help AI infrastructure providers serve larger workloads without adding GPUs at the same rate.
Read the full story