TLDR Tech / Data
Effective KV Compression with TurboQuant
TurboQuant is a quantization and compression algorithm for Key-Value (KV) caches in large language models and vector search systems. It uses PolarQuant to first map vectors into polar coordinates, followed by QJL (Quantized Johnson-Lindenstrauss), which applies a minimal 1-bit correction to remove hidden biases, enabling compression down to ~3 bits per value with virtually no loss in accuracy.
Read the full story