TLDR Tech / Data
Garbage Day: How Pinecone Safely Deletes Billions of Objects at Scale
Pinecone uses immutable storage where every write creates a new file, which leaves behind stale or orphaned data that increases storage costs. They built Janitor, a system that safely identifies, verifies, and deletes unused objects at scale using cautious multi-phase checks and auditability to avoid deleting data still in use.
Read the full story