TLDR Tech / DevOps
Rapidly scaling online storage to serve over 1 billion ChatGPT users
OpenAI's Habitat storage platform now handles over 70 million requests per second and serves more than 500 petabytes of data. The team recently rewrote the service in Rust, which is 6x more CPU efficient and 15x more memory efficient than the previous Python version. Habitat first launched as a Python library at DevDay 2023 to support GPTs and evolved into a complex distributed system supporting over 1 billion weekly users across nearly 40 regions.
Read the full story