TLDR Tech / DevOps
How OpenAI delivers low-latency voice AI at scale
OpenAI rearchitected its WebRTC infrastructure to handle real-time voice AI at scale by splitting packet routing from protocol termination, using a lightweight relay layer that forwards traffic to stateful transceiver services based on routing metadata embedded in ICE username fragments. The new split relay-plus-transceiver design reduced the public UDP surface to a small fixed number of ports (instead of one per session), enabled deployment on Kubernetes, and allowed global relay ingress points that reduced first-hop latency by letting packets enter OpenAI's network closer to users.
Read the full story