TLDR Tech / AI
OpenAI's Jalapeño Optimizes Inference Throughput and Token Latency
OpenAI's Jalapeño inference chip was designed around its own model workloads, with early benchmarks showing higher peak throughput per kilowatt and lower token latency than the commercial systems tested on GPT-OSS 120B.
Read the full story