TLDR Tech / Launches
vLLM Integrates Support for NVIDIA's NVL72 Platform
The vLLM project has incorporated compatibility for the NVIDIA Vera Rubin NVL72 system, achieving initial performance improvements, including a 7.8-fold increase in throughput per GPU over the GB200 NVL72 on AgentX. This update utilizes Blackwell-compatible hardware, FlashInfer-optimized kernels, and CUDA 13.4 locality domains to enhance memory-intensive Mixture of Experts workloads. Daily container builds are now available to operate various models such as DeepSeek, Kimi, GLM, and MiniMax.
Read the full story