TLDR Tech / Practical
Serving GLM-5.3-FP8 on MI300X with Enhanced Parallelism
This technical guide outlines how to deploy the GLM-5.3-FP8 Mixture-of-Experts model using DP8+EP8 on a single 8-GPU AMD Instinct MI300X node with vLLM v0.28.0 and ROCm 7.2.3. It includes a complete Dockerfile, necessary AITER patches, and performance data demonstrating that the data-parallel and expert-parallel configuration yields 147% greater throughput than tensor parallelism for a 96K-input and 32K-output long-context workload.
Read the full story