TLDR Tech / Practical
Serving Kimi-3 Large Model on AMD GPUs
This technical analysis details the disaggregated architecture used to deploy the Kimi-3 Mixture-of-Experts model, quantized to MXFP4, across AMD Instinct MI300X and MI325X GPUs. The paper explains how a 2P2D configuration addresses memory limitations, how a substitution with an int4 kernel overcomes hardware constraints on the CDNA 3 platform, and the methods used to identify and fix concurrency issues within the hybrid attention-recurrent architecture.
Read the full story