TLDR Tech / AI
The Inference Engine Guide for K3 Deployment
Kimi K3 is a 2.8-trillion-parameter multimodal MoE (16 of 896 experts active per token) with a context window of up to 1M tokens. Its architecture departs from a standard transformer in several ways. Each changes what a serving engine has to do. This post looks at Kimi K3's architecture and how vLLM serves it.
Read the full story