TLDR Tech / AI
22580: From GPT2 to Kimi3, Explained
KimiK3's gains come from more than scaling: it combines constant-state Kimi Delta Attention, periodic softmax retrieval, sparse experts, and selective residual access. Each architectural step improves how fixed-capacity memory stores, forgets, and retrieves information while preserving efficient inference.
Read the full story