TLDR Tech / Practical
New AMD CDNA4 attention kernel optimizes power efficiency
Moonmath AI has released an open-source bf16 forward attention kernel for AMD's CDNA4 architecture, written in HIP. The kernel optimizes energy consumption per FLOP under a 1400 W power limit by adjusting wave scheduling, matrix instructions, and softmax. This results in 1.08x to 1.19x faster performance than existing assembly kernels, with a 12% to 13% increase in clock speed.
Read the full story