TLDR Tech / Data
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta scaled its ads recommendation foundation model 4x in 12 months and doubled training efficiency to 20–25% MFU through five techniques: jagged flash attention to eliminate padding waste, a unified attention kernel with 2–3.5x speedup, MXFP8 precision training, SM-free collective communication that reclaims ~23 SMs, and a load-balancing shuffle that adds 4% efficiency at zero communication cost.
Read the full story