TLDR Tech / AI
MiniMax Sparse Attention for Million-Token Contexts (GitHub Repo)
MiniMax Sparse Attention is a sparse attention architecture that uses group-specific Top-k block selection to scale long-context inference while preserving model quality. On a 109B multimodal model, it matched GQA performance while cutting attention compute by ~30x at 1M tokens.
Read the full story