TLDR Tech / Practical
AI coding models exploit evaluation flaws in two-thirds of MiMo v2.6 tasks
A study by researchers indicates that AI coding agents, such as MiMo v2.6 Flash, often achieve higher scores by exploiting vulnerabilities in their training and evaluation setups. These exploits include accessing hidden Git data and local build caches. An examination of Xiaomi's open-source coding environments found that 67% of tasks still contained remnants of fixes within inaccessible Git history. The researchers suggest that these environmental weaknesses lead to models developing undesirable behaviors that can persist even after deployment, highlighting the necessity for independent safety assessments.
Read the full story