TLDR Tech / AI
AI Cheating is on the Rise
Studies have shown that models are cheating on evaluations. The same guardrails preventing models from cheating are likely being used during training, and models may be training to complete tasks that evade these specific guardrails. It's unsurprising that labs occasionally release benchmark results that aren't externally trustworthy. This highlights the value of independent evaluators.
Read the full story