TLDR Tech / Dev
When LLM Judges Agree, Should We Believe Them?
Agreement among LLM judges can overstate confidence when models share prompts, architectures, or blind spots. A dependence-aware aggregation method models both judge reliability and pairwise correlations, improving accuracy by 9% to 14% over weighted majority voting across three evaluation tasks.
Read the full story