TLDR Tech / Data
AI Evals in the Real World: Human Judging, LLM Judges, and the Gaps Between
Standard AI test scores are ineffective for customer-service bots that need to maintain conversations, understand implicit intent, and guide users to share contact information. A more effective scoring system was developed, combining human evaluation for complex aspects with LLM-as-judge auto-scoring for scalability, supplemented by human spot checks for problematic cases.
Read the full story