TLDR Tech / Dev
Evaluating Netflix Show Synopses with LLM-as-a-Judge
Netflix developed an LLM-as-a-Judge system to score show synopses on four quality dimensions, anchoring it to a golden set of roughly 600 expert-labeled examples built through a rigorous calibration process. This approach, which uses judges and techniques like tiered reasoning and consensus scoring, achieves 83–92% accuracy and allows them to flag and fix weak synopses weeks before a show ships.
Read the full story