TLDR Tech / Data
G-Eval, Explained
G-Eval evaluates open-ended text by having an LLM generate its own scoring steps from a rubric via chain-of-thought, then calculates a weighted expectation over token-level probabilities for each score. Key practices for reliable results include using a judge model from a different family and calibrating rubrics with human-labeled examples.
Read the full story