TLDR Tech / AI
Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs
Semantic calibration appears to emerge as a byproduct of next-token prediction. Base models are remarkably well-calibrated when using a certain sampling-based notion of semantic calibration. They can meaningfully assess confidence in open-domain question-answering tasks despite not being explicitly trained to do so.
Read the full story