TLDR Tech / Data
Predicting model behavior before release by simulating deployment
OpenAI's Deployment Simulation is a new pre-release safety technique that replays real (de-identified) user conversation prefixes from previous deployments and generates responses with a candidate model to predict how it will actually behave in production, providing far more realistic coverage and better rate estimates of undesired behaviors than traditional evaluations.
Read the full story