It passed CI. It passed your evals. The customer still got the wrong answer

2026-09-14

AI agents can return successful, well-grounded responses that are still wrong because they retrieved invalid context or followed a faulty execution path. Debugging requires end-to-end traces of model calls, tool calls, retrieval inputs, and outputs, plus separate tests for deterministic system behavior and answer quality.