TLDR Tech / Data
Smevals - a Small Eval Suite for Evaluating Models, Prompts, and Harnesses
Smevals is a lightweight Python CLI for running task-specific evals across multiple models to find the cheapest one that meets a quality bar. You define tasks, write YAML graders with custom checker scripts, run them against a set of model configs, and view scored results in a local dashboard or static site.
Read the full story