TLDR Tech / AI
DeepsecBench: evaluating model performance in finding cybersecurity vulnerabilities
DeepsecBench is a benchmark that evaluates how well different models find cybersecurity vulnerabilities in application code. The report includes recall, precision, cost, and total time for each model and combines recall and precision into a single benchmark score. DeepsecBench runs on an open-source codebase at a commit state just before a large number of vulnerabilities were fixed. The construction of the benchmark is secret so models aren't able to train against it.
Read the full story