TLDR Tech / AI
I built a vulnerable app and spent $1,500 seeing if LLMs could hack it
This developer created a vulnerable book review app to see if LLMs could find a flag in users' private reviews by reproducing a common class of exploits. GPT-5.5 performed the best, solving the task in seven out of 10 runs. DeepSeek-V4-Pro was the runner-up with only three successful runs. Claude Sonnet 4.6 was the most expensive model to run, and it only solved the task on two runs, but five of the runs stopped because of the max budget. Many models could not complete the task due to security guardrails.
Read the full story