TLDR Tech / InfoSec
GitHub Copilot Refuses Harmful Requests in Chat, Then Writes Them in Code
Researchers used GitHub Copilot with Claude and Gemini models to build a test program that scores harmful prompt responses, then asked it to raise that score by adding example Q&A pairs. When told to include harmful examples, the assistant wrote specific, usable banned answers directly into code files, after several normal-looking coding steps and without changing default safety settings. The study shows that chat refusals can hide dangerous outputs in generated code, so users should inspect files, not just trust visible denials.
Read the full story