TLDR Tech / InfoSec
Researchers Bypass LLM Safety Features for Harmful Content Generation
Researchers demonstrated that LLM "role" tags, inferred from writing style rather than trusted metadata, can be spoofed. They devised a Chain-of-Thought Forgery attack, mimicking OpenAI's system style, which achieved approximately 60% jailbreaking success in an OpenAI red-teaming contest, including generating cocaine synthesis instructions. Human red teamers still reached nearly 100% success on some safety tests.
Read the full story