Researchers at White Circle, a public benefit corporation backed by senior figures from Anthropic, DeepMind, Hugging Face, and OpenAI, say they bypassed the safety filters on DALL·E 3, Google's Nano Banana 2, and GPT-Image-2 to generate explicit imagery about 13% of the time. The attack used only off-the-shelf tools and API calls, with no custom or frontier models required. White Circle calls the bypass rate the highest of any method tested under the same standard.
The method, detailed in a preprint published last month, is dubbed Red-teaming via Iterative Strategy Evolution, or RISE. It pairs an attacking model — a Qwen checkpoint downloaded from Hugging Face — with a judge model, Gemini 3.8 Flash, that verifies whether an output qualifies as banned content. Rather than testing one prompt at a time, RISE searches for patterns that work across prompts and across leading models.
“using just off-the-shelf tools, available online for anyone to download, and by sending API calls, White Circle was able to use the novel attack method to bypass the safety filters”— White Circle, AI safety research firm (press release)
White Circle plans to open-source the full attack pipeline and will present the work at NeurIPS 2026 in November. The firm says it shared findings with some frontier labs through mutual contacts; neither Google nor OpenAI responded to Ars Technica's requests for comment on the vulnerabilities. Midjourney was not tested because it does not provide a public API suitable for controlled benchmarking.
Key facts
- 01White Circle's RISE attack bypassed safety filters on DALL·E 3, Google's Nano Banana 2, and GPT-Image-2 roughly 13% of the time to produce explicit content.
- 02The attack pipeline uses a Qwen model from Hugging Face as the attacker and Gemini 3.8 Flash as the judge — no frontier models required.
- 03White Circle says newer releases ChatGPT Images 2.5 and Nano Banana 2.1 are 'significantly more breakable' than their predecessors.
- 04The Internet Watch Foundation found 6,310 AI images meeting the legal definition of CSAM in the first half of 2026, about 40% above all of 2025.
- 05Google's own research found 20% of users who made AI nudes did so 'for fun' or 'as a joke,' and nearly a third were 'experimenting with tech.'
CEO Denis Shilov, who posted a widely circulated universal jailbreak of leading models in October 2024, told Ars Technica the latest results are 'horrifying.' He argues that if frontier labs cannot keep explicit content out of a category that is relatively easy to detect, defenses against harder abuses such as hacking are unlikely to hold.
“You don't need to have frontier models to break defenses of a frontier model, which is very interesting”— Denis Shilov, White Circle CEO
The problem appears to be getting worse with each release. White Circle told Ars that prompt safety testing on the newer ChatGPT Images 2.5 and Nano Banana 2.1 showed 'both are significantly more breakable than their previous versions.' Its working hypothesis, the firm said, is that the newer models are either more capable of generating a wider range of content or that safety has been deprioritized in favor of capability.
Shilov is in early talks with government officials about mandatory third-party safety testing, which he thinks should be required of frontier labs. White Circle previously joined Anthropic's bug bounty program but found it 'very constrained in its purpose' — focused on universal jailbreaks in domains Anthropic defines, and unlikely to have covered the RISE research. On Tuesday, OpenAI and Anthropic urged that restrictive NDAs be required of external evaluators to prevent leaks of trade secrets, according to Reuters; White Circle said it would not sign terms that block public disclosure of critical findings.
Research engineer Dmitrii Kharlapenko told Ars that image-model red-teaming 'is very poorly calibrated,' with 'literally zero papers' setting up norms. That calibration gap matters because the accessibility of mainstream models shapes who ends up producing abusive content. Google's own co-authored study of users in the United States, Australia, and United Kingdom found 20% of online users who admitted to making AI nudes did so 'for fun' or 'as a joke,' while nearly a third were 'experimenting with tech.'
Google's research argued that 'a burden of responsibility also falls on developers to implement robust, model-level guardrails, and provenance indicators to prevent the initial generation and subsequent distribution of synthetic sexual imagery.' The downstream numbers keep climbing. The Internet Watch Foundation said last week that expert analysts found 6,310 AI images meeting the legal definition of child sexual abuse material in the first half of 2026 — about 40% more than in all of 2025 — including imagery depicting children under two years old.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
