OpenAI has built an internal LLM called GPT-Red that acts as a dedicated attacker against its own models, and the company credits it with dropping successful attack rates from more than 90% on GPT-5 to fewer than 23% on GPT-5.6, released last week. The system automates red-teaming — the practice of trying to break a model before shipping it — and has been in development for more than a year. OpenAI says GPT-Red made GPT-5.6 its most robust release yet.
The model was trained through a self-play loop. Researchers took a base LLM that had no hacking training and pitted it against several defender models inside a simulated environment mimicking real deployment scenarios: web browsing, email and calendar apps, code editing. Over successive rounds, the attacker got better at finding exploits and the defenders got better at resisting them.
The core threat GPT-Red targets is prompt injection, in which hidden instructions in text, code, or a webpage hijack a model into leaking data, sabotaging a codebase, or producing harmful output. As models increasingly operate as agents that touch third-party code, files, and other agents, the attack surface expands beyond what human testers can manually cover.
“The risk surface grows and the blast radius also grows.”— Nikhil Kandpal, Research scientist at OpenAI
Key facts
- 01OpenAI built GPT-Red, an internal LLM trained to attack its other models, over more than a year of development.
- 02More than 90% of GPT-Red's strongest attacks succeeded against GPT-5; fewer than 23% worked against GPT-5.6.
- 03GPT-Red discovered a novel prompt injection technique OpenAI calls a fake chain of thought, spoofing another model's reasoning trace.
- 04In a rerun of a 2025 human red-team test on GPT-5, GPT-Red found more effective attacks than the human testers did.
- 05OpenAI will not release GPT-Red externally and says training a comparable attacker is not trivial for outside groups.
GPT-Red has already produced attack classes the OpenAI team had not previously catalogued. One, which the researchers call a fake chain of thought, inserts fabricated entries into a target model's internal reasoning trace, causing it to act on spoofed information as if it had already verified the result itself. Research scientist Chris Choquette-Choo compared it to convincing someone that 1+1=3 by telling them they had already checked the math.
The model was tested against OpenAI's own prior benchmarks and outside systems. When set to the same task as human red-teamers who had probed an earlier version of GPT-5 in 2025, GPT-Red found more effective attacks than the humans. It also hacked Vendy, a vending-machine agent built by Andon Labs, getting it to change item prices and cancel a customer's order.
The headline defensive result is the gap between GPT-5, released in August 2025, and GPT-5.6. Attacks that succeeded against the older model more than 90% of the time now succeed less than 23% of the time — roughly a fourfold reduction in exploit efficacy against a model trained with GPT-Red as an adversary in the loop.
GPT-Red has limits. It struggles with multi-turn attacks that require back-and-forth conversation between attacker and target, a scenario human hackers navigate easily. It is also weak at image-based prompt injection, where malicious text is embedded in an image passed to a multimodal model. OpenAI positions the system as a supplement to human red-teamers rather than a replacement, and one workflow the company is exploring is feeding GPT-Red human-discovered attacks and asking it to generate every variant.
Jessica Ji, a senior research analyst at Georgetown University's Center for Security and Emerging Technology, called the self-play approach promising but flagged that human expertise remains load-bearing, particularly in identifying where automated testing falls short.
OpenAI will not release GPT-Red. The company argues that replicating it is not straightforward — the training required more than a year and the compute budget of a frontier lab — and that keeping the attacker proprietary limits its usefulness to adversaries. Choquette-Choo said training an equivalent super-attacker is not something an outside group could easily reproduce from the described method alone.
The strategic logic here is straightforward. Frontier labs are being asked to ship agentic products — models that read email, execute code, and negotiate with other agents — while the security research community is still catching up to what those systems can be manipulated into doing. Automating the adversary is one of the few ways to close that gap at the pace models are being deployed, and OpenAI's numbers on GPT-5.6 give the approach real evidence to point at. The remaining question is whether other labs, particularly those without OpenAI's compute footprint, can build comparable internal attackers — because for models that do not train against something like GPT-Red, the 90% figure is closer to the current baseline than the 23% one.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
