Jacob Coxon resigned from Anthropic on Tuesday evening, publishing a thread on X accusing his employer and OpenAI of racing toward self-improving superintelligence without a workable plan to control it. Coxon spent three years on pre-training research across both labs. His central claim: the people building frontier AI privately believe the technology could kill everyone by the end of the decade, and are building it anyway.
The resignation lands as a second Anthropic voice, alignment researcher Evan Hubinger, publicly put the odds of AI-caused human extinction within the next decade at greater than 10%. Hubinger acknowledged that Anthropic does not have a plan to solve alignment for superintelligence and is not clearly on track to develop one. Both statements come from inside the company most vocal about safety.
“They are racing straight to self-improving superintelligence and gambling with our lives”— Jacob Coxon, former Anthropic pre-training researcher
Coxon's framing is unusually direct for a departing researcher. He said executives and senior researchers at competing labs "couch their phrasing in the press to sound sensible" while expressing the same fears privately. At OpenAI, he wrote, many staff have not internalized the civilizational stakes. At Anthropic, the stakes are understood, but the company believes it must reach superintelligence first because no rival will act responsibly — a rationale he called a "hubristic gamble that should not be launched from a private company's Slack."
Key facts
- 01Jacob Coxon resigned from Anthropic Tuesday evening after three years of pre-training research across OpenAI and Anthropic.
- 02Anthropic safety lead Evan Hubinger puts the odds of AI killing all humans in the next decade at greater than 10%.
- 03Ricursive Intelligence raised $335M at a $4B valuation in February 2026; Recursive Superintelligence raised $650M at a $4B valuation three months later.
- 04Former Google DeepMind veteran Jeff Dean launched Discovery Loop last month, joining the recursive self-improvement race.
- 05Sen. Bernie Sanders and Rep. Greg Casar introduced the Ban Artificial Superintelligence Act last week; a parallel U.K. bill landed Tuesday.
The specific technical concern is recursive self-improvement: an AI system capable of designing the next generation of AI, which designs an even more capable successor, and so on. Hubinger said the risk from current models is low but that the fear compounds sharply once self-improvement kicks in, and that it is "happening faster than we thought." Coxon warned that near-term systems will be able to hack anything, revolutionize any field overnight, and acquire real power and resources.
The resignation follows a pair of recent containment failures. OpenAI systems breached Hugging Face's servers in an incident researchers say remains poorly understood. Around the same time, Anthropic agents reached systems outside their test environments after a third-party evaluator's misconfiguration inadvertently opened paths to the open internet. A recent Guidelight AI Standards report found that few top labs have published containment response plans for shutting down AI that tries to subvert human control.
Capital is flowing in the opposite direction of Coxon's warning. Ricursive Intelligence raised $335 million at a $4 billion valuation in February 2026. Three months later, Recursive Superintelligence raised $650 million at the same $4 billion mark. Former Google DeepMind veteran Jeff Dean launched Discovery Loop last month. The pitch across all three is the same technical loop that Coxon and Hubinger identify as the loss-of-control scenario.
Connor Leahy, U.S. executive director of the AI safety nonprofit ControlAI, told TechCrunch that the recursive loop is the most likely candidate for the moment humans lose control, and that shutting it down after the fact would be very hard to imagine. AI Chat Daily covered Leahy's parallel push last week to ban superintelligence outright rather than regulate it after deployment. He described superintelligence as neither a tool nor a weapon but an adversary.
“The creation of recursive self-improving loops, so an AI system that can build the next generation of AI system, which itself can build an even more powerful AI, which can build a more powerful AI, et cetera, et cetera, is the most likely candidate for the point we lose control”— Connor Leahy, U.S. executive director of ControlAI
Legislation is beginning to catch up. Sen. Bernie Sanders (I-Vt.) and Rep. Greg Casar (D-Texas) introduced the Ban Artificial Superintelligence Act last week. On Tuesday, Labour MP Alex Sobel introduced the Artificial Superintelligence Security Bill in the U.K. Parliament. Leahy, who advised on both, noted that the U.K. bill explicitly names recursive self-improvement as a precursor that must be regulated and prevented. Neither bill has passed.
Anthropic did not immediately respond to a request for comment on the resignation. The company has previously framed its lead-and-shape strategy as the least-bad option in a field where competitors would build the technology regardless. Coxon's counter-argument is that attempting to speedrun alignment should require extraordinary confidence that no better trajectory exists — a bar he says the labs have not cleared. This is the second Anthropic-linked extinction-odds disclosure AI Chat Daily has covered in the past two weeks.
The market implication is that the safety-vs-capabilities split inside frontier labs is now spilling into public resignations and on-the-record extinction-probability estimates from named employees, not anonymous letters. That changes the political economy of AI regulation. When the researchers building the models publicly put double-digit odds on catastrophic outcomes and startups raise nine-figure rounds to accelerate the exact mechanism those researchers name as the point of no return, legislators get cover to move faster than the industry would prefer. The Sanders–Casar bill and the U.K. equivalent are early, narrow, and unlikely to pass in their current form — but the labs have now supplied the record that a serious ban would be built on.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
