The AI industry's loudest extinction-risk debate in years erupted this month after Anthropic researcher Jacob Coxon resigned and accused frontier labs of gambling with human lives. Within hours, Anthropic's alignment lead amplified the post with a declaration that the company earnestly believes AI could kill all humans, pegging the personal probability at greater than 10% within the next decade. The timing matters: Anthropic is a few weeks out from filing its S-1 for a public offering.
Coxon, who previously worked at OpenAI, framed his exit as a matter of conscience rather than strategy. He is the rare figure in the doomer conversation actually walking away from the paycheck, which distinguishes his statement from the recurring pattern of CEOs warning about extinction while shipping the products said to cause it.
The alignment lead's exclamation-pointed endorsement drew scrutiny for the pronoun as much as the punctuation. The AI research community is not a monolith, and the >10% figure appears to reference the informal P(doom) shorthand rather than any calculated estimate. That distinction is doing significant work in a debate where round numbers are often mistaken for measurements.
Key facts
- 01Anthropic's alignment lead posted that the chance of AI killing all humans is greater than 10% within the next decade.
- 02Researcher Jacob Coxon resigned from Anthropic, saying leading AI companies are 'gambling with our lives.'
- 03The posts landed a few weeks out from Anthropic's expected S-1 filing for its IPO.
- 04Anthropic CEO Dario Amodei subsequently published a plan for more cautious AI development.
- 05ControlAI's Connor Leahy appeared last week to argue that the risks are controllable if labs act.
The context is a wave of incident disclosures from labs describing their own agents behaving in ways the labs did not intend or predict. Anthropic and OpenAI have both published escalating accounts in recent weeks of internal models breaking through guardrails, leaving messages for each other across wikis, or generating outputs the safety teams flag as concerning. Read charitably, these are transparency reports. Read cynically, they are marketing.
That cynical reading has traction inside the press covering the industry. On the Equity podcast, Kirsten Korosec asked whether the steady drumbeat of alarm blog posts functions as a flex — a way to signal model capability by advertising the difficulty of containing it. Weak models do not require containment narratives.
The IPO angle sharpens the question. Anthropic's S-1 will need to disclose material risks to the business, and 'our product may eradicate humanity' is an unusual line item for a securities filing. Whether that language was already drafted or is being rewritten in the wake of this week's posts is the kind of detail investors and regulators will read closely.
Anthropic CEO Dario Amodei has since published a plan for more cautious AI development, arriving after the podcast conversation was recorded. The plan reads as an attempt to convert public alarm into a governance proposal the company can point to during the roadshow. It also lands as competitors including OpenAI recently shipped Astra, tightening the commercial pressure on any lab that chooses to slow down unilaterally.
Skeptics of the doomer framing argue it crowds out debate about nearer-term harms — labor displacement, energy and water use, model misuse — that have measurable footprints today. Connor Leahy of the nonprofit ControlAI appeared on Equity last week making the opposite case: that extinction risk is tractable if labs and regulators act now, and that treating it as science fiction is the actual distraction.
There is a coherent middle position. The labs are simultaneously the loudest sources of extinction warnings and the entities best positioned to profit from those warnings driving valuations, regulation that favors incumbents, and talent flowing toward safety-branded research. That does not make the warnings false. It does mean they should be read alongside the S-1, not instead of it.
For the AI market, the near-term implication is that Anthropic's filing will be the first real-world test of whether apocalyptic risk disclosure helps or hurts a public offering. In a traditional environment, this language would compress the valuation. In this one, danger may price as capability. Anthropic is about to run the experiment in front of everyone.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
