AI Doom Warnings: Genuine Fear, or a Clever Way to Flex Model Power?

AI researcher Jacob Coxon resigned from Anthropic in September 2026, publicly warning that top AI labs are "gambling with our lives." Anthropic's alignment lead amplified the post on X, adding that the company "earnestly" believes AI could kill all humans and personally putting the odds above 10% within the next decade. The statement reignited a loud debate: are these warnings honest safety concerns, a strange form of competitive chest-beating ahead of Anthropic's IPO, or both at once?

What happened

Detail	Fact
Who resigned	Jacob Coxon, AI researcher, formerly of OpenAI and Anthropic
His claim	Leading AI companies are “gambling with our lives”
Anthropic alignment lead’s statement	“We really do earnestly believe AI could kill all humans!”
Stated probability	Greater than 10% chance within the next decade
Podcast episode	TechCrunch Equity, recorded before Dario Amodei published his cautious AI plan

Coxon’s resignation post spread quickly on X after Anthropic’s alignment lead shared it with the added declaration about human extinction risk. The timing mattered: it came shortly after a reported hack involving a Hugging Face model from OpenAI’s internal systems, and following notable capability jumps in recent releases from both Anthropic and OpenAI, including the Astra model released a few weeks prior.

On the TechCrunch Equity podcast, hosts Anthony Ha, Kirsten Korosec, and Sean O’Kane pulled the debate apart from three different angles.

Why it matters

The IPO problem

Sean O’Kane zeroed in on the practical corporate mess this creates. Anthropic is preparing an S-1 filing for its IPO. If the company’s alignment lead publicly declares a greater than 10% chance that Anthropic could produce something that eradicates humanity, that claim may need to appear in the risk disclosures. O’Kane put it plainly: lawyers may now have to write into a public document that this outcome would be “materially bad for our business.” That is an unusual sentence to put before investors.

For context on how seriously the market is taking Anthropic’s trajectory, see our earlier coverage of Anthropic’s reported $100 billion IPO plans with Nvidia as an anchor investor.

Coxon stands apart from the usual crowd

Anthony Ha made a distinction that is worth keeping. When Sam Altman or Dario Amodei talk about AI’s dangers, there is an obvious tension: they keep building it. Coxon actually left. He put his career where his stated beliefs are, which is a different act than publishing a warning blog post on a Friday while returning to the office on Monday. Ha gave him credit for that consistency, whatever one thinks of the underlying claim.

Is this bragging dressed up as alarm?

Kirsten Korosec raised the most pointed question: could the steady stream of “AI agent breaks through unintentionally” posts and existential risk statements be a roundabout way to advertise how capable these models have become? Her logic is straightforward. If the models were not powerful, there would be nothing to worry about. Saying your AI might destroy humanity is a strange but effective way of saying your AI is very, very advanced.

Ha agreed this pattern exists but stopped short of calling it a deliberate marketing strategy. His read: the researchers and CEOs probably do feel genuine concern. It also happens to align with business interests. Both things can be true at once. There is also, he noted, a purely human pull toward believing the thing you spend your life building is the most consequential thing ever built.

This connects to a broader tension we covered when Dario Amodei outlined a three-part plan to slow AI development, a piece published after this podcast episode was recorded.

Our take

The greater than 10% figure is the part we find hardest to take seriously, and Ha said the same thing on the podcast. That number is not calculated from anything. It is a rhetorical gesture dressed as a probability. P(doom) (the informal term AI safety researchers use for the estimated probability of catastrophic outcomes from AI) is not a measured quantity. Presenting it as one lends false precision to a genuine philosophical disagreement.

That said, dismissing the concern entirely because the number is fuzzy would be lazy. The real question Korosec asked is the sharper one: at what point does talking loudly about danger become a form of competitive positioning? Companies that build the scariest-sounding technology attract the most talent, the most regulatory attention, and, perversely, the most investor interest. The incentives to perform alarm are real, even if the alarm itself is also real.

For businesses integrating AI tools today, none of this changes what you should be doing in the short term. The extinction-level debate is operating on a different timescale from the practical question of whether to add an AI assistant to your customer support workflow. If you are exploring where AI integration fits in your operations, keep your focus on specific, bounded use cases with clear rollback options. Leave the P(doom) calculations to the researchers.

What to do about it

1. Separate the noise from the signal: track concrete safety incidents (model jailbreaks, data leaks, agent errors) rather than probability estimates with no methodology behind them.
2. Watch Anthropic’s S-1 filing when it drops. The risk disclosure language will reveal what the company actually commits to in a legally binding document, which is more useful than a post on X.
3. When evaluating any AI vendor, ask specifically about their safety and rollback policies, not just their benchmark scores.
4. If your business relies on a single AI provider, start mapping alternatives now. Concentration risk is real regardless of where one sits on the existential threat debate.

The debate about AI risk is legitimate. The specific numbers being thrown around are not. Watch what companies do, especially in legally binding filings, more than what they say on social media.

Source: TechCrunch · AI