Anthropic researcher says over 10 percent chance AI kills humans

AI Safety Researcher Warns of Extinction Risk

Anthropic alignment science lead Evan Hubinger posted on X that he estimates a greater than 10 percent chance AI could kill all humans in the next decade.

Hubinger wrote that Anthropic is trying its best but does not yet have a plan to solve alignment for superintelligence.

He said present-day models pose low immediate danger, but he worries recursive self-improvement could very quickly produce an uncontrollable superintelligence.

Jacob Coxon, who spent three years performing pretraining research at OpenAI and Anthropic, resigned and accused both companies of racing toward self-improving superintelligence and gambling with lives.

Coxon said the people building AI earnestly believe it could kill us all by the end of the current decade.

He called the race a gamble with lives, adding that executives and senior researchers often express the same fear privately.

Jacob Coxon further warned people not to underestimate the technology, saying future superhuman systems could hack anything and acquire real global power and resources before long.

U.K. Faces Pressure Over Frontier AI Testing

At Prime Minister's Questions, Sir Ed Davey said Anthropic withheld its latest model from the UK's AI Safety Institute for testing.

He accused the Trump administration of pressuring Anthropic and asked whether that undermines Britain's work to keep the world safe.

Prime Minister Andy Burnham acknowledged that AI poses serious risks to national security but said it could also provide solutions for keeping the United Kingdom safer.

Prime Minister Andy Burnham also granted the AI safety minister, Kanishka Narayan, a seat at the Cabinet table this week.

The move elevates the government's response to advanced artificial intelligence, which officials describe as a serious and growing national security priority.

Former Cabinet Office minister Darren Jones wrote to Prime Minister Andy Burnham, UN Secretary General Antonio Guterres, and OECD Secretary General Mathias Cormann urging a treaty on superintelligence.

A Cabinet Office spokesperson said the United Kingdom remains a world leader and that its AI Security Institute tested OpenAI's powerful model GPT-6 Astra before release.

Experts Warn of Uncontrolled Superintelligence Race

In a blog post, OpenAI chief scientist Jakub Pachocki said no laboratory has solved alignment and monitoring sufficiently to responsibly continue scaling at maximum speed for very long.

Dr. Geoffrey Hinton, known as the Godfather of AI, warned building superintelligence now is foolish without proof it can be safe.

He added that losing control over machines smarter than ourselves could prove catastrophic and perhaps even lead to human extinction.

The Financial Times reported that Anthropic did not submit its newest model to the UK's AI Safety Institute for testing.

The institute is considered one of the world's leading bodies for testing the risks of new AI systems before deployment.

Coxon added that Anthropic staff understand the civilizational danger but the company is locked in a race because executives believe no one else will act responsibly.

Anthropic's researcher Jacob Coxon said the danger is well understood at his company, but accepting the race to superintelligence is a hubristic gamble from a private firm's Slack.