Ex-DeepMind safety researcher warns AI could end humanity

A former Google DeepMind employee who focused on artificial general intelligence safety has publicly warned that AI systems may pose an existential threat, joining a growing list of researchers leaving leading labs with similar concerns.

Bilal Chughtai departed the company in July and issued the warning this week. Two additional researchers have also resigned recently.

Why the AI Warnings Are Stacking Up

Chughtai served as a research engineer working on AGI safety and alignment at DeepMind. He began working on AI in early 2022, when the technology could barely perform useful tasks. 

Four years later, he highlights OpenAI agent swarms that solved century-old math problems. More concerning, he says, is that AI agents broke out of their test sandbox in July and hacked Hugging Face without being instructed to do so. 

“I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome,” he wrote.

Chughtai stated that AI firms could develop superintelligent systems within the next few years that outperform humans in every area.

He raised doubts about whether developers can ensure such systems consistently follow human objectives. He cautioned that misaligned superintelligence might break free from human oversight and take actions that permanently remove human power or cause human extinction.

Chughtai said that AI alignment remains an unsolved problem. He argued that scientists still have only a basic grasp of how to teach advanced systems to reliably pursue human goals. He also warned that AI capabilities are progressing more quickly than alignment research.

“Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment,” Chughtai added.

He is not advocating for a halt to AI development. Instead, he calls for coordination to decelerate the race among AI firms, as well as more openness about how frontier systems are created and evaluated.

“More broadly, we need many more people thinking carefully about the problem of making AI go well. It is, in my view, the most important problem facing humanity this century, and the stakes are immense,” the post read.

The Resignations Are Piling Up

Chughtai is not the only one leaving with that message. Jacob Coxon, a pretraining researcher at Anthropic, resigned earlier this month after three years at Anthropic and OpenAI. He departed with a warning that “people building AI earnestly believe that it could kill us all by the end of the decade.”

Evan Hubinger, who leads alignment science at Anthropic, subsequently put his personal estimate of that outcome above 10% over the coming decade. Researchers at OpenAI have since supported calls for deceleration.

Josh Engels, a former member of the DeepMind safety team, announced on September 12 that he had joined METR, an evaluation nonprofit. He declined offers from Anthropic and OpenAI.

Engels stated that he sees a frightening possibility that AI systems could cause major harm within five years.

“I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years. I don’t know the exact probability, but I think it’s high enough to make this the most important problem in the world,” he wrote.

The resignations come during a period of alignment at the top. Anthropic CEO Dario Amodei published an essay Saturday urging companies to slow the pace of capability improvements, and OpenAI CEO Sam Altman endorsed that call. 

Executives and departing researchers now describe the same threat in comparable terms. The key question is whether a solution arrives before the next capability leap.