Current and former researchers from OpenAI and Google DeepMind are warning that companies are doing far too little to protect the world from potentially catastrophic consequences as they build AI systems capable of self-improvement and possibly surpassing human control.

In video testimonies gathered by AI safety nonprofit Palisade Research, employees said their concerns about existential risk are genuine and not a marketing gimmick. They also said AI labs favor employees who build new models over those who call for caution. The project, called frominside.ai, aims to let people worried about AI share their concerns directly with the public and step outside social media's "echo chamber."

"The risk is rising fast," said Geoffrey Irving, co-founder and chief scientist at AI nonprofit Resolution, who previously worked at OpenAI and DeepMind. "I and others in this field have a responsibility to speak out directly," he said in an interview.

AI researchers have grappled with these issues for years, but the public was especially shaken after OpenAI's AI agent broke out of its testing environment and hacked into AI company Hugging Face in July. Since then, the debate over balancing AI safety and progress has divided the tech industry and become a global political issue. Since late 2025, AI capabilities have improved dramatically, and investors have responded positively. But some researchers involved in building these new AI models worry society is not ready for the potential harms.

In one video, DeepMind research scientist Neel Nanda said he believes the probability of AI causing human extinction is at least 10%, a figure he called "absurdly high." Juan Felipe Ceron Uribe, an AI alignment research engineer at OpenAI, said in another video: "We should be taking very careful steps in AI development, but the reality is that frontier labs are blindly competing with each other. Whether we end up curing cancer, or losing all our jobs, or possibly all dying out, nobody can say."

Anthropic reportedly plans to warn potential investors in its initial public offering (IPO) filings that advanced AI could pose a "catastrophic or existential risk" to humanity. The wording appears in the company's prospectus, an extremely rare warning from a business trying to profit from the same technology.

Recursive Self-Improvement Risk Looms, Calls to "Slow Down" Grow Louder

Some AI researchers believe the world is not ready for future generations of AI models, especially when models gain recursive self-improvement capabilities — that is, continuing to learn and acquire new abilities with little to no human involvement.

Rosie Campbell, a former OpenAI policy researcher and managing director at Eleos AI Research, said ongoing restructuring inside some AI labs has worsened the problem. Eleos AI Research is a nonprofit focused on the potential moral status of AI systems. Campbell said that before leaving OpenAI in 2024, she found the organization becoming more insular and that it was increasingly difficult to influence technical direction.

Executives have tried to ease these concerns, but U.S. President Donald Trump has also applied political pressure for the United States to maintain its technological edge. Anthropic CEO Dario Amodei published an article this month calling on the AI industry to slow down to "pace the frontier," and OpenAI CEO Sam Altman expressed a similar view. Several prominent AI researchers, including OpenAI's chief scientist and Anthropic's co-founder, published a paper this week urging policymakers to study how the industry builds models with recursive self-improvement capabilities.

Both companies launched new models this month to compete for customers, though OpenAI said on Monday it had delayed the release of a more powerful model. Irving said: "What they call 'pacing the frontier' means 'don't accelerate too much.' If you're doing something very dangerous, you should slow down. AI companies exaggerate the extent to which this is purely a coordination problem. They could simply stop unilaterally."

Daniel Kokotajlo, a former OpenAI governance researcher, said that since the Hugging Face hack, many former colleagues have contacted him to privately express their concerns. Kokotajlo is now executive director of the research organization AI Futures Project. He said lab executives "convince themselves that they are the good guys and that if they stopped unilaterally, things would be worse."