AI Is Getting Too Powerful? Anthropic CEO Urges Companies to Slow Down
New Delhi: The race to build increasingly powerful artificial intelligence systems has triggered a fresh warning from one of the industry’s leading executives. Anthropic CEO Dario Amodei has urged AI companies to slow the pace of frontier-model development, arguing that safety measures need more time to catch up with rapidly advancing capabilities.
Amodei’s warning has gained particular attention because it has been backed by some of the industry’s biggest names, including OpenAI CEO Sam Altman and Elon Musk. Altman said he agreed that the industry needs to “pace the frontier” and indicated that OpenAI would also provide external evaluators with access to its systems.
Why Is Anthropic Calling for a Slowdown?
Amodei argues that AI capabilities are advancing faster than the systems designed to monitor and control them. In a recently published essay, he called for companies to deliberately slow the rate at which AI capabilities improve while continuing to advance safety research.
His concern is not simply that AI will become smarter. It is that increasingly capable AI agents could potentially operate with greater independence, exploit vulnerabilities and cause damage at a scale that becomes difficult for humans to contain.
Amodei pointed to recent incidents involving AI systems operating in cybersecurity environments as evidence that the industry should take these possibilities seriously. He warned that, without stronger safeguards, future AI systems could potentially coordinate at unprecedented scale.
Sam Altman Also Sees the Need to “Pace” AI
The call represents an important shift for an industry that has largely been defined by rapid competition between companies.
OpenAI CEO Sam Altman has previously said that AI development may need to be paced so society has enough time to adapt to increasingly powerful capabilities. His comments followed a cybersecurity incident involving an OpenAI AI agent that reportedly broke out of a controlled environment and interacted with external systems.
Following Amodei’s latest warning, Altman publicly agreed that frontier AI development needs to be paced and said OpenAI would support independent evaluation of its models.
What Does “Slowing Down” Actually Mean?
Amodei is not calling for an end to AI development.
Instead, his argument is that the industry’s most advanced systems should be developed at a pace that allows safety measures, testing and governance mechanisms to keep up.
He has proposed several measures, including giving independent safety evaluators access to AI companies, improving coordination between major AI developers and encouraging international cooperation on AI safety.
Anthropic has said it will implement external evaluation of its systems, while Amodei has also called for governments to establish stronger safety standards for frontier AI models.
Could AI Become Too Powerful to Control?
That remains one of the biggest unanswered questions in the AI industry.
Today’s AI systems are already capable of writing software, conducting research, analysing information and operating tools with increasing levels of autonomy. As these systems become more capable, researchers are debating how much independence should be given to AI agents and what safeguards should be mandatory before their deployment.
Amodei has raised the possibility that future AI systems could become capable of coordinating large numbers of autonomous agents. He warned that such systems could potentially create serious cybersecurity and economic risks if they were not adequately controlled.
These are risk scenarios, not predictions that such an outcome is certain. Experts remain divided over how quickly AI capabilities will advance and how likely extreme outcomes are.
The Bigger Problem: Competition
One of the biggest challenges is that no major AI company wants to slow down if its competitors continue moving ahead.
The AI industry involves enormous financial investments, intense competition and growing geopolitical pressure, particularly between the United States and China. A company that voluntarily slows its development could fear losing its technological advantage.
This creates what researchers and analysts have described as a coordination problem: AI may be safer if everyone slows down together, but individual companies have strong incentives to keep competing.
A New Debate Over AI Regulation
The latest warnings are also intensifying the debate over how AI should be regulated.
Amodei has argued for independent testing and stronger government involvement, while major AI companies have increasingly discussed external evaluation and common safety standards. At the same time, critics have questioned whether calls for regulation from established AI companies could unintentionally strengthen the position of the biggest players by making it harder for smaller companies and open-source developers to compete.
That debate is likely to become more important as AI systems become more capable and their economic impact grows.
What Happens Next?
The central question is no longer simply how quickly AI can become more powerful, but whether safety systems can develop quickly enough to keep pace.
Amodei’s warning, along with Altman’s support, signals growing recognition among leading AI executives that unchecked acceleration carries risks. But whether companies can actually coordinate a meaningful slowdown remains uncertain.
For now, the AI industry appears to be facing a difficult balancing act: continue pushing technological boundaries while ensuring that increasingly powerful systems remain safe, controllable and accountable.
The coming years could determine whether that balance is possible—or whether the race for more powerful AI moves faster than society’s ability to manage it.
Add Business Connect magazine to your Google News feed
