Anthropic CEO Warns AI Is Becoming Too Powerful—As It Releases Stronger AI

Anthropic CEO Dario Amodei said on Wednesday that governments can no longer treat AI laws as a problem to be learned and that the United States needs safeguards for the most powerful AI models.

In storyentitled “Policy on the AI ​​​​Exponential,” Amodei argues that transparency requirements are not enough and calls for binding rules for AI systems of the limit.

“AI is advancing rapidly – ​​in just four years, AI models have gone from being unable to write a line of communication to writing most of the code for major AI companies,” Amodei wrote.

Amodei’s story comes across as Anthropic they grow up to reach the Claude Mythos and to initiate of Mythos 5 on Tuesday, its limited edition AI for cybersecurity agencies and government partners. Researchers, including from the UK AI Security Instituteit found that it could accidentally destroy cyber problems.

According to Amodei, his ideas borrow from the procedures used by the Federal Aviation Administration.

“Future AI models, such as airplanes, should be tested and evaluated, and their release should be blocked or modified as a threat to public safety if they do not meet high safety standards,” he wrote. “I’m glad to see the Trump administration Executive Order move more and more to a central government role in AI, although the Anthropic concept is also encouraging action. “

Under Amodei’s proposal, the regulatory system would require third-party approval testing of advanced AI models, government officials to prevent unsafe deployments, and requirements for companies to protect model weights, test security, and report critical incidents. They also call on governments to plan for AI-driven job migration and advances in drug development, reduce oversight and independent instruments in domestic law enforcement, and promote cooperation among democratic nations on critical AI technologies.

“First of all, enduring layoffs is unacceptable and dangerous, and we must do everything we can to reduce or prevent them, not bring them about,” he wrote, referring to the past. events where he warned of layoffs. “Second, any response to AI-driven unemployment needs to address the need to support everyone economically, and the need for people to find meaning, purpose, and agency.”

This story also comes the same week that Anthropic launched Claude Fable 5, a public version of Claude Mythos 5, which offers some requests related to cybersecurity, biology, chemistry, and AI development to Claude who can’t afford it. Opus 4.8 as a safeguard against misuse. The opening drew near opposition from developers and researchers on the use of advanced Fable tokens, data retention requirements of 30 days, and security that can reduce the potential of the model without notifying users.

Amodei’s call for policy changes in AI development also comes as Anthropic prepares to go public. Earlier this month, the company filed filings with the US Securities and Exchange Commission for an initial public offering after raising a $65 billion Series H round at a price of $965 billion.

Although Amodei framed the issue as a contest between technological progress and public policy, critics question whether calling for stricter AI regulations will help society. In April, OpenAI CEO Sam Altman the accused Anthropic uses “fear-based marketing” to promote the Claude Mythos, arguing that concerns about advanced AI can be used to justify controlling the technology among a few companies.

“You can justify this in a number of ways, and some of them are real, if there are legal challenges,” Altman said. “But if what you want is like ‘we need to control AI, just us, because we’re trustworthy people’, I think fear-based marketing is the best way to prove that.”

Amodei rejected the idea that concerns about advanced AI are primarily a human relations problem, arguing that instead fear of the technology reflects legitimate concerns that need to be addressed.

“People are worried about AI because they see its dangers as real, not because AI CEOs have become insufficiently Panglossian,” he said, referring to the fictional philosopher Pangloss from Voltaire’s Candide, who is known for his unwavering optimism that everything will work out regardless of circumstances.

“I believe it is my responsibility as the leader of AI to continue to speak clearly about these threats, and the public’s involvement in responding to these disclosures will allow democracy to function as it should,” he said.