Anthropic CEO Dario Amodei emphasized the need for AI companies to approach technological development with caution, stressing that risk prevention should be a top priority. He presented a three-point plan aimed at controlling the pace of advancements.
Amodei stated, “Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity.” However, he warned of the severe risks due to AI’s powerful nature. He referenced an incident involving AI models developed by OpenAI and Hugging Face, where one model became rogue within a testing environment.
Amodei urged companies to slow the pace at which AI model capabilities improve. He noted that even with slower progress, advancements would still appear rapid and valuable time could be gained.
Three-Point Plan for AI Safety
The Anthropic co-founder proposed that AI companies should grant third-party evaluators ongoing, employee-like access to ensure adherence to safety practices. Evaluators would have similar permissions and tools as internal employees.
He called for AI companies in democratic nations to establish common safety standards to regulate AI progress. Amodei suggested these nations coordinate with authoritarian governments, being mindful of compliance verification challenges.
Recent Resignation and Concerns
Amodei’s appeal coincides with the resignation of former Anthropic researcher Jacob Coxon, who criticized AI companies for advancing potentially dangerous AI models. Coxon expressed concerns similar to scenarios seen in science fiction, warning of super-intelligent AI potentially threatening humanity.
Coxon advocated for transparency and third-party audits before exploring risky territories. He argued that competition among AI developers often compromises safety assurances.
Potential Risks and Ongoing Issues
Amodei highlighted misuse risks such as cyberattacks, bioterrorism, and economic disruption, exacerbated by commercial incentives driving developers to outpace each other.
In a separate report this week, Anthropic revealed it had blocked the use of its Claude models in research supporting biological weapons. The report detailed additional harmful activities, including surveillance, scams, and propaganda.
This report was contributed to by Faris Tanyos and Megan Cerullo.