Anthropic CEO Dario Amodei clarifies stance: open-weight AI models are safe, Chinese AI is the real threat

Anthropic founder and CEO Dario Amodei on Monday directly addressed growing industry speculation that his company supports a U.S. government ban on open-weight AI models, particularly those from China. In a blog post, Amodei stated unequivocally that Anthropic has never advocated for banning open-weight models, clarifying that his primary concern is the potential for authoritarian governments, especially China, to develop more powerful AI systems for military or repressive purposes.

Amodei’s position on open-weight models

Amodei wrote that open-weight models without dangerous capabilities are a public good, providing value to businesses, developers, and researchers at minimal cost. He emphasized that the threat from Chinese AI labs is fundamentally different from the risks posed by open-weight models themselves. His statement came after Nvidia CEO Jensen Huang posted an open letter on X — his first post on the platform — urging policymakers not to impose broad premature restrictions on open-weight AI. The letter was signed by companies including Hugging Face, Meta, Microsoft, Mistral, and Nvidia.

Distillation and IP theft concerns

Amodei acknowledged the industry debate over Chinese AI labs allegedly using distillation — a technique where one model bombards another with prompts to learn its workings — to steal intellectual property from American counterparts. He called for a formal crackdown on distillation and reiterated support for restricting China’s access to powerful chips, a policy the U.S. has already pursued. The U.S. has threatened sanctions if IP theft involving U.S. models is confirmed.

Broader fears: biological attacks and global testing

Amodei expressed concern that AI could enable biological attacks, not just cybersecurity threats, and argued that open-weight models are more dangerous in such scenarios because guardrails are harder to apply and monitor. He cited a UK AI Security Institute report noting that once open-weight models are released, they cannot be withdrawn. This contradicts open-source advocates who argue that accessible models help defenders protect themselves.

Call for global safety testing

Interestingly, Amodei supported growing efforts, including those led by the Trump administration, to create a model safety testing organization. He stressed that to be effective, testing must be global, requiring China’s participation. He noted that limited cooperation around preventing AI biological weapons may be possible because it is in China’s interest as well.

Conclusion

Amodei’s clarification marks a significant moment in the ongoing debate over open-weight AI regulation. By distinguishing between the public good of open models and the distinct threat posed by authoritarian governments, he offers a nuanced position that may influence policymakers and industry leaders alike. The conversation now shifts to how global safety testing can be implemented effectively, and whether China will participate.

FAQs

Q1: Does Anthropic support banning open-weight AI models?
No. CEO Dario Amodei stated that Anthropic has never advocated for a ban on open-weight models, describing them as a public good when they lack dangerous capabilities.

Q2: What is Amodei’s main concern regarding Chinese AI?
He fears that authoritarian governments, particularly China, could build more powerful AI systems for permanent military superiority or to repress their own people.

Q3: What is model distillation and why is it controversial?
Distillation is a technique where an AI bombards another model with prompts to learn its workings. U.S. officials and companies allege Chinese labs use it to steal intellectual property from American AI models.