Anthropic, the company behind the artificial intelligence assistant Claude, has proposed new legislation that would give the United States government powers to restrict the deployment of advanced AI systems considered a serious threat to human safety or property.

The proposed bill, which the company is sharing with US lawmakers, would allow authorities to block the release or use of an AI model if it poses an imminent risk of killing or seriously injuring more than 50 people or causing over $1bn in damage. Such restrictions would require judicial approval.

The proposal comes as the United States continues to rely largely on voluntary commitments from technology companies and executive orders to address the risks associated with artificial intelligence.

Despite growing concerns about the technology, Congress has yet to enact comprehensive federal legislation governing AI development and deployment.

Anthropic said the proposed law would require developers of the most advanced AI models to conduct safety tests, disclose their findings publicly, undergo independent assessments and maintain strong security measures.

The company believes these requirements would help strengthen oversight of powerful AI systems and reduce the likelihood of dangerous applications.

Concerns about AI security have intensified following reports of incidents involving autonomous AI agents, which can perform tasks with limited human supervision and potentially act beyond the control of their developers.

Anthropic chief executive, Dario Amodei, also warned last month about the potential dangers of increasingly capable AI agents, suggesting that a group of such systems could gain control over large parts of the internet within six to 12 months.

The proposed legislation would also take precedence over certain state-level AI regulations, including laws introduced in California.

This represents a shift from Anthropic’s earlier position in June, when the company argued that states should retain their authority to regulate AI in the absence of strong federal legislation.

Charlie Bullock of the Institute for Law & AI described the proposal as reasonable and said it went further than measures introduced by some competing AI developers. However, he argued that it was weaker than the FRONTIER Act, a separate bill introduced in the US House of Representatives earlier this year.

The debate over AI regulation continues to expose differences between efforts to accelerate technological development and calls for stronger safeguards against potential harm.

US President Donald Trump has opposed calls to slow AI development, describing such concerns as negative forces, while his administration has generally favoured policies aimed at maintaining the country’s competitive position in the technology sector.