Here we go: anthropic wants to enshrine in US legislation a mechanism to prevent the deployment of artificial intelligence models considered exceptionally dangerous. The creator of Claude circulates in Congress a draft text that would define precisely under what circumstances an AI could be discontinued before it is available.
A threshold set at 50 victims or $1 billion in damages
The system would only target catastrophic risks considered foreseeable and imminent. The intervention could be triggered when a model threatens to kill or seriously injure more than 50 people, or cause more than $1 billion in property damage.
The danger would still have to be linked to certain specific situations: aid in the creation or deployment of a biological weapon, cyber attack or loss of control of a model that escapes the mechanisms provided by its developer. These concerns took on a very concrete dimension after several incidents, including unexpected behaviors observed in Claude and several experiments that demonstrated that AIs could carry out unauthorized actions.
Independent evaluators before the intervention of a judge
Anthropic also suggests that independent organizations registered with the U.S. Department of Commerce periodically evaluate more advanced models. The latter could be remunerated by the laboratories examined, but their payment could not depend on the result of the evaluation.
In the event of an imminent risk, the appraiser would have 72 hours to notify the Department of Commerce. However, he himself would not be able to stop the model: he would have to convince a federal judge in Washington to order its suspension.
This approach follows voluntary commitments recently made by major AI groups around independent audits. It also uses thresholds quite close to those of the Californian law SB 53. However, the project currently remains a simple Anthropic proposal and Congress has not yet adopted any corresponding text. However, this is a notable novelty: a major AI laboratory is now explicitly asking that the State be able, in certain extreme circumstances, to prevent the dissemination of an artificial intelligence model.
