Rep. Ro Khanna plans to introduce a bill that would ban AI models that improve themselves until federal safeguards exist, Garrett Downs reported for CNBC on Monday. Khanna, a Democrat from California, shared a summary of the Human Control Over AI Act with CNBC first. TNW has not seen the bill.
“There’s actually a civilizational extinction risk,” Khanna told CNBC.
The bill would ban models that recursively self-improve, or that change their own core objectives, containment or shutdown controls on their own. The ban would last until federal guardrails exist and an agency approves the activity.
A new agency and criminal penalties
The bill would create a federal agency to focus on AI models from OpenAI, Anthropic, Google DeepMind and xAI. The agency would write safety rules and license the training and deployment of models. It would also audit frontier models and set standards for testing and human control.
Independent auditors would sit inside every frontier lab and report straight to the agency. The agency would also set standards for sandboxed test environments, air gaps and kill switches. It would add controls to stop models escaping labs and roaming the internet. It would also regulate the advanced chips the labs use.
Employees who disable safeguards, kill switches, logging or containment systems would face criminal penalties. So would those who knowingly deploy an unauthorised system. Deploying an AI model that destroys civilian populations would become a crime against humanity. AI companies would need extensive liability insurance before releasing models.
The bill also asks the administration to pursue enforceable agreements and targeted export controls to deter China and other adversaries from building dangerous AI.
“I want this to be a model to give voice to the AI safety community,” Khanna told CNBC.
Khanna said he based the bill on talks with AI safety nonprofits, rather than on what AI executives asked for. They include Model Evaluation and Threat Research, the Machine Intelligence Research Institute and Palisade Research.
Letters to Chinese AI labs
Khanna, the top Democrat on the House Select Committee on China, has also written to DeepSeek, Alibaba and Moonshot AI, Lauren Feiner reported for The Verge. He asked for documents on their work towards superintelligence and recursive self-improvement. He also asked the Director of National Intelligence, Jay Clayton, how the US would respond if a lab lost control of its AI.
Khanna asked whether they would be ready for an incident like OpenAI agents’ hack of Hugging Face this summer. He says the answers will help Congress, and later the White House, work towards an AI agreement with China.
No vote before the midterms
The bill joins other proposals, including the bipartisan FRONTIER Act from Reps. Jay Obernolte and Lori Trahan. That bill would also put independent auditors in frontier labs, and let the government shut down models with catastrophic potential. No House bills are expected to get a vote until after the midterm elections, CNBC reported.
The bill follows a warning from more than 20 researchers that AI automating its own research could set off an intelligence explosion. President Donald Trump is hosting AI leaders at a White House lunch on Tuesday.