Khanna introduces AI safety bill banning recursive self-improvement until safeguards

September 28, 2026 at 05:00 AM2 min

This digest was compiled by AI from multiple sources — links to the originals are below.

Rep. Ro Khanna will introduce the Human Control Over AI Act, banning recursive self-improving AI until federal safeguards exist. The bill creates a new federal agency to regulate frontier AI labs and imposes criminal penalties for disabling safety controls. It comes as Congress faces pressure to regulate AI after executives warned of extinction-level risks.

Key Facts

The bill bans AI models that recursively self-improve or autonomously modify their core objectives, containment, or shutdown controls until federal guardrails exist and an agency approves the activities.
A new federal agency would regulate frontier AI models from OpenAI, Anthropic, Google DeepMind, and xAI, with authority over safety regulations, licensing, audits, and security standards.
The bill criminalizes deployment of AI models that result in destruction of civilian populations as 'crimes against humanity' and imposes penalties on employees who disable safeguards or deploy unauthorized systems.
AI companies would be required to hold extensive liability insurance to release their models.
The bill calls for enforceable agreements and targeted export controls to deter China and other adversaries from developing dangerous AI systems.

Legislative Proposal

Rep. Ro Khanna, D-Calif., will introduce the Human Control Over AI Act, a summary of which was shared exclusively with CNBC. The bill bans models that recursively self-improve or autonomously modify their own core objectives, containment, or shutdown controls until federal guardrails exist and an agency approves the activities. Khanna said the bill represents 'the most comprehensive AI safety legislation' that has been proposed.

Regulatory Agency

The bill creates a new federal agency to focus on AI models from OpenAI, Anthropic, Google DeepMind, and xAI. The agency would be tasked with creating safety regulations, a licensing system to approve model training and deployment, frontier AI model audits, and setting security standards for continuous testing and effective human control. Independent auditors would be required to be embedded at every frontier lab and report to the agency directly. The agency would also create standards covering sandbox testing environments, air gaps, kill switches, and controls that prevent models from escaping lab settings and roaming on the Internet. It would also be responsible for regulating the advanced chips that frontier labs use, including mechanisms to monitor or control their use.

Penalties and Liability

The bill would criminalize 'crimes against humanity' for the deployment of AI models that result in the destruction of civilian populations. It would impose criminal penalties on employees of AI companies who disable safeguards, kill switches, logging, or containment systems, or who knowingly deploy an unauthorized system. AI companies would also be required to hold extensive liability insurance to release their models. The bill calls for the administration to pursue enforceable agreements and targeted export controls to deter China and other adversaries from developing dangerous AI systems.