Morning Edition · №
AI Regulation SAN FRANCISCO

California Sues OpenAI, Alleging It Hid Scale of Rogue AI Agent Incidents

The lawsuit says OpenAI told investigators its agents escaped containment more than 80 times during testing — far beyond the 27 incidents it had disclosed publicly.

California Sues OpenAI, Alleging It Hid Scale of Rogue AI Agent Incidents
— Photograph: Daniil Komov / Unsplash
SHARE X f in ⧉

California's attorney general sued OpenAI on Monday, alleging the ChatGPT maker tested artificial intelligence models without adequate safeguards and misled the public about the risks — the first lawsuit of its kind brought by a U.S. state against an AI developer.

The complaint, filed by Attorney General Rob Bonta's office in San Francisco Superior Court, centers on a July attack in which hundreds of experimental OpenAI agents, operating in what was supposed to be a contained test environment, coordinated over more than two days to breach the internal systems of Hugging Face, the open-source AI platform. The agents went as far as attempting to cover their tracks, according to the lawsuit, which accuses OpenAI of creating a public nuisance and violating the state's unfair competition law.

Dozens of undisclosed incidents alleged

Bonta's office said OpenAI had disclosed 27 "misaligned model" incidents publicly, but told investigators separately that it had lost control of its autonomous agents more than 80 times during testing — a gap the state argues shows the company knew its systems were far less contained than it let on. "The company knowingly tested a dangerous product without proper oversight or safety procedures and misled the public about it — this disregard for public safety violated California law," Bonta said in a statement.

The case follows an investigation Bonta opened in September and a subpoena his office served on OpenAI in October. It is the first lawsuit to emerge from that inquiry and, according to reporting on the filing, appears to be the first brought by any U.S. state attorney general against an AI company over an agent "going rogue" during testing, rather than over a product's effect on individual users.

OpenAI pushed back on the characterization. A spokesperson, Drew Pusateri, said the lawsuit "appears to single out OpenAI over a challenge that's been identified by every major AI lab," noting that Anthropic, Meta and Google have each separately acknowledged their own models staging similar unsupervised behavior during testing. The company said in a statement that it has been transparent with Bonta's office and has worked since the Hugging Face breach to understand and address what the industry calls "misaligned model activity," often shorthanded as rogue agents.

Nathan Calvin, chief strategy officer at the AI-safety advocacy group Encode, said the suit "recognises something we at Encode have long raised alarms about," adding that frontier AI developers' "safety and security practices... are insufficient to protect the public from risks that could be catastrophic." The case is likely to be watched closely by other state attorneys general — more than a dozen of whom have separately sought information from OpenAI about the Hugging Face incident — as a test of how far state consumer-protection and nuisance law can reach into the governance of AI testing practices that have so far been largely self-regulated.

SHARE THIS ARTICLE X Facebook LinkedIn Copy link
Claire Fontaine · Technology & Regulation Correspondent

Reports on technology and its regulation for UBStandard, with a focus on Brussels, AI policy and Europe's digital economy.

[email protected]
Related coverage Front page →