Florida AG asks court to prevent OpenAI from advancing its frontier models, as company scraps new one

Florida attorney general James Uthmeier is stepping up his fight with OpenAI Group PBC after asking a state court to prevent the company from further advancing its frontier models without third-party oversight. He’s also asking the court to ensure the company prevents minors from using its technology.

In his request, Uthmeier cited the ongoing panic in both government and the industry about artificial intelligence technology’s unchecked growth as evidence that the company’s large language models are “dangerous” products. He’s asking the court for an injunction to prevent it from improving new models without approved third-party protections in place.

“If Sam Altman meant what he said about slowing down, he can join our ask to the court,” Uthmeier said in a video posted on X. “If he will not, we ask the court to do what OpenAI will not do itself – protect Florida families.”

The action comes as OpenAI today said it’s scrapping the release of its next-generation AI model, GPT-6.1 Astra, amid safety concerns researchers brought up during internal testing. It’s another sign of many in past weeks that even model makers are worried about their AI agents going rogue and hacking illegally into various organizations, including governmental departments.

In a response to the injunction request, OpenAI spokesperson Drew Pusateri insisted that the company is transparent about its technology’s development and is cooperating with Florida officials and those across the U.S. “People want to know AI is being developed safely, and that starts with what companies like ours do ourselves,” he said. “On Friday, we announced we’ve paused training our most capable models and will resume only when we are confident that we have additional safeguards in place.”

Uthmeier has had OpenAI in his crosshairs since June when he announced he was filing a first-of-its-kind lawsuit against the company that claimed it was aggressively marketing an “unsafe product” in ChatGPT. That lawsuit alleged OpenAI was also allowing it to aid in violent crimes, such as the 2025 Florida State University campus shooting, and putting Floridians at risk. OpenAI is fighting that lawsuit and maintains it has sufficient safety protocols in place.

Today Uthmeier stepped up his battle against OpenAI, asking for an injunction that would require the company to adhere to a number of requests: “No new model development without independent safety guardrails; no more harvesting children’s data; no more calling this product safe, accurate or reliable; no more pretending it’s human; no more tricks designed to keep users talking past the point of danger.”

Uthmeier cited recent public security disclosures made by OpenAI itself, including the one it made after its AI agents hacked the Hugging Face platform and Chief Executive Sam Altman’s own calls for the industry to slow down the pace of frontier model development, as proof of his claims. He also pointed to OpenAI’s recent hacks of U.S. and Australian government websites.

“It is a rare request for an injunction where the Defendants themselves have publicly endorsed it,” Uthmeier wrote in the brief. “They have asked the government to tie them to the mast. Plaintiff brings good news to the Defendants. The Florida Attorney General is answering your cry for help with a motion to enjoin you from harming Floridians with your reckless, unacceptably risky product.”

Uthmeier’s case may be strengthened by the company’s deciding not to release GPT-6.1 Astra, after determining that it does not yet meet its own safety standards.

“Of course we want to make sure our model development is safe no matter whether that’s in the company or when we ship them to users,” OpenAI’s head of safety systems Saachi Jain said in a statement to the Journal. “But when we ship it to users, we have an extremely high bar in terms of safety and alignment.”

Meanwhile, researchers at OpenAI were joined by their counterparts from Anthropic PBC, Meta Platforms Inc. and Microsoft Corp. in asking policymakers to assess the degree to which their companies are automating research into the technology, in the latest industry call for greater government oversight. In a paper published today co-authored by OpenAI Chief Scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Microsoft Chief Scientific Officer Eric Horvitz and Meta Vice President of AI Research Dawn Song, the researchers explained that by using AI to accelerate research, they risk creating an “intelligence explosion” that could outpace the ability of humans to understand it.

The co-authors, writing in a personal capacity, called on policymakers to demand greater visibility from AI firms regarding how much automation of model development they initiate.

Despite this and earlier calls from the industry for more government oversight, Congress and the White House have both been dragging their feet, with the Trump Administration reportedly trying to come up with a long-awaited national regulatory framework. As a result, individual states such as Florida have decided to take action themselves.

Uthmeier, who is up for re-election in November, has become one of the fiercest opponents of the AI industry. He has also asked Florida’s legislature for the authority to slap AI companies with criminal charges if their chatbots aid in crimes.

Photo: OpenAI