OpenAI, Anthropic, and Google Want to Regulate AI Together. What Could Go Wrong?

For most of the last three years, OpenAI, Anthropic, and Google DeepMind have competed like their hair was on fire. They poached researchers from each other, raced to announce bigger models, and argued publicly about whose benchmarks mattered. So it was genuinely surprising when OpenAI confirmed on September 15 that it has been meeting with its two biggest rivals for weeks to figure out how to keep their own technology from causing catastrophic harm.

The talks are not casual. They center on a proposal that Google DeepMind CEO Demis Hassabis published in July: a new industry standards body, modeled after FINRA, the organization that oversees Wall Street brokers. The idea is a public-private partnership, funded mostly by the companies themselves, that would independently test the most powerful AI systems before they are released to the public.

It is a radical shift in posture for an industry that has treated safety as a marketing bullet point at best and a public relations liability at worst.

From Proposal to Working Group

Hassabis laid out the blueprint in a July essay, arguing that the United States needs an organization with real technical muscle and federal oversight to evaluate frontier models for dangerous capabilities. He cited FINRA as the template because it is a self-regulatory body that operates under SEC supervision, not a slow-moving government agency trying to keep pace with private innovation.

According to reports from CNBC and The Information, representatives from all three labs have been meeting as a working group since at least July. OpenAI chief global affairs officer Chris Lehane confirmed the talks in Washington, where he was meeting lawmakers, and said the company does not believe an antitrust waiver is needed for competitors to coordinate on safety. He compared the cooperation to commercial aviation, where rivals share crash data without colluding on ticket prices.

That legal distinction matters. There are already two bills in Congress that touch on this exact question. The FRONTIER Act, introduced in the House in July, would require independent verification of the largest AI developers. The Collaboration on Adversarial Threats and Security Risks Act, introduced the same day, would create a narrow antitrust exemption for security coordination between AI companies. OpenAI has said it supports both frameworks.

Why the Sudden Urgency

The timing is not random. The last month has been brutal for AI safety optics. In late August, Anthropic researcher Jacob Coxon resigned and published a letter stating that AI companies are "gambling with our lives." A few days later, former Google DeepMind researcher Bilal Chughtai wrote publicly that he believes "AI has the potential to kill us all, and that we might be running out of time to avoid this outcome."

On September 12, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier," calling for the industry to voluntarily slow development of its most advanced models. Hassabis and OpenAI CEO Sam Altman both endorsed the sentiment. Altman said a slowdown has been a "primary topic" of discussion at OpenAI in recent weeks.

That is a striking reversal from the "move fast and break things" ethos that has defined the generative AI boom. For years, safety researchers at these labs complained internally that commercial pressure overrode caution. Now the CEOs are saying it out loud.

Not Everyone Is Buying It

Skepticism is coming from multiple directions. Canadian AI company Cohere, which competes with the big three but at a much smaller scale, accused the dominant US labs of forming a "cartel" under the guise of safety. Cohere CEO Aiden Gomez wrote that the dispute was never about whether AI needs guardrails, but about "who writes them, who gets to participate and whose interests the rules are protecting."

That criticism resonates because a self-regulatory body funded by the largest players could easily become a moat. Smaller labs and open-source projects would not have the resources to participate in expensive pre-release testing regimes, and the standards themselves could be written to favor the incumbents' architectures and business models.

Meanwhile, President Donald Trump and Nvidia CEO Jensen Huang have gone in the opposite direction, publicly dismissing AI safety concerns as a "hoax" and "not grounded on science." Huang said the dire predictions are "irresponsible" during an appearance at the All-In Summit where Trump called him on stage. That creates a bizarre political landscape where the tech CEOs begging for regulation are to the left of a Republican president who campaigned on deregulation.

What Would This Body Actually Do

The details are still vague, but the FINRA comparison gives some shape to the proposal. FINRA writes rules for broker-dealers, audits firms for compliance, and can fine or expel members. Applied to AI, a similar body would likely define safety benchmarks, conduct pre-deployment evaluations of models above a certain capability threshold, and publish findings without necessarily blocking releases.

Hassabis argued that funding would need to be "substantial and likely mostly come from industry" in order to attract world-class technical talent and provide the compute necessary for large-scale testing. That raises an obvious tension: a regulator funded by the regulated is only as independent as its members allow it to be.

There is also the question of whether this would complement or replace federal legislation. Lehane said any industry-led standards would "complement, not replace, mandatory federal safeguards and democratic oversight." But in practice, a functioning self-regulatory body could become an excuse for Congress to kick the can down the road, especially given that lawmakers in both parties have expressed pessimism about passing comprehensive AI legislation before the midterms.

The Bottom Line

The most interesting thing about this story is not the policy proposal itself, but what it reveals about where the industry thinks it is. Three years ago, these companies were racing to release chatbots to hundreds of millions of users with minimal external review. Now their own CEOs are publicly saying that pace was dangerous, and their chief policy officers are walking Capitol Hill hallways pitching voluntary slowdown frameworks.

Whether the FINRA-style body materializes or becomes another talking point that fades after the headlines move on, the fact that OpenAI, Anthropic, and Google are sitting in the same room talking about it at all is a sign that the frontier AI labs no longer believe they can regulate themselves alone. The question is whether they can regulate themselves together without locking everyone else out.