As OpenAI rolls out its latest advanced LLM, Sol, for wide public access, many are left wondering how this model was deemed safe to release. Sol is considered to be at least on par with Anthropic’s Fable, a model whose capabilities (or ownership) stressed out the White House enough that it was briefly banned from public access.
But how did these models get the OK for release? Short answer: Nobody’s quite sure.
Frankly, I don’t have visibility into those exact processes, so yes, I don’t feel like I have enough information to say whether they’re adequate or not, Mina Narayanan, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, told TechCrunch.
Anthropic did say that they were in conversations with the government, and that they developed a classifier to detect jailbreak attempts, and they’ve implemented defense-in-depth strategies to prevent future jailbreaks, but exactly what that dialog looked like between the government and Anthropic and OpenAI is unclear.
The Lack of Clarity in Frontier Model Regulation
Eighteen months into the Trump administration, there is still little clarity about how to move forward, despite — or, some critics allege, because — of the industry figures setting policy. Last month, after weeks of infighting, an executive order was published laying out a roadmap for evaluating frontier models, but the specifics have yet to be filled in, other than what won’t exist.
There will not be an FDA for AI, Sriram Krishnan, a former Andreessen Horowitz partner who served as a senior advisor for AI in the White House until last month, told Financial Times.
Notably, there’s still no agreement on what kinds of models require government scrutiny, or what agency or agencies should perform those evaluations. For now, the Department of Commerce’s Center for AI Standards and Innovation seems to be taking the lead, but the executive order instructs six cabinet agencies to determine a final process by early August.
What has emerged in the meantime is, at best, ad hoc.
The Involvement of Industry Figures in Policy-Making
As the industry figures set policy, the process involved conversations with officials like Secretary of Commerce Howard Lutnick, Secretary of the Treasury Scott Bessent, and U.S. national cyber director Sean Cairncross, but it’s not clear who the experts that tested the models were or how they did that.
OpenAI CEO Sam Altman said on CNBC that the process involved conversations with the officials, but it’s not clear who the experts that tested the models were or how they did that.
OpenAI declined to share details on the government’s process with TechCrunch, but pointed to the results of several external evaluations by organizations like U.K. AISI, SecureBio, and Irregular in the latest model’s safety card.
As with Anthropic’s Fable roll-out, OpenAI previewed the model for the government and select users ahead of wider release, but we don’t know who all of those users were or how they were chosen.
The Role of Personal Connections in the Regulation Process
Anthropic’s Fable, on the other hand, was briefly pulled from wider access when the U.S. government forbade its use by foreign nationals, partly because of real concerns about users jail-breaking the model to access hacking capabilities and partly due to personality clashes between Anthropic and the Trump administration.
The threat of an export ban may have also led OpenAI to be more cooperative with the government’s (unknown) requests.
From an industry perspective, a hands-off approach to regulation might be nice, but one that depends on personal connections to administration officials creates uncertainty and bad incentives.
Konwinski told TechCrunch that he worries true experts in this technology — safety researchers, alignment researchers, interpretability researchers, but also data people, and people from all over the stack — aren’t playing enough of a role in the model release process.
Konwinski argues that an open commons is the best way to actually balance safety and innovation. He points to models like the FDA, the NIH, or the national labs, which convene researchers, government officials, and private companies to reach a consensus on safety issues.
Some of that comes down to the incentives of capitalism that have motivated AI researchers for more than a decade, and played out in the court room during Elon Musk’s lawsuit challenging OpenAI’s corporate structure.
Ball points out that the nature of the AI business requires companies to recoup much of their training costs shortly after their models are released and are further ahead of the competition.
Even if their intentions are good, there’s very clear legal obligations and fiduciary responsibility that are built right into the operating procedures, Konwinski points out.
Ball, in his post, argued that the way forward will depend on third-party auditing organizations, licensed by the government, that will evaluate frontier labs’ approach to safety.
Konwinski, too, is bullish about new institutional formats like focused research organizations that could help more disinterested experts from academia and the nonprofit world access and evaluate frontier models.
For now, the secrecy around the development of AI isn’t going away, but it also will seed political challenges for an industry that Americans increasingly view with skepticism.
There’s not a sense that responsible people are driving forward these changes, University of Wisconsin-Madison computer science professor Remzi Arpaci-Dusseau said last week at the Open Frontier conference.
At the same event, David Siegel, the computer scientist who founded Two Sigma, one of the most successful quantitative hedge funds, asked attendees to imagine a situation, which I think would be very bad, [where] a small number of firms control the technology; the government, in their secretive laboratories, is evaluating whether or not the technology is suitable for use; and the general public and scientific community doesn’t really have any access to any of that stuff.
