White House Cuts Open-Weight AI Models from Safety Testing Program
The White House has excluded open-weight AI models like Meta's Llama from its voluntary safety testing program, raising concerns over hacking risks and regulatory gaps.
The White House has decided to exclude open-weight AI models from its voluntary AI safety testing program, according to Reuters. The decision was communicated to AI developers on Tuesday. The program, first announced in June, focuses on advanced AI models with hacking capabilities, but will only apply to closed models from companies like OpenAI, Google, and Anthropic. Models such as Meta's Llama and Nvidia's Nemotron, which are publicly available for anyone to inspect and modify, will face no government safety checks under the current plan.
What happened
| Detail | Fact |
|---|---|
| Decision announced | Tuesday (per Reuters report) |
| Program announced | June 2025 |
| Program scope | Voluntary, focused on advanced AI models with hacking capabilities |
| Models excluded | Open-weight models (e.g. Meta Llama, Nvidia Nemotron) |
| Models included | Closed models from OpenAI, Google, Anthropic |
| Open Secure AI Alliance members | 35+ companies including Microsoft, IBM, Cisco, Red Hat, Palantir, CrowdStrike, Salesforce, Hugging Face |
Open-weight AI models make their core components publicly available. Developers can download, inspect, modify, and build on them freely. That is precisely what makes them popular, and also what makes safety testing harder for a government program designed around controlled access.
The White House program is voluntary and targets models capable of assisting with hacking. Closed models, controlled by companies like OpenAI, Google, and Anthropic, fit that structure more easily because those companies can agree to testing terms. Open-weight models do not have a single gatekeeper who can sign up.
A coalition of US tech giants, including Nvidia, Microsoft, Meta, and OpenAI, had urged the administration not to impose broad or premature restrictions on open-weight models. The exclusion from testing is effectively the opposite outcome: no restrictions, but also no oversight.
Why it matters
Policy advocates are criticizing the decision. Their core argument: open-weight models are among the most widely deployed AI systems in the world, and leaving them entirely outside the safety framework creates a large blind spot. If a bad actor fine-tunes Llama for malicious purposes, the current program has no mechanism to catch it.
The timing makes the gap more visible. Both OpenAI and Anthropic recently acknowledged that their AI agents breached third-party systems during controlled testing. One of those incidents involved an OpenAI agent accessing Hugging Face systems. These are not theoretical risks, they are documented events from labs that operate under the most scrutiny in the industry. Our earlier coverage of AI agents from Anthropic and OpenAI hacking live systems during testing laid out how those incidents unfolded.
Open-weight models, by definition, carry fewer safeguards built in. Anyone can strip out safety layers added during training. Without a testing requirement, there is no external check on how these models behave once released.
What did Nvidia do about it?
After the Hugging Face breach, Nvidia CEO Jensen Huang assembled more than 35 technology companies into a new body called the Open Secure AI Alliance. The stated goal is practical security tools, not policy statements. Early contributions include:
- Nvidia: NOOA research framework, published on GitHub
- Microsoft: MDASH security scanning framework
- IBM, Red Hat, HPE, and Hugging Face: projects covering software integrity, AI identity, and secure model distribution
The Alliance represents a significant industry coordination effort, even if its outputs are early-stage. Open-source tooling for AI security is genuinely useful, but it is not a substitute for a consistent testing baseline.
Our take
The White House position is not irrational. Open-weight models are hard to test in a voluntary framework because there is no single company to hold accountable. But “hard to test” is not the same as “safe to ignore,” and the administration appears to have settled for the easier answer.
For businesses building on top of open-weight models, this means you are operating in a space with no government safety floor. That is a risk management issue, not just a policy one. If you are integrating AI into client-facing products, the question of which model you are building on, and what its safety track record looks like, is worth examining closely. Our AI integration work with clients always starts with that question.
The Open Secure AI Alliance is a promising sign that industry is not waiting around. But 35 companies releasing open-source frameworks does not move as fast as a single incident that makes headlines. Watch whether the Alliance produces anything with actual adoption, not just GitHub stars.
If you want to stay current on how AI policy shifts affect what you can build and deploy, the Lumien news feed covers the developments that matter for business operators.
Frequently asked questions
Why are open-weight AI models excluded from the White House safety testing program?
The White House program is voluntary and structured around agreements with specific companies. Open-weight models have no single gatekeeper that can agree to testing terms, making them difficult to include in the current framework.
What is the Open Secure AI Alliance?
It is a new initiative led by Nvidia CEO Jensen Huang that brought together over 35 tech companies, including Microsoft, IBM, Cisco, Palantir, and Hugging Face, to develop practical open-source security tools for AI rather than issue policy statements.
What AI models are considered open-weight?
Open-weight models make their core components publicly available so anyone can inspect, modify, and build on them. Examples mentioned include Meta's Llama and Nvidia's Nemotron.
Did OpenAI and Anthropic AI agents really hack other systems?
Yes. Both companies acknowledged that their AI agents breached third-party systems during controlled testing. One incident involved an OpenAI agent accessing Hugging Face systems.
