The Trump administration signed agreements this week with Google DeepMind, Microsoft and xAI to let the Center for AI Standards and Innovation run government safety checks on frontier models before and after release. The deals revive a Biden-era framework that Trump had publicly dismissed as overregulation, and they arrive after Congress approved just $10 million in January to expand the agency. CAISI has completed roughly 40 evaluations to date, including tests on unreleased models.
The reversal traces directly to Anthropic, which announced it was too risky to release its latest Claude Mythos model over fears that bad actors could exploit its cybersecurity capabilities. White House National Economic Council Director Kevin Hassett told Fortune that Trump may now issue an executive order mandating government testing of advanced AI systems prior to release. That would convert this week's voluntary handshake into a binding regime across the frontier labs.
CAISI was the US AI Safety Institute under Joe Biden until Trump rebranded it earlier this year, stripping the word 'safety' from the name. The agency's own press release acknowledges the new partnerships 'build on' the Biden policy the administration had spent months disowning. CAISI Director Chris Fall did not name Mythos in announcing the deals.
Key facts
- 01CAISI signed voluntary pre- and post-deployment safety testing agreements with Google DeepMind, Microsoft and xAI this week.
- 02The agency has completed roughly 40 model evaluations to date, including unreleased frontier systems with safeguards stripped.
- 03Congress approved $10 million in January 2026 to expand CAISI, a sum the America First Policy Institute calls insufficient.
- 04Trump may issue an executive order mandating government testing of frontier models, per NEC Director Kevin Hassett.
- 05The reversal followed Anthropic's decision to delay Claude Mythos over cybersecurity misuse risk.
"Independent, rigorous measurement science is essential to understanding frontier AI and its national security implications," Fall said. CAISI said it frequently gains access to models with 'reduced or removed safeguards' to evaluate national-security-related capabilities, and that an interagency task force has been set up to track emerging concerns. The agency has not published the standards it will test against.
“Evaluations are a policy tool, they are not actually data-driven. My concern is that this is another political tool that the administration wants to own and wield.”— Jaeden Schafer
Tom Lue, Google DeepMind's vice president of frontier AI global affairs, said on LinkedIn he was 'pleased' with CAISI's testing plans. Microsoft credited expertise 'uniquely held by institutions like CAISI' and said it will work with the agency and NIST on adversarial assessment methodology. xAI, currently in trial against OpenAI over which firm's leadership cares more about AI safety, did not comment.
The unanswered question is what 'evaluation' actually means. Devin Lynch, formerly of the White House Office of the National Cyber Director, wrote that capability assessments 'are only as good as the threat models behind them' and that CAISI 'will need to define, and publish, what it's testing for, not just who it's testing with.' Microsoft's own blog framed the methodology as something to be developed on the fly, comparing it to stress-testing airbags and brakes.
Sarah Kreps, director of the Tech Policy Institute at Cornell University, warned that 'the definition of safe is contested' and that 'once you build a government vetting process for technology, you get the good with the bad.' Without published standards, she said, 'the process can be politicized,' producing a system where 'whoever holds power gets to shape how the vetting works.' Neither administration has solved that, she said.
Gregory Falco, a Cornell assistant professor who tracks AI governance, argued the federal government lacks the in-house technical expertise, infrastructure or day-to-day insight to directly evaluate frontier systems. He proposed an IRS-style independent audit model with real consequences for reckless deployment, rather than a politicized review of model outputs. "Government oversight of AI cannot simply mean political review of model outputs, nor should it become a mechanism for deciding whether a model says favorable or unfavorable things about a president or administration," Falco said.
Funding is the other constraint. The $10 million Congress allocated in January is, by the America First Policy Institute's own analysis, below what peer institutes abroad receive — a notable critique given the think tank's alignment with the administration. Rumman Chowdhury, founder of Humane Intelligence, told Fortune that 'evaluations are a policy tool, they are not actually data-driven' and warned the program risks becoming 'another political tool that the administration wants to own and wield.'
This is the second time in as many weeks the White House has been pulled back toward the AI safety architecture it inherited and rejected, following the Mythos-driven scramble we covered last week. The pattern suggests the frontier labs themselves — by flagging genuinely dangerous capabilities — are now setting the pace of US AI policy. Whether CAISI can build credible technical authority on a $10 million budget against models that cost billions to train is the question the next executive order will have to answer.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
