Campbell Brown's Forum AI is building expert-architected benchmarks to grade how frontier models handle what she calls high-stakes topics, and the New York startup raised $3 million last fall led by Lerer Hippeau to scale the work. Founded 17 months ago, Forum AI recruits domain experts to design evaluations on geopolitics, mental health, finance and hiring, then trains AI judges to apply those evaluations at scale. The target benchmark is 90% consensus between the AI judges and the human experts — a threshold Brown says her company has hit.
The expert roster on the geopolitics work is unusually heavy. Brown has signed Niall Ferguson, Fareed Zakaria, former Secretary of State Tony Blinken, former House Speaker Kevin McCarthy, and Anne Neuberger, who led cybersecurity in the Obama administration. The pitch is that the foundation labs have optimized for coding and math, where answers are verifiable, and underinvested in the messier domains where, as Brown put it, there are no clear yes-or-no answers.
Brown's vantage point matters here. She was Facebook's first and only dedicated news chief, running the fact-checking program that no longer exists, and she joined Meta in time to watch ChatGPT launch publicly. "I was at Meta when ChatGPT was first released publicly, and I remember really shortly after realizing this is going to be the funnel through which all information flows. And it's not very good," she told TechCrunch's Tim Fernholz at a StrictlyVC evening in San Francisco.
Key facts
- 01Forum AI raised $3 million last fall led by Lerer Hippeau, 17 months after Campbell Brown founded the New York company.
- 02The startup targets 90% consensus between AI judges and human experts on high-stakes topics like geopolitics and hiring.
- 03Expert panel includes Niall Ferguson, Fareed Zakaria, former Secretary of State Tony Blinken, former House Speaker Kevin McCarthy and Anne Neuberger.
- 04New York City's hiring-bias audit law caught more than half of violations going undetected, per the state comptroller.
- 05Brown cited [Gemini](/gemini) pulling from Chinese Communist Party websites for stories unrelated to China.
Forum AI's early evaluations of leading models surfaced specific failures rather than vague concerns. Brown cited Gemini pulling from Chinese Communist Party websites for stories that have nothing to do with China, and a left-leaning political bias across nearly all the models tested. Subtler problems showed up too: missing context, missing perspectives, and arguments straw-manned without the model acknowledging it had done so.
“Forum AI is targeting 90% consensus between its AI judges and human experts including Niall Ferguson, Fareed Zakaria, Tony Blinken, Kevin McCarthy and Anne Neuberger.”— Jaeden Schafer
"There's a long way to go," Brown said. "But I also think that there are some very easy fixes that would vastly improve the outcomes." The frustration, in her telling, is that accuracy hasn't been a priority at the labs because news and information are harder problems than the ones that show up on standard benchmarks.
Her Meta years inform the diagnosis. "We failed at a lot of the things we tried," Brown said of the Facebook fact-checking effort. Optimizing for engagement, in retrospect, was bad for users and bad for the information environment, and she sees AI labs at a similar fork. "Right now it could go either way" — companies can give users what they want, or they can give people what's real.
The business bet is that enterprise compliance demand will fund the work. Businesses deploying AI for credit decisions, lending, insurance and hiring care about liability, and Brown thinks they will want vendors who can prove they got it right. The catch is that the current compliance market is largely satisfied with checkbox audits and standardized benchmarks she considers inadequate.
Brown calls the compliance landscape "a joke." When New York City passed the first hiring-bias law requiring AI audits, the state comptroller found that more than half of violations went undetected. Real evaluation, she argued, requires domain experts who can stress-test edge cases the developers never imagined. "Smart generalists aren't going to cut it."
Forum AI is not the only company in this space — public-sector audit shops and academic benchmark teams cover overlapping ground — and turning compliance interest into recurring enterprise revenue is the open question for any evaluation startup. The $3 million seed gives Brown runway but not yet proof that buyers will pay for deep expert-built benchmarks rather than the cheaper checkbox products that already satisfy regulators.
Brown frames the broader gap as a disconnect between Silicon Valley's self-image and what users actually experience. "You hear from the leaders of the big tech companies, 'This technology is going to change the world,' 'it's going to put you out of work,' 'it's going to cure cancer'. But then to a normal person who's just using a chatbot to ask basic questions, they're still getting a lot of slop and wrong answers." Trust in AI, she noted, sits at low levels, and she thinks much of that skepticism is earned.
"The conversation is sort of happening in Silicon Valley around one thing, and a totally different conversation is happening among consumers," Brown said. The wager behind Forum AI is that closing that gap is itself a market.
The interesting bet inside Forum AI is structural, not editorial. If enterprise liability becomes the binding constraint on model deployment in regulated domains — and the New York City audit results suggest regulators are starting to notice when the existing audit market underperforms — then independent expert-built evaluations become procurement infrastructure rather than a research curiosity. That is a real business if buyers come, and a hard one to scale if they don't. The labs themselves have every incentive to keep evaluation in-house; Forum AI's pitch only works if customers decide they need a second opinion.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
