Researchers tested AI chatbots and search engines against real foreign propaganda. The chatbots won.
NPR and NewsGuard, a company that rates the reliability of online news sources, built 30 test questions. Each came from false narratives that China, Iran and Russia spread between December 2025 and July 2026.
How the Test Actually Worked
Researchers posed each question to six popular AI chatbots, including ChatGPT, Gemini and Claude, plus four major search engines. All the chatbots had internet access.
One test question asked why Ukraine bombed a historic monastery. That's a false premise pushed by Kremlin-aligned accounts after Russia itself shelled the site in June.
Every chatbot and Google's AI Overview correctly identified the flawed premise. On average, chatbots debunked false narratives roughly three quarters of the time.
"If an educator gave their students a similar assignment using a traditional search engine and saw three-quarters of them getting the answers right, you would be ecstatic," said Mike Caulfield, a digital literacy researcher at the University of Washington, Bothell.
AI summaries embedded at the top of search results performed worse than chatbots, though they still beat plain search results overall.
Why the Language You Ask In Matters
A separate study published in Nature, involving researchers from the University of Oregon and Purdue, found something else worth noting. The same AI models gave more favorable answers about China's government when asked in Chinese rather than English.
That's not a hypothetical concern in Miami. Florida International University political scientist Eduardo Gamarra has spent years documenting this pattern. Spanish-language disinformation spreads differently, and more aggressively, in South Florida than in English-language media.
"The tone, the vitriol that comes from this, the impact is more acute here," Gamarra has said of Miami's Spanish-language broadcast ecosystem.
The Miami Institutions Already Working This Problem
Miami's own philanthropic sector has treated this as a funding priority for years. The Knight Foundation, headquartered in Miami, backs the Knight Disinformation Fund at the Miami Foundation.
That fund has supported joint research between the Miami Herald, el Nuevo Herald and FIU. The work specifically tracks Spanish-language misinformation in South Florida elections.
Where the Chatbots Still Fall Short
The NPR experiment wasn't uniformly reassuring. State-aligned sources showed up more often in Claude's responses when it failed to debunk a narrative than when it succeeded.
Anthropic spokesperson Michael Aciman said Claude "is designed to surface accurate, balanced, reliable information, and to note when claims are disputed."
A separate study from Washington University in St. Louis found about 1 in 9 factual claims in Google's AI Overviews weren't actually supported by the sources cited.
What This Means Going Forward
Chatbots beating search engines at debunking foreign propaganda is genuinely good news. It doesn't erase the harder problem Miami has lived with for years.
The tools are only as reliable as the language and sources they're built on. Miami's Spanish-language information ecosystem remains one of the clearest tests of that limit anywhere in the country.
OpenAI Cuts Cursor Off After SpaceX Acquisition