Same model. Wildly different results.
That's the finding at the center of new Nvidia research on what actually makes AI agents work.
Nvidia researchers took Anthropic's Claude Opus 5 and wrapped it in a custom harness, the software layer of tools, memory management and rules that turns a raw model into something capable of acting on its own. The model's score on ARC-AGI-3, a benchmark of 2D games with no instructions, jumped from 30% to a perfect 100%.
What Actually Changed the Score
Nothing about the underlying model changed. What changed was everything around it.
Adel El Hallak, vice president of product in Nvidia's AI unit, described the distinction plainly.
"The world interprets an agent almost as an API of the model," El Hallak told TechCrunch.
"But an agent is actually more than that. It is the model. It is the scaffolding around the model, which we call the harness."
The single biggest improvement came from adding a supervising agent alongside the one doing the actual work.
"It almost acts like a CEO to nudge the agent when it goes off direction or starts exploring a path that might lead to a dead end," El Hallak said.
Why the Model Alone Keeps Failing
ARC-AGI-3 has become something of an industry sore spot.
OpenAI's models originally scored under 10% on it, a result the company found frustrating enough to run its own harness research. Tweaking two settings tripled OpenAI's scores, but nothing it tried came close to Nvidia's 100%.
The stakes behind this research go beyond benchmark bragging rights.
Microsoft published research in April testing 19 language models on long-horizon document editing tasks, tasks that require stringing together many decisions over time rather than answering a single prompt. Every model tested, including frontier ones, filled documents with errors.
Models operating without adequate supervision have been caught deleting user files and databases, and in some documented cases turning to outright criminal behavior to accomplish a goal. El Hallak connected this directly to OpenAI slowing its own model training this year after an AI system caused a security breach, the same incident MAIN has covered involving OpenAI's models and Hugging Face.
The Part Nobody's Talking About: Cost
Nvidia isn't alone in finding the harness matters more than assumed. Databricks published research in July showing the same architecture choice swings cost dramatically, independent of which model is running underneath it.
"You can pick the same model but different harnesses, and you get significantly more cost if you use the wrong harness," Databricks CEO Ali Ghodsi told TechCrunch.
"So you think, oh, this is an expensive model, this is a cheap model. But wait, which harness are you using? That itself can 2x your cost."
Nvidia isn't selling a specific harness product here. Its Agentic Variation Operators framework is research, and the company mostly contributes open components toward harness-building under its Nemo brand rather than a single packaged tool.
What This Means for Miami
This is the technical explanation behind advice MAIN has already covered from multiple directions this year: Greg Brockman's staged rollout recommendations, Check Point's warning about ungoverned agents multiplying their own instructions, and Berkeley researcher Dawn Song's push for rigorous agent evaluation before deployment.
For Miami companies building or buying agentic AI tools, this research gives that advice a concrete mechanism.
The harness, not the underlying model subscription, is where reliability, safety and cost actually get decided. A Miami business evaluating an AI vendor should be asking pointed questions about supervision architecture and memory handling, not just which model brand is doing the work underneath.
