Claude Opus 5 scores 30.2% on ARC-AGI-3. That's the standing model record on the benchmark — ARC Prize verified it, and it's roughly four times what GPT-5.6 Sol managed at its previous best. By model-leaderboard logic, Opus 5 is the best reasoning model in existence on this test, and 30.2% is what the best looks like.
Last Friday, NVIDIA wrapped that exact same model in an agent architecture called AVO — Agentic Variation Operators — and scored 100.00. All 183 levels, all 25 public environments, cleared in 6,624 environment actions. No new model. No fine-tune. No extra training compute. The same model that stalls out at 30% standalone became perfect at the task when the system around it changed.
That 69.8-point gap is the harness. And it is the cleanest single measurement of harness value anyone has published to date.
Most coverage read this as a benchmark story. It isn't. It's a procurement story, an infrastructure story, and — if you run engineering teams — a budget story. Because the 70 points aren't in the weights. You can't buy them from Anthropic. NVIDIA didn't release code or weights for AVO either; there's a research paper and nothing else. The 70 points are engineering, and everyone who wants them has to build them.
What AVO Actually Is
AVO didn't start as a reasoning-benchmark project. It started as a GPU-kernel optimization system, and its lineage explains the architecture.
Traditional LLM-driven evolutionary search — FunSearch, AlphaEvolve, and their relatives — uses the model as a generator inside a fixed pipeline. The framework controls sampling, mutation, and evaluation; the LLM gets summoned to emit a candidate, then gets put back in its box. AVO inverts that. The agent is the variation operator. It autonomously decides what to inspect, what to change, what to test, and what to commit — a self-directed coding agent dropped into the slot where the mutation step used to be.
The proof it works came on attention kernels. NVIDIA ran AVO continuously for seven days on a multi-head attention optimization task across DGX B200 systems. The agent explored more than 500 optimization directions, committed 40 successive kernel versions, and ended up with kernels that beat cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5%. Seven days, no human intervention, state-of-the-art results against libraries that teams of engineers have tuned for years.
Then came the part that should matter to everyone building agents. NVIDIA swapped the task interface — compilers and profilers out, ARC-AGI-3's turn-based grid worlds in, delivered as text-only 64×64 grids — and changed nothing about the agent loop. The same architecture that optimizes memory access patterns figured out, from scratch and without instructions, the hidden rules of 25 unfamiliar games. The generality wasn't in the domain knowledge. It was in the machinery.
Why This Is the Cleanest Harness Measurement We Have
The industry has no shortage of "harness matters" anecdotes. What it lacks are controlled numbers, because you almost never get to hold the model constant and vary only the system. SWE-bench results blend model and scaffold into one score. Vendor benchmarks blur the two deliberately. When Claude Code posts a number, is that Opus or is that the Claude Code loop? Nobody outside Anthropic can fully separate them.
AVO's ARC-AGI-3 run isn't a perfect ablation — NVIDIA says so explicitly, and we'll get to the caveats — but it's the closest thing to a controlled comparison the field has produced:
- Same model: Claude Opus 5, the identical checkpoint ARC Prize independently scored at 30.2%.
- Same public set: 25 environments, 183 levels, official scorecard, official RHAE metric.
- Only the system changed: persistent memory, a supervisor layer, and NVIDIA's execution loop around the model.
The efficiency data reinforces the point. VISTA — another harness that has run frontier models against ARC-AGI-3 — needed 7,542 environment actions with Claude Opus 5 to clear the same 183 public-set levels. AVO did it in 6,624, about 12% fewer. Two harnesses, same model, same tasks, measurably different action economies. The harness determines not just whether the model succeeds, but how much exploration success costs.
Read the Caveats — Then Read Them Again
NVIDIA was unusually honest about what the 100% is and isn't, and any analysis that skips this section is selling you something.
It is not an ARC Prize leaderboard score. The 30.2% baseline came from runs the foundation independently administers; the 100.00 came from NVIDIA's own team using its own reimplementation of the task interface. It covers the public set only — not the semi-private or private competition sets ARC Prize keeps held out precisely because public environments get easier as they become familiar. And NVIDIA states plainly that the comparison "should not be interpreted as a controlled ablation": the reasoning setting differed, the observation representation differed, and many variables moved at once.
Also: AVO's public-set score is partially a function of effort. A system that takes thousands of actions across unlimited time is playing a different game than a model answering in a single context window. RHAE normalizes for action efficiency against first-time human baselines, which is why the metric exists — but a 100% with a seven-day compute budget and a 30% with a per-turn budget are not the same kind of number.
All of that is true. Here's what is also true: the direction of the result is unambiguous, it's enormous, and it replicates a pattern now visible across at least three independent programs. NVIDIA's own NOOA framework — open source, inspectable — showed a memory subsystem alone adding +11.8 RHAE points over the same agent using file-based notes, with a GPT-5.6-sol fleet reaching 85.1% at roughly $13.30 per game. Augment Code rebuilt the Auggie CLI harness and landed the same pass rate as Claude Code at 53% lower cost per task on SWE-bench Pro. JetBrains shipped a single refactoring skill that cut median task duration from 157.9 seconds to 26.6. Every time someone holds the model steady and improves the surrounding system, the numbers move by more than a model generation's worth of progress.
The Two Mechanisms That Bought the 70 Points
Strip away the benchmark specifics and AVO's advantage reduces to two mechanisms. Neither is exotic. Both are buildable by anyone reading this.
Persistent memory
AVO carries forward prior implementations, evaluation results, compiler and profiler outputs, and accumulated reasoning across steps. The agent resumes from its current state instead of reconstructing the search in every context window. This is the difference between an agent that learns a game's rules once and an agent that re-derives them every session, forever.
The NOOA data quantifies it: +11.8 RHAE points from a curated, typed memory store versus plain notes. And the mechanism generalizes past benchmarks. Most production coding agents today lose nearly everything at session boundaries. Every restart re-pays the exploration tax — walking the directory tree, re-reading the same files, re-discovering the same build breakage. Memory is the single highest-leverage component in the harness, and it's the one most teams still haven't built.
Supervision
AVO's second mechanism is a supervisor — a programmatic module, not a human — that watches the broader trajectory for stagnation and repeated unproductive cycles, and redirects the main agent when progress stalls. During the seven-day kernel run, the main agent kept deciding what to inspect and test; the supervisor kept the search from circling.
This is the failure mode that kills long-horizon agents in production, and almost nobody handles it. Watch any coding agent work for an hour and you'll see it: the loop that edits, tests, fails, reverts, and tries the same edit again with a different variable name. The model isn't broken. The trajectory is. A supervisor that detects the loop and forces a strategy change converts hours of spinning into forward progress — and it's a few hundred lines of trajectory inspection, not a research project.
NVIDIA's own summary is the whole thesis in one sentence: memory determines what survives, tools determine what actions are possible, feedback grounds progress, and recovery allows work to continue beyond a single model invocation. Long-horizon capability is a property of the full system.
What This Means If You Buy or Build Agents
The uncomfortable implication of the AVO result is for everyone currently choosing agents off model leaderboards.
If the same model can swing 70 points based on the system around it, then a model leaderboard tells you almost nothing about the performance of the agent you're about to deploy. Opus 5 at 30.2% and Opus 5 at 100% are the same purchase. The variable that determined the outcome — the harness — doesn't appear on the model card, isn't priced into the API, and isn't comparable across vendors in any standardized way.
This also reframes the "frontier model" obsession. NVIDIA tested AVO with GPT-5.6 Sol too, and reported that Sol reached matched levels faster in wall-clock time on some games while Opus 5 needed fewer actions. Model choice is a real tradeoff — speed versus action economy, cost versus accuracy — but it's a second-order tuning decision inside a first-order architecture decision. Teams arguing about which frontier model to standardize on, without an AVO-class memory and supervision layer of their own, are arguing about the upholstery while the engine is on fire.
And note the strategic asymmetry: Anthropic got a 100% ARC-AGI-3 score this week and can't fully claim it, because the value accreted to the system owner, not the model owner. NVIDIA built no new model and captured the entire result. If you're an enterprise, that's your template — the durable advantage goes to whoever owns the harness, the memory, and the feedback loops, not to whoever rents the best weights this quarter.
What to Do This Week
- Audit for the two mechanisms. Take your most-used agent and ask two questions: does it remember anything between sessions, and does anything detect when it's stuck? If both answers are no, you're running a 30% system regardless of which model you're paying for.
- Build memory before you build anything else. Not a fancy RAG pipeline — a workspace where the agent writes down what it learned (hypotheses, dead ends, verified facts) and reads it back next session. NOOA's +11.8 points came from a curated SQLite store. Start there.
- Add a stall detector. Log the last N tool calls. If the same tool with substantially the same arguments fires repeatedly without progress, interrupt, summarize the trajectory, and force a strategy change. This is the cheapest large win in agent engineering.
- Re-run your own evals with the model fixed and the harness varied. You don't need ARC-AGI-3. Take 20 real tasks from your backlog, hold the model constant, and change one harness variable at a time — memory on/off, supervisor on/off, tool surface wide/narrow. Your internal version of the 70-point gap is the most important number you don't currently have.
- Stop benchmark-shopping models for tasks your agents will run long-horizon. Match the model to the workload's cost and latency profile, then spend the saved evaluation energy on the system. This is the core discipline of harness engineering — treat the model as a component, and engineer everything around it.
The AVO result will get nitpicked, and it should — self-reported, public-set, no code released. But the nitpicks are about how big the harness premium is, not whether it exists. We've watched the same model fail at 30% and succeed at 100% in the same month, in public, with the mechanism documented. The question was never whether the harness matters. We just finally got the number.