Augment Code says its coding agent completed Terminal-Bench tasks at the same accuracy as Claude Code while using 33% fewer tokens, running on the same underlying model. Vinay Perneti, Augment's VP of Engineering, framed the result as validation of a semantic-retrieval harness — one that pre-indexes a codebase with embeddings and a vector database — over the grep-based approach favored by Anthropic's Claude Code and OpenAI's Codex. The claim lands in the middle of an unresolved argument about how much scaffolding should sit between a developer and a frontier model.
The dispute is about the harness, not the model. A harness is the software layer that decides what code the model sees, what tools it can call, and how it navigates a repository. Anthropic's Cat Wu, head of product for Claude Code, told Ars Technica earlier this summer that Claude Code's internal evals show no measurable gain from pre-built semantic indexing, and that the team prefers to ship a leaner harness rather than bet on opinionated tools that may age poorly over 6 months or 12 months of model improvement.
Augment took the opposite bet. Perneti said the company spent 18 months at its 2022 founding — pre-ChatGPT — researching retrieval and embedding models tuned for large codebases. The product pairs a custom embedding model with a retrieval model and a vector database engineered for sub-millisecond lookups, then hands the retrieved code to the agent as context.
Key facts
- 01Augment Code says it matched Claude Code's accuracy on Terminal-Bench with the same underlying model while using 33% fewer tokens.
- 02Augment spent 18 months from its 2022 founding researching retrieval and embedding models for large private codebases, before ChatGPT shipped.
- 03Anthropic's Claude Code team says its internal evals show no measurable gain from pre-indexed semantic retrieval, and prefers a lean harness.
- 04Competing harnesses include OpenAI's Codex, Google's Antigravity, open-source OpenCode, and Cursor — each with different opinions on context.
- 05Perneti argues semantic retrieval's advantage shows up in private repos, since public benchmarks run on repos the models have effectively memorized.
Perneti argued the two teams are effectively measuring different problems. Public benchmarks run against open-source repositories that today's frontier models have already ingested during training.
“For all the public, open source repos where most of the benchmarks are run, every single model has basically memorized the repo.”— Vinay Perneti, VP of Engineering, Augment Code
In a private repo, he said, the model has no prior exposure, and the exploration loop lengthens sharply. A semantic index over the private codebase, in Perneti's telling, is what closes that gap — and it's a gap that public benchmarks systematically hide because the model already knows the answer.
The Terminal-Bench number is Augment's headline evidence. Same model, similar accuracy, 33% less token spend — an efficiency claim rather than a capability claim.
“We've certainly seen it in certain situations. In fact, we published a blog post—we ran Terminal-Bench with Claude Code and Augment Code, same model. And we completed at similar accuracy, but we were 33 percent more efficient than Claude Code.”— Vinay Perneti, VP of Engineering, Augment Code
That reframing matters because token cost is now the constraint engineering leaders are pushing back on. Agentic coding workflows can burn through context windows quickly when the agent has to grep, read, and re-read files to reconstruct what a semantic index would surface in one call. If Augment's number holds up in independent testing, the harness-level design choice translates directly into infrastructure bills.
Perneti's broader argument is that intelligence and context are separable inputs. Model intelligence is scaling exponentially, he said, but a smarter model doesn't automatically have the right context — it has to spend tokens to get it.
“There's two ingredients that you need for higher quality outcomes, intelligence and context.”— Vinay Perneti, VP of Engineering, Augment Code
Harness design, in his view, is the systems-engineering problem of routing token budget efficiently between context gathering and reasoning. That's a defensible position, but it's also the position of a company whose product is a context engine. Anthropic's counterargument — that models improve fast enough that any fixed context infrastructure becomes a liability — is equally self-interested and equally plausible.
On developer skepticism, Perneti was more measured than the sales pitch usually allows. He conceded that treating agents as fire-and-forget produces real technical debt, and that agents are not good at writing specs. His framing was teams of humans working with teams of agents, with humans retaining judgment on spec review and agents executing once the spec is solid. It's a workflow claim more than a product claim, and it's closer to how the more careful shops are already operating.
The unresolved question is whether context engineering is a durable moat or a temporary arbitrage. If models keep extending effective context windows and improving in-context retrieval, the advantage of a pre-built semantic index shrinks. If enterprise codebases keep growing faster than context windows — and private code stays outside training sets — the advantage grows. Neither Anthropic nor Augment can prove their thesis on today's benchmarks alone.
For the AI coding market, the split matters more than the winner. Claude Code, Codex, Google's Antigravity, Cursor, OpenCode, and Augment are now visibly diverging on harness philosophy rather than converging on a single design. That's healthy for buyers, who get to test both approaches on their own repos, and it's a tell that the category is still early — mature markets standardize on architecture, and this one hasn't. The 33% efficiency gap, if it survives scrutiny outside Augment's own blog, is the first hard number worth arguing over.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
