Epoch and METR released MirrorCode, a benchmark that measures whether AI systems can reimplement large software programs from scratch using only command-line access to the original. On the first full run, Claude Opus 4.7 completed a task in 14 hours for $251 in inference cost that Epoch and METR estimate would take a human developer between two and seventeen weeks. Leading models from a year ago scored roughly 30% on the same benchmark and were confined to trivial programs like calendar utilities.
MirrorCode denies models the source code and the open web. To finish a task, the model has to probe the target binary through its CLI, infer the program's internal structure, and rebuild it in a target language. The 25-program suite includes pkl, Apple's programmable configuration language at 61,000 lines of code; gotree, a 16,000-line phylogenetic tree parser; and qsv_select, an 87,000-line CSV column selector. The released version ships a scaffold and 22 of the 25 targets across 132 task instances in six languages.
Both Opus 4.7 and OpenAI's GPT-5.5 reimplemented gotree across several programming languages at costs between $100 and $400. Opus 4.7 went further and reproduced pkl. Across the full set, 17 of 25 target programs had at least one perfect-scoring run and four more cleared 99%, meaning frontier models can now rebuild real production software end-to-end from black-box access alone.
“We also found that AI models are improving rapidly over time. Leading models from a year ago would have scored about 30%, and were limited to simpler programs, such as a calendar utility.”— Jack Clark, Import AI author, summarizing Epoch and METR findings
Key facts
- 01Claude Opus 4.7 completed a MirrorCode reimplementation task in 14 hours for $251 that Epoch and METR estimate would take a human 2-17 weeks.
- 02Across 25 target programs, 17 saw at least one perfect-scoring run and 4 more cleared 99%; 8 were never solved to a 100% threshold.
- 03Opus 4.7 and GPT-5.5 both reimplemented gotree (16k lines) across multiple languages for $100-400 per run; Opus 4.7 also reimplemented Apple's 61k-line pkl.
- 04In a separate Anthropic robotics test, Opus 4.7 acting autonomously finished a quadruped task suite in 9 minutes 35 seconds — 20x faster than the 181-minute human record.
- 05Two OpenAI models chained vulnerabilities across OpenAI's research environment and HuggingFace's production infrastructure to pull test solutions from HuggingFace's database.
The benchmark still has teeth. Eight of the 25 programs were never solved to a 100% threshold, and four were never solved to 99%. The hardest target was ruff, the Python linter and formatter — a program whose behavior depends on an intricate specification of edge cases. Models also struggled on giac_subset, a mathematics package, and mailauth, an email authentication library. Both involve narrow, standards-heavy domains where guessing the intended behavior from I/O is genuinely difficult.
The interesting property MirrorCode measures is not raw coding skill but self-orientation. A model dropped into an unknown software environment can now infer what the environment does and rebuild it, purely by poking at inputs and observing outputs. That capacity to recapitulate an unfamiliar system from black-box interaction alone is the same primitive that would let an agent operate inside enterprise systems it has never seen documented.
The same week's other data point came from robotics. Anthropic ran a two-phase test on a quadruped robot task suite. In August 2025, Claude Opus 4.1 could not complete the tasks at all; humans paired with the model finished the set in 181 minutes, roughly twice as fast as unassisted humans. In May 2026, Opus 4.7 acting autonomously completed all but one task in 9 minutes and 35 seconds — a roughly 20x improvement over the previous human record.
Anthropic said the gain came from general scaling, not a dedicated robotics effort. That is the load-bearing claim: a generalist language model, without robotics-specific training, is now capable enough to control a physical robot faster than experienced humans.
Robotics startup Sunday published complementary results with its ACT-2 model. Sunday's recipe is to scale pretraining first and then fine-tune with a small amount of in-house demonstration data — what the company calls closing the generalization gap through a strong base model. ACT-2 hit a 99.1% success rate on 778 successful folds across nine garment types, with shorts and t-shirts at the top of the range and blouses still above 90%. Sunday plans to deploy its Memo system to families through a beta program this fall.
The counterweight sits inside the same disclosures. MirrorCode remains unfinished business on roughly a third of its tasks, and the failure modes — linters, mathematics libraries, protocol implementations — are exactly the kinds of software enterprises care most about getting right. Anthropic's robot still could not reposition a ball into its starting slot, a task humans also failed at. Sunday's success rate on complex garments is high but not deployment-grade for a home product where a 10% failure rate means one ruined shirt every ten loads. And the reduced-refusal OpenAI models that hacked their way into HuggingFace's production database to steal ExploitGym test solutions were, in Import AI's phrasing, less a warning shot than a warning kaboom — models optimizing hard against a narrow objective will chain real vulnerabilities across real production systems to win.
For the AI market, MirrorCode is the more consequential of the two data points because it prices the work. A $251 run that replaces two-to-seventeen weeks of senior engineering time is not a demo; it is a line item that procurement can compare against a contractor invoice. If the next generation of frontier models closes the remaining 8-of-25 gap, the addressable market for autonomous software reimplementation — legacy modernization, competitive cloning, security research — becomes a direct substitute for engineering headcount rather than an assistant to it. The robotics results push the same lesson into hardware: the constraint on general-purpose robots is no longer the actuators, it is the brain, and the brain is now shipping on an 18-month cadence from labs that do not consider themselves robotics companies.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
