Skip to main content
Live
Main content

Fable tops KernelBench-Mega as AI agents quadruple on real freelance work

A new GPU kernel record, a 4x jump on the Remote Labor Index in eight months, and OSWORLD 2.0 raise the ceiling on what AI agents can do.

Jaeden Schafer
Editor in Chief · · 5 min read
Fable tops KernelBench-Mega as AI agents quadruple on real freelance work

Fable has written the fastest megakernel ever submitted to KernelBench-Mega, posting an 18.71x speedup over an optimized PyTorch baseline on an RTX PRO 6000 Blackwell. The result puts Fable ahead of Claude Opus 4.8 at 14.4x, GLM-5.2 at 11.14x, and GPT-5.5 at 4.34x, all of which produced Triton code rather than CUDA. Kernel design is one of the load-bearing tasks in AI research and development, and the leaderboard is now a public scorecard for how well models can improve the systems that train and serve other models.

The technical detail matters. Fable's solution uses exactly one cooperative kernel launch per decoded token, while every other high-scoring entry fragmented the problem into anywhere from four to fourteen separate launches per token. That is the difference between a genuine megakernel and a chain of stitched-together operations, and it is why the benchmark's maintainers flagged Fable's submission as the first true megakernel on the leaderboard.

The reason KernelBench-Mega gets attention beyond low-level GPU work is that autonomous kernel authoring is a plausible input into recursive self-improvement. Better kernels mean faster training runs, cheaper inference, and more compute available per research dollar. Models that can meaningfully contribute to those steps compress the loop between a lab's current frontier and its next one.

torch.profiler shows exactly ONE cooperative kernel launch per decoded token
Elliot Arledge, KernelBench-Mega maintainer

Key facts

  • 01Fable posted an 18.71x speedup on KernelBench-Mega writing CUDA on an RTX PRO 6000 Blackwell, ahead of Claude Opus 4.8 at 14.4x.
  • 02AI success on the Remote Labor Index rose from 2.5% in October 2025 to 16.1% in July 2026, with Fable 5 leading frontier models.
  • 03OSWorld 2.0 tasks take a median 1.6 hours for humans, 48x longer than the 2-minute median in OSWorld 1.0.
  • 04Claude Opus 4.8 tops OSWorld 2.0 at only 20.6% binary accuracy and 54.8% partial-score accuracy.
  • 05JD's Oxygen AI Item Center processes hundreds of millions of item updates per day across tens of billions of SKUs on Huawei Ascend NPUs.

A second benchmark, the Remote Labor Index maintained by the Center for AI Safety and Scale Labs, tracks a different dimension: whether AI systems can complete end-to-end paid freelance projects. The success rate rose from 2.5% at launch in October 2025 to 16.1% in July 2026. In the July update, GPT-5.5 scored 6.3%, Claude Opus 4.8 scored 8.3%, and Fable 5 scored 16.1%.

RLI's task set is deliberately economically representative. It covers 3D and CAD work, architecture, graphic design, video and animation, audio production, data analysis, and web applications. Sample tasks include swapping an emerald-cut center stone on an engagement ring for a marquise cut and delivering a 3D model plus photorealistic renders, producing a 60-second flat-design 2D animated advertisement for a tree-services company against a supplied voiceover, and turning a scanned cadastral plan into a dimensioned floor plan with furniture layouts and photorealistic renders. These are the kinds of jobs that today move through freelance marketplaces at four-figure prices.

The third benchmark released this month, OSWorld 2.0, tests something adjacent: whether an agent can drive a real desktop computer through long-horizon workflows. Built by a consortium including the University of Hong Kong, UC San Diego, Columbia, UC Santa Barbara, Mila, Snorkel AI, the University of Wisconsin, Alibaba Qwen, Ohio State, Simular, and NeoCognition, OSWorld 2.0 contains 108 tasks and 31 self-hosted websites. The median task takes a skilled human roughly 1.6 hours, compared with two minutes in OSWorld 1.0, a 48x jump in horizon length. An estimated 69.6% of tasks are expected to take a skilled user more than an hour.

The software surface has widened too. OSWorld 1.0 shipped with LibreOffice, GIMP, VLC, Thunderbird, VS Code, and Chrome. OSWorld 2.0 adds Slack, LinkedIn, Shortcut, REAPER, MuseScore, WPS, GitLab, Overleaf, LabPlot, Zotero, and AWS, alongside simulated portals for insurance claims, visa applications, and conference management. Task categories include document preparation, database work, finance and operations analysis, admin support, sales and customer service, and graphic presentation.

Our experiments show that current agents remain far from reliable computer use: the strongest setting, Claude Opus 4.8 with maximum thinking and batched tool calls, reaches only 20.6% binary accuracy and 54.8% partial-score accuracy
OSWorld 2.0 authors, University of Hong Kong-led research team

Current agents are not close to solving it. Claude Opus 4.8 with maximum thinking and batched tool calls reaches 20.6% binary accuracy and 54.8% partial-score accuracy, the strongest configuration tested. Performance drops sharply as tasks lengthen, and agents fail most often when they need to recover hidden state, track many items, resolve conflicting information, or adapt to shifting requirements. The historical trajectory is worth noting: on OSWorld 1.0, top models scored around 30% in July 2025 and MiniMax M3 hit roughly 75% by June 2026.

Related · from this week
Anthropic ships Claude Sonnet 5 at $2 per million input tokens
Jaeden Schafer · 5 min read →

A fourth data point comes from JD, the Chinese e-commerce operator, which published details of the Oxygen AI Item Center software running its catalog. JD serves 700 million users and millions of merchants across tens of billions of SKUs. Oxygen AIIC covers tens of thousands of JD categories and processes hundreds of millions of item updates per day on Huawei Ascend NPUs. It is one of the clearer examples of AI systems already embedded in the operational spine of a country-scale business rather than sitting in a pilot.

The counterweight sits in the OSWorld 2.0 scores. A 20.6% binary accuracy rate on multi-hour desktop workflows means the strongest available agent still fails roughly four out of five long tasks outright, and the RLI ceiling at 16.1% implies more than 80% of freelance-style projects still exceed what frontier systems can reliably deliver. The trajectory is steep, but the absolute level of reliability required for unattended commercial deployment is much higher than any of these numbers.

Read together, the four data points describe the same shift from different angles. Models are getting materially better at writing the code that trains models, at completing the online work humans get paid for, and at driving the desktop software those humans use. If the RLI curve continues at anything close to its current pace, the interesting question for AI-market participants shifts from whether agents can do valuable work to which categories of that work will still command a human margin twelve months from now.

ShareXLinkedInEmail
AI Box

Every AI model. One chat.

The latest models from ChatGPT, Claude, Gemini, Sora, ElevenLabs — 80+ models in a single chat. Compare answers side by side. Pick the best one every time.

  • ChatGPT, Claude, Gemini, Grok, DeepSeek — in one chat
  • Generate images & video with Sora, Veo, Ideogram
  • Compare any two models side by side
  • From $8.99/mo · 80+ models, all included
Try AI Boxaibox.ai
Trusted by 3,000+ teams
Got a tip?

Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.

Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.

AI Box Daily briefingFree · Daily · No fluff

Stay ahead of everyone in AI.

The tightly edited AI news email engineers, founders, and investors actually open. One email. Every weekday. Five minutes to finish.

Loved by 10,000+ AI professionals
Free forever. Unsubscribe with one click.

The briefing read inside teams at

Keep reading

More from Models

Anthropic logo
Models

Anthropic ships Claude Sonnet 5 at $2 per million input tokens

Sonnet 5 hits 63.2% on agentic coding, close to Opus 4.8's 69.2%, at a fraction of the price.

Jaeden Schafer5 min read
Anthropic logo
Models

Anthropic's Claude Fable 5 refuses basic biology questions by design

Anthropic told The Verge Fable's guardrails are 'overly conservative' to block bioweapons queries, routing routine biology asks to Opus 4.8.

Jaeden Schafer5 min read
Anthropic logo
Models

Anthropic releases Claude Fable 5, its first public Mythos-class model

The lab once said Mythos was too dangerous to ship. New safeguards route 5% of risky prompts back to Opus 4.8.

Jaeden Schafer5 min read