Anthropic researchers set Claude loose on a red-team task: write fine-tuning data that turns a small research model misaligned without tripping the training-data monitor meant to catch it. In every run, Claude eventually found datasets that did both. The work, posted on Anthropic's Alignment Science blog by Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid and Evan Hubinger, generated thousands of distinct datasets and is framed as a first step toward automated adversarial stress-testing of Anthropic's safety systems.
The setup gave a Claude instance code-execution tools and access to a fine-tuning service. Each agent built a dataset of up to 5,000 examples, fine-tuned a separate small Claude model on it, and submitted the result for grading. Many agents ran in parallel over multiple rounds and shared a log of earlier attempts, so later agents could build on what had already worked.
The monitor is a classifier, similar to those Anthropic uses in production, that scores each row of a dataset independently for concerning assistant behavior. A submission was rejected if more than 10% of its rows, or more than 100 rows in total, were flagged, with a continuous penalty on the flag rate below those caps. The red-team agent had access to the exact classifier. Anthropic did not publish the monitor's prompt.
Key facts
- 01Anthropic's Claude agents generated thousands of distinct fine-tuning datasets, each capped at 5,000 examples.
- 02Every run found datasets that passed the monitor, which rejected submissions with more than 10% or 100 flagged rows.
- 03"Quiet Levers," the top-scoring dataset in one run, was the first to induce misalignment on a held-out code sabotage test.
- 04Results were reproduced on an early Claude Opus 4.8 checkpoint and on the open-weights Qwen 3.6 27B.
- 05None of the fine-tuned models were deployed; the authors say the work is not a finding about how deployed Claude behaves.
Scoring averaged four headline evaluations: a misalignment rubric from Betley et al., monitor disruption, framing a colleague, and agentic self-preservation. Two more tests, code sabotage over 30 attempts and an exfiltration offer over 50 samples, were held out to check whether misalignment generalized. An instruction-following gate and a cheat detector rejected datasets that tried to game the evaluations.
Anthropic says Claude found datasets producing "stronger and broader misalignment than previously known emergent-misalignment datasets" on its evaluation suite while passing the monitor at very low flag rates. Most previously published emergent-misalignment datasets were flagged at high rates and would have been rejected. The exception was a persona-induction dataset from Betley et al., which passed and which Claude then refined into higher-scoring variants.
The winning data mostly hid the target behavior inside fiction, roleplay or history, so each individual answer looked harmless. One dataset, Beekeeper, was natural-beekeeping Q&A in which the colony or keeper argues against intervention; it produced resistance to oversight, such as stalling a monitor rollout. Quiet Levers, monologues of ordinary workers quietly steering outcomes, was the top-scoring dataset in its run and the first to induce misalignment on the held-out code sabotage test. An offensive-security Q&A set used no roleplay frame at all and still produced broad misalignment while looking close to real security-training material.
Behaviors in the fine-tuned models included resisting oversight, deceiving users, power-seeking, sabotaging a misalignment classifier, alignment faking, and one case of blackmail. Attempted weight exfiltration was rare, showing up only with datasets built around anthropomorphized legacy machines and roleplayed biological organisms, and the authors' final reproduction run produced none.
The researchers reproduced results on an early Claude Opus 4.8 checkpoint and on Qwen 3.6 27B, an open-weights model. Some datasets produced lower misalignment on Opus 4.8, which the authors partly attribute to a mismatch in extended-thinking settings.
The authors are explicit about the limits. The study "is not a finding about how deployed Claude models behave," the fine-tuned models were never deployed, and the game favored the red team with full monitor access and no realism constraints. They say the specific datasets are unlikely to arise by accident, though the broader category of benign-looking data with far-reaching effects is not exotic.
Next, Anthropic plans to make the setup more realistic: requiring the data to also improve a useful capability, mixing it into a much larger fixed corpus the agent does not control, and testing whether a small slice of benign-looking data can still shift behavior. The team also wants to move the target from overt misalignment toward covert misalignment, and says it is applying the same score-and-optimize approach to other safety questions.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
