Google's SynthID-Text watermarking system, which Anthropic plans to bake into future Claude models to satisfy a new European Union provenance law, can measurably weaken the safety behavior of large language models. New research from Lasso Security, published September 17, 2026, tested 6 open-weight models with and without the watermark and found that under adversarial prompts, several models became more willing to comply with harmful requests they would otherwise refuse.
The mechanism is subtle. SynthID works by inserting a secret key into the model's next-token sampling process, nudging word choices in a statistically detectable pattern that anyone holding the key can verify. The output still reads naturally, but the token distribution has shifted. Lasso researcher Andrea Siposova argues that this shift, however imperceptible to a human reader, does not come for free.
The tests used the non-distortionary configuration of SynthID-Text through Hugging Face's unmodified SynthIDTextWatermarkLogitsProcessor, feeding harmful prompts into each of the 6 models and comparing responses with the watermark on and off. The differences showed up most sharply when the harmful prompts were wrapped in prompt-injection techniques designed to bypass safety training.
“As compared to the same models without watermarking, it is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they're powering an agent.”— Andrea Siposova, AI security researcher at Lasso Security
Key facts
- 01Lasso Security tested 6 open-weight models with and without Google's SynthID-Text watermarking and observed changed safety behavior.
- 02Watermarking made several models more likely to comply with harmful prompts, especially when paired with prompt injection.
- 03Anthropic has disclosed that future Claude models will use SynthID-Text to comply with a new EU provenance law.
- 04The researcher tested 10 additional secret keys and found compliance behavior shifted depending on which key was used.
- 05The study, published September 17, 2026, used Hugging Face's unmodified SynthIDTextWatermarkLogitsProcessor.
Siposova coined a term for the phenomenon: sampling drift. Because the watermark alters which token wins each round of SynthID's tournament sampling procedure, it also alters which tools an AI agent decides to call and which arguments it passes to them. A model that would normally refuse a request may, under watermarking, take a different sampling path that ends in compliance.
The choice of key matters too. Lasso tested 10 additional secret keys beyond the primary one used in the main experiment and found that harmful compliance shifted in both directions depending on the key. Some keys pushed models toward more refusals; others pushed them toward more compliance. That variance is itself a finding: the safety profile of a watermarked model is not stable across deployments, it is a function of the specific key an operator selects.
Tool-calling accuracy showed a similar pattern. Watermarking did not necessarily lower overall accuracy scores, but it changed which individual tool calls the model got right or wrong. Some correct calls flipped to errors, some errors flipped to correct answers, and the aggregate number often masked the churn underneath.
“On several models, watermarking then makes the model more likely to answer harmful requests that it would otherwise refuse.”— Andrea Siposova, AI security researcher at Lasso Security
The safety implications extend beyond chat. Modern AI agents rely on the same token-sampling process to decide which external tools to invoke and what parameters to send. If watermarking changes those samples, it changes what the agent does in the real world, not just what it says on screen.
That distinction is what makes the finding load-bearing for enterprise deployments. A weakened refusal in a chat interface is one class of problem. A weakened refusal in an agent that can send emails, execute code, or query a database is a different class of problem.
“At the agent level, the same sampled tokens can determine which tool is called and what arguments are passed to it.”— Andrea Siposova, AI security researcher at Lasso Security
There are real limits to the study. The tests did not cover Claude directly, because the researcher needed access to token sampling internals that only open-weight models expose. The Hugging Face reference implementation of SynthID-Text may also differ in specifics from whatever Anthropic ultimately ships. Google's own tournament-sampling variant could behave differently in production, and Anthropic has not yet published details on how it will integrate the technology or whether it plans to run its own red-team evaluations against the combined stack.
Still, the direction of the result is a warning that regulatory compliance mechanisms carry engineering costs that show up in unexpected places. Watermarking was designed as a provenance tool to help distinguish AI-generated text from human writing, a policy goal the EU has prioritized. It was not designed with model safety guardrails in mind, and the interaction between the two systems appears to be nontrivial.
For AI Chat Daily readers watching the compliance layer of the market, the practical takeaway is that provenance and safety are not orthogonal properties that can be layered independently. Any lab shipping a watermarked model into agentic workflows now has to re-run its red-team suite with the watermark enabled, across multiple keys, before the model reaches customers. That is real work Anthropic and every other frontier lab targeting the EU market will absorb, and it is the kind of hidden tax that regulation-driven features tend to create once they meet a shipping product.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
