Watermark Detection as a Vendor Screening Tool for Content Buyers

Watermark detection scores, particularly the z-scores produced by the KGW method, give content buyers a real way to check AI writing vendors for provenance risk, and that turns an abstract technical worry into a step anyone running procurement can actually take. Most buyers commission AI-generated writing today without knowing which model produced it or how it got made. That gap is going to matter more, not less, because the regulatory floor is rising fast, affecting what buyers can rely on: as of August 2026, both the EU AI Act and the California AI Transparency Act require watermarking on AI-generated images, video, and audio, though neither law touches plain text yet. So a strange asymmetry has opened up: the compliance duty sits with the model provider that built the tool, while the liability for what gets published sits with the buyer who paid for the words and put them out into the world. A vendor's claim about which model wrote a batch of copy is, by itself, unverifiable. Nothing stops a shop from promising a top-tier model and quietly routing the work through something cheaper, and until recently there was no way to catch that by looking at the delivered text itself.

Why a watermark is invisible to normal inspection

Clear up the misconceptions first, because most people guess wrong here. A text watermark is not hidden characters slipped into a file, not invisible Unicode padding the spaces between words, and not metadata bolted onto the end of a document. The watermark is the text. More precisely, it is a statistical lean toward certain ordinary words over their equally plausible neighbors, and to see why that works, it helps to know how a language model actually picks a word. At every step, the model scores tens of thousands of possible tokens across its vocabulary and then samples one, turning that spread of probabilities into an actual word on the page. If "large," "big," "substantial," and "considerable" would each fit reasonably well, each one gets a score, and sampling determines which one appears in the output.

The KGW approach works right at that decision point. Before each token gets sampled, a pseudorandom function keyed to the token right before it splits the whole vocabulary into two piles, a green list and a red list https://en.wikipedia.org/wiki/Text_watermarking. The model then adds a fixed bias, called δ, to the green-list scores, which nudges the choice toward green without forcing it. Over a full paragraph or article, green tokens appear more often than pure chance would produce, and that excess is the signal a detector goes looking for. None of this requires the model to know anything unusual is happening; the sampling layer injects the bias, separate from whatever the model thinks it's doing, and that is why the resulting text reads completely normal to a human eye. Only something that actually swaps tokens out, a real paraphrase pass rather than a light edit, gets anywhere near disturbing it.

How z-scores measure the watermark signal

Detection just runs the generation process backward. Using the same secret key the model used to build its green lists, a detector reconstructs those lists token by token, counts how many green tokens actually turn up in the submitted text, and runs a one-proportion z-test against what pure chance predicts. The z-score grows with the square root of the document's length, so short pieces, anything under roughly 100 tokens, simply cannot produce a reliable signal in either direction. A detector built with any care flags that honestly and returns "TOO SHORT TO TELL" instead of guessing.

Where there is a real signal, the numbers are stark. Clean watermarked text, untouched since it left the model, produces a mean z-score around 69.22, which is not a subtle statistical whisper https://github.com/cpeoples/ai-watermark-detector. Running that same text through a full paraphrase, a genuine rewrite rather than a touch-up, drops the mean z-score to 0.15, with a p-value of 0.469 https://github.com/cpeoples/ai-watermark-detector. At that point the text reads as statistically indistinguishable from something a person wrote from scratch. A working detector has to give buyers three plain verdicts: WATERMARK SIGNAL FOUND, NO WATERMARK SIGNAL, and TOO SHORT TO TELL. That third option carries as much weight as the other two https://en.wikipedia.org/wiki/Text_watermarking. A tool that only ever answers yes or no is lying by omission the moment the sample is too small to say anything.

Which major model providers now embed watermarks

Coverage across the major labs is uneven, and each gap has real consequences for a buyer trying to screen vendors. Google has run SynthID-Text inside Gemini since 2024, and by May 2026 the company reported more than 10 billion pieces of content carrying the watermark https://www.textsight.ai/blog/google-synthid-watermarking-explained/. Anthropic moved later but harder on disclosure, and as of August 2, 2026, every newly launched Claude model embeds a watermark by default across Claude.ai, the API, and Claude Code, with cloud partner coverage rolling out model by model. Older Claude models get folded in before the EU's grace period closes on December 2, 2026. Anthropic has confirmed its scheme is a version of SynthID-Text and released a detection API in private preview, but that access is limited to regulators, law enforcement, media organizations, fact-checkers, researchers, and EU civil society groups. It is not open to the public, and it is certainly not something a mid-sized content buyer can just sign up for.

OpenAI sits furthest behind on public disclosure. The company built a watermarking system internally but held it back, citing concerns about false positives for non-native English speakers and competitive risk, so ChatGPT ships no public text watermark as of now.

On top of that sits a compatibility problem that trips up anyone running a single detector and assuming it covers everything. Detectors do not cross scheme families: a buyer running only a KGW check will walk right past SynthID output and call it clean, and the reverse holds just as true. A screening setup that actually works has to cover multiple schemes at once, and the open-source tooling available today documents six distinct families: kgw, synthid, exp, unigram, sweet, and exp-edit. A vendor routing traffic through a provider using a skipped scheme slips through invisible. Empirical testing via ai-watermark-detector shows a sample watermarked under a different scheme is at z-score baseline (mean z ≈ 0.08) when scored against a mismatched detector, which confirms the point.

A z-score alone cannot tell a buyer whether the vendor paraphrased before delivery

A high z-score feels reassuring, and that feeling deserves some suspicion. Benchmark testing found that a single ChatGPT paraphrase pass drops every tested watermarking method below a 30% true positive rate https://dev.to/sanjay_singh_1/ai-text-watermarking-the-statistics-hiding-inside-every-sentence-2g9f. SynthID, despite being one of the more resilient schemes on the market, fell from a 0.998 true positive rate on clean text to 0.498 under moderate paraphrasing, half its detection power gone in one pass https://dev.to/sanjay_singh_1/ai-text-watermarking-the-statistics-hiding-inside-every-sentence-2g9f.

It gets worse for anyone hoping watermarking alone holds the line. Research presented at ICML 2025 found what the paper called a "no free lunch" result: every current watermarking system tested could be evaded using optimized text rewriters, at success rates above 96% https://icml.cc/virtual/2025/poster/46148. The total cost of running that attack came in under $10 https://icml.cc/virtual/2025/poster/46148. These were not bespoke attacks tuned to one scheme, either; the same methods worked across watermarking systems they were never specifically designed to beat. A vendor sophisticated enough to detect a watermark in its own output before delivery is also the vendor best positioned to scrub it cleanly, since targeted removal beats blind rewriting every time. Detection, in other words, turns out to be a prerequisite for effective evasion as much as a defense against it.

Watermark detection works as an audit tool. It works best against raw, unedited model output, and commercial vendors rarely hand over raw output, most run some kind of editing or humanizing pass before anything ships. The strongest use is retrospective: auditing a vendor's archive, spot-checking a corpus delivered over the past year, looking for patterns across a body of work rather than trying to clear any single file in real time. A buyer has to sit with one uncomfortable fact going in: a missing watermark signal is an ambiguous result. It could mean human-written text, or it could just as easily mean AI text that got paraphrased once before it landed in an inbox. In the open-source tool's empirical data, a full rewrite (re-composition from a summary) causes detection to collapse toward chance, with the z-score falling from 69.22 to 0.15.

Fingerprinting as a parallel check: identifying which model a vendor used

Watermarking and fingerprinting solve related problems from opposite directions, and buyers do themselves a disservice treating the two as interchangeable https://en.wikipedia.org/wiki/Text_watermarking. Watermarking is proactive: a signal gets embedded during generation, and reading it later requires the vendor's secret key. Fingerprinting runs the other way, a retrospective, forensic read on statistical patterns already baked into a model's natural output, with no embedding step and no key required.

LLMmap, built by Pasquini, Kornaropoulos, and Ateniese, was designed to identify LLM versions, open-source or proprietary, operating under unknown system prompts, stochastic sampling settings, and complex generation setups like RAG or Chain-of-Thought. For a content buyer, the application is direct: fingerprint a batch of deliverables and check whether a vendor billing for premium-model output is actually routing requests through something cheaper. FLIPS needs far fewer text samples than LLMmap to verify a single instance, which matters when a buyer only has one or two suspect deliverables to check rather than a whole archive https://en.wikipedia.org/wiki/Text_watermarking.

Keep the provenance categories straight here, because they get used loosely and that looseness costs buyers clarity. Model provenance identifies rebranded or fine-tuned derivatives and traces them back to whatever foundation model they're actually built on, which matters whenever a vendor white-labels a model without saying so. Dataset provenance is a different animal, concerned with auditing what data trained a model in the first place, and it matters far more to regulators than to a buyer just trying to confirm which model wrote this week's blog post. FLIPS, an instance-level fingerprinting method, addresses black-box settings where only query-level access is available, and needs substantially fewer extraction samples than LLMmap for that kind of instance-level check.

C2PA as the file-level provenance layer that works without a secret key

C2PA takes a completely different approach to the same underlying question, and it applies to files rather than word choices. It's a cryptographic provenance standard built on public-key signing, and anyone can verify a C2PA credential: no secret key, no private detector access, nothing gated behind a vendor relationship. Think of it as a signed receipt stapled to a file, rather than a forensic signal buried inside the words themselves. Detection tools built around C2PA return a straightforward verdict, TRUSTED, UNTRUSTED, or TAMPERED, and they check it in real time against images, video, audio, and PDFs.

Adoption is real but patchy. OpenAI, Adobe Firefly, and Google Imagen have all implemented C2PA on the generation side. Google's Pixel 10 signs every photo it takes natively, while Samsung's Galaxy S25 only signs AI-edited photos natively, a narrower commitment by comparison. On the distribution side, LinkedIn, TikTok, and Cloudflare all support or preserve credentials at scale.

Even where C2PA gets applied at the point of generation, a practical problem undercuts it downstream. Most platforms strip embedded metadata during their normal processing pipelines, so a C2PA manifest that left the generator intact often arrives at the eventual viewer carrying no credential at all. That is not deliberate suppression, it is a side effect of ordinary transcoding and compression steps that were never built with provenance metadata in mind. A buyer checking a downloaded image for C2PA credentials may find nothing there even when the original file was signed, simply because the file passed through a platform that quietly drops metadata on the way through.

The regulatory floor buyers are now operating above

The rules are catching up unevenly, and that unevenness is itself useful information for anyone drafting a vendor contract. The EU AI Act's Article 50 became fully enforceable on August 2, 2026, and it requires providers and deployers of generative AI systems to mark synthetic output in a machine-readable format, alongside a broader set of transparency and disclosure duties. Anthropic's own rollout of Claude watermarking was timed to that exact date, which says something about how directly providers now read enforcement deadlines into their product calendars. California's AI Transparency Act entered force the same month. China moved earlier on paper, announcing AI watermarking requirements back in 2022, though the rules did not actually roll out until September 2025.

Biden's Executive Order 14110 identified watermarking as a policy goal, but Trump rescinded it in January 2025. The AI Labeling Act, first introduced in 2023, got reintroduced in 2026 and still has not become law. Washington state passed its own measure, HB 1170, but no federal mandate is in force. And every one of these rules, without exception, targets image, video, and audio output. None of them currently reach plain text, and that gap is exactly where this entire piece sits. Buyers commissioning written content operate in a space where no law requires a watermark and no law requires disclosure. Whatever verification happens has to get built into the contract voluntarily, because no regulator is showing up to backstop it anytime soon.

A practical z-score screening workflow content teams can run today

Vendor onboarding is the moment to build a baseline, before a single dollar changes hands on ongoing production. Ask any new vendor for raw, unedited samples of model output, long enough to carry a real signal, roughly 100 words or more given how the z-score scales with length. Run those samples through a detector covering multiple scheme families at once, kgw, synthid, exp, unigram, rather than betting on just one, since a vendor's actual model won't announce itself and a single-scheme detector will miss anything outside its own family. Whatever comes back, a clean spike, a flat zero, or a "TOO SHORT TO TELL" verdict, gets written down and filed as that vendor's baseline signal profile, tied to the specific claim they made about which model they use.

That baseline earns its keep later. Come back to it every few months with a fresh sample pulled from live deliverables, and compare. None of this replaces judgment, and none of it catches every evasion, the paraphrase and rewrite numbers above make that plain enough. But it turns a vague worry about provenance into a number a buyer can point to, sitting in a file, dated, ready to back up whatever conversation comes next. Across nearly 20 million Gemini responses, watermarked and non-watermarked outputs showed no significant difference in user thumbs-up or thumbs-down rates, so the signal costs nothing in output quality that anyone can actually notice https://en.wikipedia.org/wiki/Text_watermarking. Seven AI detectors showed a 61.3% false-positive rate when tested on 91 TOEFL essays written by non-native English speakers https://dev.to/sanjay_singh_1/ai-text-watermarking-the-statistics-hiding-inside-every-sentence-2g9f.