Quality Degradation Risk When Optimizing for Watermark Z-Score

Optimizing a watermark z-score is not a neutral way to improve generated content. In token-level watermarking, the same bias that strengthens detection also distorts the text. Forcing token changes to lower the score can also hurt quality. The structural alternative is dispersed multi-model authorship, which changes how the signal forms rather than forcing one model’s output away from its preferred language.

How KGW watermarks work and why the z-score is the thing detectors measure

KGW watermarking splits the token vocabulary into green and red lists using a hash of the previous token. During generation, green tokens receive a logit boost controlled by δ, making them more likely to be selected. The detector then measures their frequency in the finished text. It compares the green-token count in the text with the count expected at the baseline green-token rate, γ.

The detection statistic is z = (|S|_G − γ|T|) / √(|T|γ(1−γ)), where |S|_G is the green-token count and |T| is the text length. If z exceeds the threshold ẑ, the detector classifies the text as watermarked. The higher the proportion of green tokens, the higher the score. The score falls as that proportion approaches γ. AI Watermark Evidence Fails Forensic Readiness: An Empirical Evaluation examines KGW, Unigram, and SynthID, placing this statistical detection approach within a broader evaluation of watermark evidence.

Why δ is not a free parameter: raising it to strengthen detection also raises perplexity

Increasing δ strengthens the detection signal and increases distortion because both effects come from the same intervention. The model is pushed toward green tokens even if it would have preferred other tokens before watermarking. The two outcomes are not independently adjustable. A larger reference model registers that departure as higher perplexity, so the quality penalty is a direct consequence of changing the token distribution.

The reported KGW results show perplexity climbing step by step as δ increases from 1 through 3. Watermarking with Low-Entropy POS-Guided Token Partitioning and Z-Score-Driven Dynamic Bias for Large Language Models, published at EMNLP 2025, identifies the semantic cost as a central limitation of existing methods. The paper states that they “reduce the fidelity of semantics” because they fail to distinguish positions that tolerate bias from positions that do not.

The broader results also weaken the case for simply increasing watermark strength. KGW, Unigram, and SIR exhibit perplexity increases of two to three times compared with the default δ, while gaining only marginal robustness improvements. Lowering δ weakens the embedded signal, but that does not show that arbitrary token changes to lower a finished text’s z-score will keep its quality intact.

Where the quality cost is worst: low-entropy token positions

The most vulnerable positions are those where the model has little freedom to choose. In low-entropy contexts, one token is far more probable than the alternatives, so a green-token boost may leave the output unchanged. Forcing a different choice creates the opposite problem. It can replace the right token precisely where there is little room for substitution. The GSM8K math example makes this concrete, with an unwatermarked model producing a correct answer and KGW-induced sampling at key positions producing a wrong one.

The ACL 2026 SSG paper formalizes this bottleneck through the lower bound of watermark strength. With random vocabulary partitioning, that bound depends on the next-token probability distribution. In mathematical reasoning and code generation, most high-logit tokens may fall into the same subset, leaving near-zero watermark strength at tightly constrained positions. With KGW, such tokens make up a much larger share. They do little to help detection and occur where forcing a green choice is especially risky.

The POS-guided paper identifies the same mismatch in linguistic terms. KGW “overlooks the determinacy disparities among different parts of speech,” applying equal pressure where choices are abundant and where they are narrow. Proper nouns and grammatical particles illustrate the constrained case. A bias that is tolerable at an open-ended wording choice can break semantics at those positions. A document-wide fluency measure therefore cannot fully describe the damage.

Task accuracy and alignment failures that perplexity scores do not capture

Perplexity measures an aspect of quality distinct from answer correctness and the survival of safety behavior. The math failure therefore represents more than awkward wording. It shows that δ-driven logit distortion can change the result of a task even when the unwatermarked model succeeds. The reported downstream classification results extend that concern, with accuracy losses ranging from ten to twenty percentage points.

The alignment example is more consequential still. An LLaMA-Instruct model using KGW with δ=2 and γ=0.25 generates unsafe content in a case where its unwatermarked form correctly refuses the harmful request. The watermark changes the behavior being evaluated. This example demonstrates that the intervention can cross a boundary that aggregate perplexity does not measure.

A configuration cannot be judged solely by how reliably its text exceeds a detection threshold. Its task accuracy and alignment behavior are separate outcomes. A stronger statistical signature offers no compensation for a wrong answer or a lost refusal, because the detector evaluates token patterns and leaves the validity of the response unassessed.

The robustness paradox: watermarks that preserve quality are the ones that collapse under attack

A loss in quality would be easier to accept if it made attribution last. The forensic evaluation instead finds substantial fragility under paraphrase. Across hundreds of valid runs, every initially detected KGW and Unigram text lost its watermark after paraphrasing, producing a conditional removal rate of 100%. SynthID performed only marginally better. High false-negative rates across all three methods were already present before any attack.

The evaluation also finds that methods with weaker evasion score highest on quality, while the strongest evasion method loses enough fidelity to show up in semantic similarity and judged quality. A pipeline that watermarks text and then paraphrases it to beat detection can therefore lose quality twice, first through generation bias and then through the rewriting pass.

ICML 2025 robustness work defines the desired security property as requiring that evasion cause “(significant) degradation of the content’s quality,” without guaranteeing that result. The same work notes that most proposed methods are tested against non-adaptive attackers who lack knowledge of the watermarking method, while its adaptive attacks evade all surveyed watermarks. Performance against an uninformed attacker does not demonstrate that a signal remains dependable when the method is known.

Why researchers are trying to escape the tradeoff at the token level — and what those attempts reveal

SWEET (Lee et al., 2024) selectively watermarks high-entropy positions, while EWD (Lu et al., 2024) weights tokens by entropy in the detection statistic. Both approaches reduce the influence of constrained positions. However, their need for model logits at detection time prevents public verification, exchanging part of the distortion problem for an access requirement.

STELA replaces model-specific entropy signals with corpus-derived part-of-speech n-gram entropy. It weakens the signal in grammatically constrained contexts and strengthens it where linguistic choices are abundant. This permits public verification without logit access. The POS-guided framework pursues a related goal through low-entropy token partitioning and z-score-driven dynamic bias, reporting a 24.53% improvement in semantic fidelity, measured by RC-PPL, over KGW, with a modest robustness improvement. These are meaningful gains achieved through more selective control within one model’s token distribution.

SSG replaces random token assignment with logit-balanced vocabulary partitioning that raises the lower bound of watermark strength at each prediction, addressing near-zero-strength positions without compromising text quality in the reported work. That is a genuine technical advance. Whether those improvements jointly deliver quality, robust detection, and usable verification remains unresolved.

ICLR 2026 fingerprinting work points toward a different division of responsibilities. It maintains a fingerprint success rate of 1.0 under adversarial system prompts where baselines “almost systematically fail.” Model identity becomes a distinct tracing problem. That result supports separating model-identity tracing from the effort to preserve content quality while embedding a token-level signal.

What the compliance and provenance frameworks actually require — and why z-score optimization cannot deliver it

The EU AI Act requires markings that are “sufficiently reliable, interoperable, effective and robust.” California’s SB 942 requires disclosure that is “permanent or extraordinarily difficult to remove.” Neither standard is expressed as an initial detection score. Both demand properties that must survive beyond the generation step, making removability and reliability operational concerns.

For content teams, the gap is between producing a signal and demonstrating dependable provenance. Raising δ can improve the first while degrading the text, without establishing the second. The paraphrase findings make that gap concrete. A marking that disappears from initially detected text under the tested attack does not gain permanence simply because its original z-score was higher. Optimizing the statistic therefore cannot substitute for evaluating the property the framework requires.

The forensic evaluation adds a narrower evidentiary warning. None of its three tested methods satisfies more than two of five Daubert admissibility factors, which undermines their readiness as forensic evidence. That finding is limited to the tested methods. Still, the finding shows why crossing a threshold is not enough to support a legal attribution claim. Content quality, marking durability, and evidentiary reliability each have to be met, even when one detector reports them through a single classification.

Why the structural fix is dispersed multi-model authorship rather than further parameter tuning

Dispersed multi-model authorship changes the generation architecture instead of increasing or counteracting one model’s token bias. Within KGW, the signal and distortion originate in the same biased distribution. Changing that relationship requires a different intervention. When authorship is genuinely dispersed across providers and mixed at the token level, no single model’s distribution dominates the passage in the way assumed by a coherent watermark signal.

A passage drafted by one model, extended by another, and refined by a third may consequently have no reliable z-score attributable to any of them. The distinction is how that outcome arises. Paraphrase-based evasion works on a signal already embedded in text and loses fidelity when the rewriting it needs changes meaning or expression. Dispersed generation instead avoids concentrating one signal in the first place, so it need not impose a separate quality penalty to remove that signal afterward.

That architectural argument is not a guarantee of flawless output. Voice consistency across a long document remains the practical test, and mixing models can dissolve coherence if the pipeline does not control their contributions. Selection, judge panels, and enhance passes address that engineering problem. They operate on the quality of the combined passage rather than demanding that one token distribution serve competing goals through a single δ setting. Nor does the absence of a coherent watermark itself supply the reliable provenance required by compliance frameworks.

The conclusion is therefore architectural, with a clear boundary. Further parameter tuning can improve particular watermark schemes, but it cannot turn the z-score into a measure of correctness, alignment, or durable attribution. The target is coherent, high-quality content with a defensible account of its origin.