Limits of Current LLM Fingerprinting Against Multi-Model Text

This article examines the limits of current LLM fingerprinting against multi-model text. Its central conclusion is structural. A detector built to find one source’s repeatable statistical signature cannot establish provenance when one model after another replaces the token choices that carry it. Multi-model workflows therefore require an approach to provenance that accounts for compound authorship, rather than treating a detectable watermark as a durable record of origin.

How KGW watermarks work and what they are built to detect

Kirchenbauer et al. introduced a watermarking scheme that divides the vocabulary into green and red subsets using a secret key. During generation, the model applies a positive logit bias to green tokens, increasing their probability of selection. The green list is therefore a preferred set, with selection still open to tokens outside it. Across a passage, that preference leads to a statistical excess of green tokens.

Detection measures that excess using a one-proportion z-statistic on the fraction of green tokens in the sample. The detector flags the text when its z-score exceeds a threshold τ. The test does not reconstruct how the document was written or which parts were later edited.

Google’s SynthID-Text illustrates how generation-time watermarking has reached production. Open-sourced on GitHub, integrated into Hugging Face Transformers, and integrated into Gemini output for every user since May 2024, it uses a pseudo-random function applied to the generation context to adjust token probability scores. Those adjustments are intended to be invisible to humans but algorithmically detectable. For KGW, the structural assumption remains that the signal accumulates reliably when a single consistent source applies its bias across the full text.

Why factual and low-entropy text already weakens the signal from one model

Factual text often leaves little freedom to choose among tokens without changing the answer. Google’s documentation acknowledges that watermark application is less effective on factual responses because there is less opportunity to alter generation without decreasing accuracy. The watermark must compete with the task the model is meant to perform.

The Invisible Entropy paper formalizes this problem in low-entropy settings, where the next token is highly predictable. A code import statement is one example. If the required token falls in the red list, avoiding it may disrupt fluency, while selecting it weakens the watermark. If it falls in the green list, its predictability can inflate the green-token count and raise false positive risk. The count then reflects a constraint of the content.

A stronger preference makes the signal easier to detect, but it can also increase text distortion. The robustness-quality trade-off is therefore an information-theoretic constraint. Within one model’s output, creative passages offer more room to carry the signal than factual or structured passages.

How targeted rewriting attacks exploit the z-score's statistical brittleness

A later model can retain what a passage says while replacing the choices that made its watermark detectable. This separates semantic fidelity from statistical continuity. The rewritten passage need not become incoherent for the original detector to lose its signal. It only needs to have less of the expected green-token excess.

Watermark evasion research distinguishes query-based attacks from query-free attacks. Query-based attacks infer the green-token set through many crafted prefix prompts, requiring access to the watermarked model and partial knowledge of the scheme. Query-free attacks instead rewrite the text that the model has produced in a black-box setting. Their significance is that access to the watermark’s secret key is not a necessary condition for removing its detectable effect. The text itself can be enough.

The BIRA paper gives this weakness a theoretical basis. Reducing the average conditional probability of sampling green tokens by a small margin δ causes detection probability to decay exponentially in δ². BIRA, the Bias-Inversion Rewriting Attack, applies a negative logit bias to a proxy suppression set identified through token surprisal. It requires a rewriting model and the generated text.

BIRA reports evasion rates above 99% across diverse watermarking schemes while preserving semantic fidelity substantially better than prior query-free baselines. On KGW-watermarked text from a large Qwen2.5 model, it reduced z-scores from roughly 6 to below 1. Earlier query-free methods had achieved only limited evasion or had severely distorted meaning. These results show that apparent resistance to rewriting was not a fundamental security property of the watermark.

Why multi-model pipelines dissolve the watermark signal by construction

An ordinary multi-model pipeline introduces the same kind of discontinuity without requiring an evasion objective. A draft model selects tokens, a rewriting model replaces them, and a style-enforcement model may replace them again. Each stage generates from its own distribution rather than preserving the originating model’s green-list preference. Token-level authorship is thus spread across the providers and the steps that generate the text. The final document is not the uninterrupted output that a single-source detector assumes.

KGW signatures are model-specific and session-specific. Passage through another model does not carry a guarantee that the original signature will survive, because the second model applies its own token-selection logic. The first model’s bias can be diluted, overwritten, or misattributed at each handoff. BIRA demonstrates how sharply detection can fall under targeted suppression, although its measured evasion rates should not be treated as measurements of ordinary pipelines. The point about how pipelines work holds even if each rewrite lowers the score by a different amount.

A surviving signal would identify statistical traces of a source. Conversely, a missing signal would not establish that the document lacked AI authorship. No deployed standard currently tracks provenance through these chains and re-establishes the watermark’s chain of custody at each handoff. A detector for one source cannot supply that missing continuity.

The December 2025 EU draft Code of Practice proposes a multilayered approach combining metadata embedding, imperceptible watermarks, and fingerprinting/logging, with detection tools as a separate commitment. Its framework nevertheless assumes a single originating model. PAN 2026 at CLEF 2026, continuing the Voight-Kampff Generative AI Detection task with ELOQUENT Lab, introduces a Text Watermarking task focused on watermarking existing text. That benchmark recognizes post-hoc marking of already-generated and already-edited text as an unsolved capability. It does not make the original generation-time signature persistent through later edits.

Why stylometric attribution also collapses on multi-model text

Stylometry appears to offer an alternative because it looks for characteristics of writing. Its attribution logic, however, also depends on sources having stable, distinguishable fingerprints. A multi-model document complicates that premise because its final style reflects successive transformations. After several models have acted on a passage, attribution depends on whether the remaining features can still distinguish their contributions.

Przystalski et al. (2025) report strong results in single-domain, single-model attribution, including seven-class attribution and balanced binary comparisons of human texts and GPT-4 outputs. They did worst when they compared model families that were closely related. Accuracy fell to only 52–56% for comparisons such as Orca vs. LLaMA 2 and LLaMA 3 vs. GPT-4. These results show that separability can approach chance even before a document has passed through multiple models. Strong performance on a clean attribution task does not establish reliability on compound authorship.

Sequential rewriting compounds that weakness by narrowing the stylistic separation between pipeline stages. Style mimicry and authorship obfuscation further erode the premise that a stable fingerprint survives in the final text. As architectures converge across providers, distinctions between model families are expected to narrow further.

Current classifier-based, stylometric, and watermarking techniques carry elevated false positive rates for non-native English writers. Detection errors are therefore not distributed evenly across writers. That limitation makes it especially important not to treat an uncertain attribution as a complete account of authorship.

The live legal exposure that converts a research problem into an operational one

The transparency obligations under EU AI Act Article 50 took effect on August 2, 2026. Providers of generative AI systems in the EU must mark synthetic outputs in a machine-readable format, while deployers carry separate human-facing disclosure duties. A technical assessment of watermark persistence must not collapse them into a single detector result.

Violations carry fines scaling into the tens of millions of euros or a percentage of global annual turnover, whichever is higher. The operational exposure arises when an originating system marks an output but intermediate rewriting removes the signal before publication. A pipeline can then contain an originally watermarked draft without retaining that marking in its final text. The presence of a watermark at the first stage is not evidence of its survival through the workflow. No current tool re-embeds provenance at every handoff in such a chain.

The December 2025 EU draft Code of Practice’s multilayered approach does not make a single originating mark sufficient for a compound workflow. The technical fragility now sits beside a legal obligation.

What a structurally sound response to multi-model provenance looks like

A sound approach must account for multi-model authorship when the system is designed. Dispersing token-level authorship across providers explains why a single model’s green-list signature cannot be relied on to accumulate throughout the output. It does not, on its own, satisfy a duty to mark synthetic content or disclose its use. Undetectability and compliance are different claims.

Post-hoc paraphrasing and single-model re-embedding do not restore the history of a multi-model document. BIRA and knowledge distillation removal results demonstrate that rewriting can destroy a watermark signal. Watermarking with Low-Entropy POS-Guided Token Partitioning, presented at EMNLP 2025, reports a 24.53% improvement in semantic fidelity measured by RC-PPL, but that improvement comes at the cost of detection sensitivity.

Behavioral fingerprinting through refusal vectors offers a different research direction. The 2026 approach identifies models through internal activation-space representations. It is still a research-stage method and has not been used in production. It also assumes an attributable model identity, leaving compound authorship unresolved. A different fingerprint does not automatically provide a chain of custody.

For claims about KGW detectability, the practical test is quantitative measurement against real KGW detectors. Z-scores measure the actual detection criterion, whereas paraphrasing depth and word-change counts do not. A low score demonstrates the absence of a detectable signature under that test. Content teams with clear briefs, approved outlines, and drafts they can rely on should not have to give up those workflows. The architecture should fit their existing handoffs while treating provenance across those handoffs as the problem to solve, rather than mistaking the disappearance of a fingerprint for its solution.