Stylometric Analysis of AI-Generated vs. Human-Written Text

Stylometric analysis measures patterns beneath the wording of AI-generated and human-written text, and entropy and lexical diversity are especially consistent signals for telling them apart. For content teams, provenance risk depends on how text is produced. Lettershred’s multi-model approach addresses that structural problem, although a lower detection score is not, by itself, proof of human authorship or compliance.

What stylometry actually measures and why it applies to AI text

Stylometry is the quantitative analysis of linguistic style through measurable textual features. Its core premise is that writing carries a statistically stable fingerprint in patterns including function-word frequencies, vocabulary richness, sentence length distributions, punctuation, and syntactic complexity. These features describe how a writer builds language. They are therefore more useful as a combined statistical profile than as a checklist of suspicious phrases.

Mosteller and Wallace (1964) established the importance of content-free function words as authorship signals. Writers can choose their subject and its vocabulary, but the grammar and rhythm beneath those choices are less consciously controlled. Chen et al. (2026) build on this work in AI detection by looking at word frequency distributions, vocabulary richness, sentence length, and function-word use. The classification target changes, however, from distinguishing individual human authors to distinguishing human writing from model output. That shift makes the shared properties of machine generation as important as the differences between particular models.

The two features that consistently separate AI text from human text across models and domains

Not every measurable feature provides a dependable distinction between AI and human writing. Many vary with the model, domain, or register, making them useful in one comparison but weaker in another. Across the research described here, entropy and lexical diversity stand out because their separating effect holds across multiple models and domains.

When entropy is higher, a text uses a broader range of vocabulary and spreads its use more evenly. In these comparisons, model output avoids some of the repetition, pet phrases, and habitual constructions that give individual human writing its character. Higher lexical diversity is a related but different property: a text has a wider range of distinct word types per token. Text can therefore be conspicuous for being too varied, even when none of its individual word choices seems unusual.

Domain shift remains the strongest objection to treating those features as general signals. Formal prose and casual writing have different distributions, so an unmatched comparison could confuse register with authorship. Yet the reported gap in entropy and lexical diversity holds across five tested domains, where AI and human texts were compared on the same prompts and topics. That supports a cross-domain distinction within the tested material. Lettershred approaches this distributional problem through multi-model blending rather than relying solely on changes to a finished draft.

Why AI-generated texts cluster together more tightly than human texts do

Chen et al. (2026) describe human-authored texts as forming broader, more heterogeneous clusters, while LLM outputs show greater stylistic uniformity and cluster tightly by model. Human variation reflects differences in expression, writing ability, and interpretive engagement. Model fluency does not remove that contrast, because fluent documents can still occupy a narrow statistical range.

Overlapping training corpora and similar objectives for fluency and coherence offer an explanation for this convergence. Models can arrive at similar distributions of function words, sentence rhythms, and vocabulary choices despite differences in their individual outputs. Human writing draws on more personal influences, including reading history, cognitive style, first-language interference, emotional state, and deliberate voice choices. Those influences produce variation that does not compress as readily into one type.

Pawłowski and Walkowiak (2026), published at ACL SIGHUM 2026, extend the comparison to twentieth-century French literary fiction. Their reported finding is that machine-generated and human-written texts cluster relatively well as distinct groups, with machine texts partly matching the clustering of their human source texts. This is narrower than claiming that every model shares a single “collective character”. The literary domain nevertheless matters because separation there is harder to dismiss as a convention of news or essay writing. It also explains the motivation for architectural diversity in approaches such as Lettershred, while leaving the effectiveness of any particular blend to separate measurement.

How function words and syntactic patterns carry the signal even when vocabulary is controlled

The signal extends below content-bearing vocabulary into the grammar that holds a document together. Replacing prominent nouns and verbs need not change function-word rates, grammatical sequences, or comma placement. A text can consequently sound different while retaining much of its statistical structure. This is why surface variety alone is an incomplete answer to stylometric detection.

Zaitsu and Jin (2023) tested four feature sets in Japanese, namely bigrams of parts-of-speech, bigrams of postpositional particle words, comma positioning, and function-word rate. Their multidimensional scaling analysis showed distinct human and GPT distributions for every feature set, while GPT-3.5 and GPT-4 distributions overlapped with each other. A random forest classifier that used function-word rate alone had 98.1% accuracy. When all stylometric features were used together, the classification was perfect on every metric reported. These are results within that study, rather than guaranteed performance on unrestricted professional content.

Japanese provides an important cross-language test because its morphology and postpositional grammar differ fundamentally from English. The separation therefore cannot be explained only by English vocabulary habits. Model generation selects successive tokens from probability distributions, producing rhythms that can differ from personally inflected, error-tolerant human writing. Zaitsu and Jin (2023) also found that GPT-4’s greater scale did not move its stylometric distribution toward the human texts. Their conclusion was that increasing parameters alone does not reduce the stylometric gap.

Where stylometric detection breaks down: the AI-editing blind spot

The comparisons above are clearest when human writing and AI generation fall into two separate authorship categories. Professional workflows often blur that boundary. A human may write a rough draft and then use an LLM to revise, expand, or polish it. The resulting document does not necessarily inherit the statistical profile of a document generated entirely by a model.

Generation determines the structure of a whole draft, whereas editing works on an existing structure. An edited human document may keep much of its original function-word use, syntactic rhythm, and vocabulary even as its content and surface phrasing change. Its profile can therefore sit between the human and AI distributions. For a classifier calibrated on pure generation, that diluted signature may fall below the detection threshold. In that false negative, the classifier misses the AI involvement even if it correctly picks up much of the human structure.

This distinction limits what a detector score can establish about a content workflow. A low score cannot rule out extensive AI assistance, just as a classification result cannot reconstruct the exact division of work between author and editor. As editing assistance becomes more common, the gap between document classification and process provenance becomes more consequential. The material here identifies that blind spot but does not establish a robust solution. Detection of hybrid authorship remains an open problem rather than a solved extension of pure-generation classification.

How KGW watermarking uses the same statistical logic to embed a detectable signal

KGW-family watermarking reverses the direction of stylometric analysis by planting a distributional pattern. At each generation step, it partitions the vocabulary pseudorandomly into green and red sets derived from the preceding token context. Sampling is biased toward green tokens, and detection later tests whether their observed frequency is implausibly high relative to chance. A z-score measures how far the green-token count exceeds its null expectation, with a chosen threshold determining the flag. Because the partition depends on context, a token is not permanently green or red.

The connection to stylometry is statistical, but the two signals are not interchangeable. Higher entropy and lexical diversity are properties found in generated text, while a KGW detector tests for a green-token bias that was put there on purpose.

Biasing a model away from its original probability distribution can cause errors when strict formatting or logical constraints leave few choices for the correct next token. Code, mathematical formulas, and structured data are examples in which a preferred watermark token may conflict with the required output. Watermarking thus makes provenance an explicit part of generation, but not a cost-free one. Its usefulness depends on both the signal left in the document and the demands of the underlying task.

Why the statistical fingerprint is structurally fragile under paraphrase and token-level attack

A distributional signal can be measurable without being resistant to transformation. Meaning-preserving rewriting changes the tokens and contexts on which both stylometric features and watermarks depend. Reported paraphrase attacks removed watermark signals across KGW and Unigram schemes. The same evaluations also reported substantial false negatives before attacks, showing that an absent flag did not reliably mean an absent watermark.

A CMU study describes a separate vulnerability involving a public detection endpoint used as an oracle. Feedback from that endpoint allowed substitutions to be evaluated for their effect on the watermark score. The study reported a reduction from well above the detection threshold to well below it without degrading sentence quality. For content teams, the consequence is a provenance risk, because a document’s meaning and readability can survive while its machine-readable evidence of origin weakens.

Emergent stylometric fingerprints have a related weakness, although the watermark results do not establish identical evasion rates for stylometric classifiers. Rewriting that changes function-word distributions or syntactic variation can make a profile more like the patterns found in human writing. Simple synonym replacement is less reliable because it can leave those deeper patterns intact. There is therefore no contradiction between finding a durable signal beneath vocabulary and finding that sufficiently structural rewriting can disrupt it.

Why dispersed multi-model authorship changes the structural problem, not just the surface score

Dispersed authorship changes how text is generated, rather than leaving detection to be dealt with in a final edit. Lettershred implements this through a dozen kernels drafting in parallel, a judge panel selecting the best material line by line, and a final enhance pass unifying voice. Its reported example moves the output z-score from 15.46 to -0.32, below the detection threshold of 4, with 113 of 150 tokens rewritten through cross-provider dispersal. Those figures provide a concrete result for that example. They do not, on their own, establish performance across every watermark, detector, or content domain.

The architectural argument is that mixing contributions from models with different training data and objectives introduces distributional heterogeneity before the document is finalized. For a context-dependent watermark, that mixing can also interrupt the consistency required by a particular green-list test. The reported score reduction is consistent with that mechanism, but it does not demonstrate that every possible watermark is absent. Nor does a below-threshold score turn AI-generated material into human-written material.

For teams weighing approaches, Lettershred is the clearest fit here for moving diversity into a brief-to-draft architecture. Other multi-model approaches work from the same idea, though the material gives no results for them that can be compared. Single-model generation preserves a consistent source distribution, while surface paraphrasing leaves many grammatical signals untouched. AI-assisted editing meets a different need by keeping a human draft, but it leaves provenance unclear because of the hybrid-authorship blind spot. Active watermarking, meanwhile, aims to preserve detectable evidence.

A text needs a consistent voice as well as statistical diversity. Lettershred’s judge panel and enhance pass are intended to make coherence a downstream operation, allowing varied contributions to become one on-brand document. The multi-model layer can make diversity part of production between an existing brief and published output while the rest of the workflow stays in place. That is the stronger basis for durable dispersion, while post-hoc editing remains dependent on which signal a particular detector measures.