Guide

Stylometry and authorship analysis: what it can and cannot show

Stylometry is the oldest quantitative method for asking who wrote a text, and it is now being asked a new question: whether a person wrote a text at all, or a machine did. This guide explains what stylometry measures, where it has worked, how it differs from AI detectors, and why it cannot settle questions of AI use on its own.

What does stylometry measure?

Stylometry treats writing style as something that can be counted. The features most often used are ones writers rarely notice they are choosing:

• Function words. How often a writer uses words such as "the", "of", "upon" or "also". These carry little meaning, so they vary less with topic than with the person writing.
• Word pairs and character sequences. Recurring two-word combinations and short runs of letters, which capture habits such as favoured word stems.
• Word and sentence length. The distribution of lengths, rather than the average alone.
• Punctuation habits. Use of semicolons, dashes, commas and similar marks.

Patrick Juola, who carried out a widely reported 2013 analysis, explained the idea using the preposition people choose for where a fork goes on a table: "on", "to" or "at" the left. "There's a lot of different right answers," he said, "that people don't even notice they're making" (NPR, 2013). Small differences across the 50 or 100 most frequent words add up to a profile that can be compared with other documents.

The task of deciding whether two texts were written by the same person is called authorship verification. The PAN research workshops have run shared evaluations of it in several years since 2013, defining it as "deciding whether two texts have been written by the same author based on comparing the texts' writing styles" (PAN at CLEF 2023). A survey of the field is Juola, Authorship Attribution, in Foundations and Trends in Information Retrieval.

Where has stylometry worked?

The Federalist Papers. Twelve of the essays published in 1787 and 1788 to argue for the US Constitution were claimed by both Alexander Hamilton and James Madison. In a study first published in the Journal of the American Statistical Association in 1963 and expanded into a 1964 book, statisticians Frederick Mosteller and David Wallace compared the frequencies of common words in the essays of known authorship and concluded that Madison was very likely the author of the disputed papers (History of Information). Their work is also regarded as the first full-scale applied statistical analysis using Bayesian methods.

Robert Galbraith. In 2013 a Sunday Times reporter asked Juola to examine The Cuckoo's Calling, a detective novel published under the name Robert Galbraith. His tests of common words, word pairs, character sequences and word lengths pointed to J.K. Rowling as the most likely author among those compared. Juola was careful about what that showed: it was "either Rowling, or somebody who writes surprisingly like Rowling" (NPR, 2013). Rowling then confirmed she was the author.

Both cases share the conditions stylometry needs: a small, known set of candidate authors and plenty of verified writing from each.

How is stylometry different from an AI detector?

The two methods answer different questions. An AI detector, such as the free Walter Writes AI detector, compares a text with patterns that are common in machine-generated writing. Stylometry compares a text with the known writing of a specific person.

The methods can support each other. A detector flag on a text that also differs sharply from the author's verified writing is stronger evidence than either signal alone, which is how the AFIP evidence-weight scale treats agreement between independent methods.

What are the limits of stylometry?

• Sample size. Maciej Eder's experiments on English and Polish novels first suggested that a sample needs at least 5,000 running words to be attributable, then found that in many cases less than 2,000 words may be enough, though some authors' style is so faint that much longer samples are needed (Eder, 2017). A single email or short essay is well below these figures.
• Style is not fixed. A person writes differently in a report, an email and a speech. PAN's evaluations deliberately test pairs of texts of different types because that is harder (PAN at CLEF 2023).
• A difference is not an explanation. A text unlike someone's usual style may have been edited by a colleague, written in a hurry, adapted from a template or produced with AI. Stylometry cannot tell these apart.
• Candidates matter. Attribution picks the closest match among the authors compared. If the real author is not in the comparison set, the method will still name someone.
• Disguise. Writers who know they may be analysed can change their habits, and AI tools can rewrite text in a different style.

Stylometry is best used as one item of evidence alongside drafts, version history and file metadata. AFIP's research on document forensics covers the metadata side, and its methodology explains how AFIP weighs different kinds of evidence.

A worked comparison

AFIP is preparing a worked comparison using two public-domain texts of known authorship from the Federalist Papers: an essay accepted as Madison's and one accepted as Hamilton's. The comparison counts the frequencies of the same set of common function words in each essay and in a third essay, then shows which known author the third essay sits closer to and how the result changes as the sample is shortened. Results will be published here once the analysis has been run and checked.

For related guidance, see why AI detector false positives add up at scale, how to tell whether text was written by AI, the glossary entry on stylometry, or browse all AFIP verification guides.

Frequently asked questions