How AI Detectors Work: Perplexity, Classifiers and Watermarks

Most of the available explainers for AI detectors will tell you that they measure perplexity and burstiness. GPTZero, the tool that popularized these terms, announced on their website that they stopped using these metrics in autumn 2023.

That gap is the whole problem with the question. "How does an AI detector work" has had three different answers since 2022 and the industry has moved through all of them, yet the public explanation is still stuck on the first.

Here are the three generations, what each one measures and what it means when your name appears on the score.

Generation one: perplexity and burstiness

The first detectors did not classify anything. Instead, they scored the text against a language model and asked how surprised that model was.

GPTZero defines perplexity as "a measure of how likely an AI model would have chosen the exact same set of words as found in the document". For example, if the sentence fragment is "Hi there, I am an AI ___", an AI model would choose "assistant" which would have a low perplexity. However, if an AI model chose "potato" instead, the perplexity would be high, which is taken as evidence a person wrote it. The threshold for this is set at a perplexity above 85, showing that it is more likely that a human wrote the sentence than an AI model.

The second half is burstiness. This measures how much the surprise varies within a document. Humans tend to write unevenly within a document. If the model is asked for five paragraphs, it will produce five paragraphs that are roughly of the same length. Burstiness is an attempt to measure this variable in writing style.

Both of these measures have an obvious failure mode, which is the reason for most of the false positives. A student writing in a second language, a scientist writing a methods section or anyone writing in a constrained register will all produce predictable, evenly shaped prose that results in low perplexity and low burstiness, and a machine verdict on human writing.

OpenAI built a detector on this generation's assumptions and killed it in public. The classifier was released on 31 January 2023 and correctly detected AI-written text in 26% of cases but also labelled human written text as AI written text in 9% of cases. On 20 July 2023 OpenAI withdrew the tool stating that the classifier was "no longer available due to its low rate of accuracy". Six months, start to finish.

The most interesting work in this generation came at the end of it. DetectGPT, a model created by Eric Mitchell and colleagues at Stanford in 2023, used the fact that GPT text lies in regions of negative curvature of the log-probability function of a model to instead measure the curvature of the function through perturbations. This led to an increase in the detection of fake news in GPT-NeoX from 0.81 to 0.95 AUROC, which is a genuine result that approaches the ceiling of AUROC for models that only look at probabilities.

Generation two: trained classifiers

What runs in production today is not a perplexity score. It is a model trained on a large corpus of human and machine text that learned to tell them apart but it does not know why.

GPTZero describes its current system as "a deep-learning based architecture that does not directly use perplexity and burstiness", with perplexity being one of seven indicators. Around the classifier, it has bolted a sentence-by-sentence scan of the text, an internet text search to see if the text already exists in archives and a "Paraphraser Shield" to detect text that has been through a rewriting tool.

Turnitin works the same way and says even less about it. Vanderbilt University's note on disabling the detector in 2023 stated that Turnitin would only describe the tool as looking for patterns common in AI writing, which is a description that fits any classifier ever built.

The second generation is measurably better than the first and has a specific, structural weakness: its confidence does not survive mixed authorship. According to Turnitin's own guidance, "in cases where we detect less than 20% of AI writing in a document, there is a higher incidence of false positives", which is why the product now prints an asterisk beside low scores. Turnitin puts its document-level false-positive rate below 1% only above that 20% line and its sentence-level rate at around 4%.

The February 2026 study by Hadra, Cambridge and Mesbah at Sultan Qaboos University put numbers on the same weakness from outside. Testing Turnitin and Originality.ai against 192 texts, the accuracy for each tool was 0.61 and 0.69 respectively and the recall for Originality.ai on hybrid human-AI compositions was close to zero. So, neither tool is reliable enough for high-stakes decisions.

So the generation that is actually deployed is good at text a model wrote in one pass but poor at text a person and a model made together, which is how most text now gets made.

Generation three: watermarks, which are not detection

The third answer abandons the premise altogether. Instead of inspecting the text after the fact, the model will mark its output at the moment of generation.

Kirchenbauer and colleagues at Maryland published the mechanism in 2023. Before each token, a secret key picks a pseudo-random "green list" of words. Sampling is nudged toward that list. No single choice looks odd. Across a few hundred tokens, the bias is measurable by anyone holding the key, with a statistical test that returns an interpretable p-value.

Google DeepMind took the idea to production and published the results in Nature in October 2024. The SynthID-Text algorithm uses tournament sampling to eliminate candidate tokens that are scored poorly by the watermarking functions. In a live evaluation of nearly 20 million Gemini responses, the thumbs-up rate differed by 0.01% between watermarked and unwatermarked output and the latency of the algorithm increased by 0.57%.

Anthropic adopted the same approach for Claude, announced on 14 August 2026. Two sentences from that announcement matter more than anything else written about detection this year:

Nothing is added to the text and there are no hidden characters.

and

Light editing probably won't remove the watermark completely; a complete rewrite where every word is replaced will.

The first eliminates the whole category of tools that scan for zero-width characters. The second is the most precise public statement anyone has made about what defeats a detection method.

The catch is access. Anthropic's detection API is in private preview and only available to regulators, law enforcement, media, fact-checkers, researchers, educational organisations and EU civil society groups. Your instructor almost certainly does not have access to it and neither does GPTZero.

The three generations, side by side

Three things can be deduced from this table. The generation that everyone explains is the one that nobody deploys. The generation that everyone is judged by is the one that cannot show its working. And the only generation with a mathematically checkable answer is the one that almost nobody can run.

Why two detectors disagree about the same paragraph

Because they are not measuring the same thing and because the published testing is genuinely split.

William Walters tested 16 detectors on 126 documents for Open Information Science in 2023 and found that Turnitin and Copyleaks correctly identified all of the documents. Weber-Wulff and colleagues compared 12 public tools and two commercial systems in 2023 and concluded that the available detectors are "neither accurate nor reliable", as they are biased towards considering machine-generated text as human-generated text and that their performance collapses if the text is paraphrased.

Both results are real. Walters used the clean, unedited output of the 2023 models. Weber-Wulff used obfuscated text. The difference between these two document sets is the difference between the two conclusions that can be drawn from each. Your essay resembles the second set far more than the first.

GPTZero, to its credit, states a limit rather than a guarantee of the tool's accuracy. No tool can honestly claim to be 100% accurate and detectors struggle with short, edited or mixed writing. This is the accurate summary of the whole field, from the vendor with the most to gain from saying otherwise.

The short version

Perplexity and burstiness were how the detectors worked in 2022 and the company that named them dropped them in 2023. What is used today is a classifier that returns a percentage but cannot explain it, which is accurate for clean machine output but weak for anything that a human has touched. The only method with a checkable answer is watermarking, which is now live in Gemini and Claude. The detectors that can read it are not in your instructor's hands. Any score that comes back on your work came from generation two and generation two is the one with the asterisk.

If you want the longer treatment:

• How Claude's watermark works, and what removes it — generation three in full
• How Turnitin's AI detection works — the classifier most students meet
• What AI detector do colleges actually use? — who has it switched on, and who turned it off
• Can AI detectors detect Claude, ChatGPT and Gemini? — model by model
• Are AI detectors accurate? Every published number, in one table — what generation two actually scores, vendor claims next to independent tests
• Our AI detector — free, and worth running before someone else does