Key takeaways
- No AI detector can prove authorship. Every one of them outputs a probability score, not evidence.
- Detectors mostly measure predictability. Clear, plain, well-structured writing scores as 'AI-like' because it is statistically unsurprising.
- Stanford researchers found detectors misclassified more than half of TOEFL essays written by non-native English speakers as AI-generated, while classifying native-speaker essays almost perfectly.
- OpenAI withdrew its own AI Text Classifier in July 2023 because its accuracy was too low to be useful.
- If you are accused, your process evidence — draft history, revision timestamps, notes — is far stronger than any counter-score.
What an AI detector actually measures
It helps to be precise about what these tools do, because the marketing language around them ('AI detection', 'originality check') implies something they cannot deliver.
An AI detector does not recognise text that came from ChatGPT. It has no record of what any model has produced. What it does is score your text on how statistically predictable it is, then convert that score into a percentage that looks like a verdict.
The two properties nearly every detector leans on:
- Perplexity — how surprising each word is given the words before it. Low perplexity means the text takes the expected path. Language models are trained to produce likely text, so their output tends to have low perplexity.
- Burstiness — how much sentence length and structure vary across a passage. Human writing tends to lurch between long and short sentences. Model output is often more even.
Both are real signals. Neither is proof. A human who writes in short, clear, consistent sentences produces exactly the pattern these tools are tuned to flag — and there is no feature in the text that distinguishes 'plain because the writer is careful' from 'plain because a model wrote it'.
A probability is not a finding
A detector reporting '92% AI' has not found anything. It is expressing a confidence level from a statistical model that can be — and demonstrably is — wrong about individual documents.
Why false positives are structurally unavoidable
False positives are not a bug that better engineering will remove. They follow from what is being measured.
Human and AI writing overlap
Language models were trained on human writing. They are good precisely because they reproduce patterns people actually use. So the distribution of 'text a model would write' sits on top of the distribution of 'text a person would write' — they are not two separate clouds with a clean line between them.
Any threshold drawn through overlapping distributions produces both false positives and false negatives. Moving the threshold trades one for the other; it never eliminates both.
Base rates make small error rates large
This is the part that is most often missed. A '1% false positive rate' sounds negligible until you apply it at scale.
Consider a course with 1,000 submitted essays where, say, 100 involved undisclosed AI use. A detector with a 1% false positive rate flags roughly 9 of the 900 genuine essays incorrectly. Those 9 students each face an accusation generated by arithmetic rather than by anything they did.
| Genuine essays | False positive rate | Students wrongly flagged |
|---|---|---|
| 900 | 1% | ~9 |
| 900 | 4% | ~36 |
| 900 | 10% | ~90 |
Scale that across a university term and a small percentage becomes a substantial number of real people defending work they did themselves.
The vendors say so themselves
Detector companies are typically more careful in their documentation than their marketing pages suggest.
Turnitin has stated a document-level false positive rate below 1% for its AI writing indicator, while also cautioning that documents flagged at 20% or less AI writing carry a higher risk of false positives — which is why it suppresses some low-percentage results. Its guidance is explicit that the score is a starting point for a conversation, not a determination of misconduct.
OpenAI is the sharpest example. It launched an AI Text Classifier in January 2023 and withdrew it that July, citing a low rate of accuracy. At launch the company reported the tool correctly identified only about 26% of AI-written text while incorrectly flagging about 9% of human-written text. The organisation with the most direct knowledge of how these models produce text could not build a reliable detector for their output.
Who gets hit hardest
False positives are not distributed evenly. They concentrate on people whose writing is, for entirely legitimate reasons, more regular than average.
- Non-native English writers. The most robustly documented group. Writing in a second language tends to use a narrower vocabulary and more consistent sentence construction — which reads to a detector as low perplexity.
- Anyone writing in a formulaic genre. Lab reports, legal summaries, clinical notes, technical documentation and standardised business correspondence are supposed to be predictable. Conventional structure is the goal, and it is also what gets flagged.
- Writers who have been taught to write plainly. Plain-language training, journalism style guides and accessibility standards all push toward shorter sentences and common words. That is good writing and low perplexity at the same time.
- Autistic writers and others with distinctive consistent styles. Any writer with a stable, systematic style produces low burstiness. Consistency is a stylistic trait, not evidence of automation.
- Anyone who used ordinary writing software. Grammar and spelling tools smooth text toward conventional phrasing. Running a spellchecker over your own work can move it toward the pattern a detector reads as machine-like.
The research finding that matters most
A 2023 Stanford study by Liang and colleagues, published in Patterns, ran seven widely used GPT detectors over TOEFL essays written by non-native English speakers and essays by US eighth-graders. The detectors classified the native-speaker essays almost perfectly, but misclassified more than half of the non-native essays as AI-generated. Notably, rewriting the non-native essays with more varied vocabulary substantially reduced the misclassification — which tells you the detectors were responding to language proficiency, not to authorship.
Related reading
- AI detector bias against non-native English writers — the research, and what it means if English is your second language
- AI humanizer for ESL writers — polish English you wrote yourself
What to do if you are wrongly accused
The instinct is to argue about the score. That is usually the weakest move available to you, because you end up debating a number neither side can interrogate.
Argue about process instead. Evidence that you wrote the document is concrete; a competing detector score is just another opinion from another black box.
A practical sequence:
- Preserve your version history immediately. Google Docs, Microsoft Word and Notion all retain revision history. Do not copy the text into a fresh document — that destroys the record. Export or screenshot the revision timeline while it exists.
- Assemble the trail around the work. Notes, outlines, annotated sources, library loans, browser history, messages where you discussed the assignment, earlier drafts sent to anyone. Incremental development over time is very hard to fabricate and easy to recognise.
- Ask what the accusation actually rests on. Request the specific tool, the score, and whether any evidence beyond the score exists. Ask what the institution's policy says about detector output as sole evidence — many now state explicitly that it cannot be.
- Point to the tool's own documented limits. You are not claiming detectors never work. You are noting that the vendors publish false positive rates, that OpenAI withdrew its own detector for poor accuracy, and that peer-reviewed work shows systematic bias against particular groups of writers.
- Offer to demonstrate your understanding. Volunteer to discuss the content, explain your argument, or write on the topic under observation. Someone who did the work can do this; it moves the question onto ground where you can actually be right.
- Escalate in writing if needed. Keep everything documented. If an institution treats a probability score as proof, that is a procedural issue worth raising through a formal appeal.
Do this before you need it
Write in a tool that keeps revision history, and keep your notes. It costs nothing while nothing is wrong, and it is the single most useful thing to have if you are ever questioned.
How to lower the odds of being flagged
You should not have to write defensively. But if you are in a context where your work will be scanned, a few things genuinely reduce false-positive risk — and all of them also make the writing better.
- Vary your sentence length deliberately. A short sentence after two long ones raises burstiness and reads better anyway.
- Include specifics only you would have: a particular source, a detail from your own reading, a concrete example from your context. Generic writing is what detectors find suspicious.
- Keep your own phrasing rather than smoothing everything to the most conventional wording. Some roughness is a signal of a person.
- Write in a tool with revision history, so the record exists by default.
- If you use AI at any point and your institution requires disclosure, disclose it. A disclosed workflow is a policy question; an undisclosed one that gets flagged is a misconduct question.
You can also check where your own text sits before submitting it. Our AI detector gives a score band rather than a verdict — under 30 reads as human, 30 to 49 likely human, 50 to 59 uncertain, 60 to 69 likely AI, and 70 or above as AI-written. Treat it the way you should treat any detector: as a rough signal about how predictable your text is, not as a judgement about who wrote it.
Paste your own draft to see how it reads. English only — the model is optimized for English, and results in other languages may be inaccurate.
An honest limit
Editing tools, ours included, change how a text scores. They cannot prove you wrote something, and no tool should claim to. Scores also move as detectors update. If you are facing an accusation, your draft history is worth more than any score you can generate.
Sources
- Liang et al., 'GPT detectors are biased against non-native English writers' — Patterns (Cell Press), 2023
- OpenAI: AI Text Classifier announcement and withdrawal note — OpenAI, 2023
- Turnitin guidance on its AI writing indicator and false positives — Turnitin
- Guidance on AI detection in Turnitin — Vanderbilt University, 2023