Guide
Can AI detectors be wrong?
AI detectors are now used to make decisions about students, employees and authors, so the question of whether they can be wrong is a practical one. This guide explains why detector errors happen, what a score can and cannot establish, and how AFIP recommends weighing a detector result against other evidence.
Why do AI detectors get it wrong?
AI detectors make two kinds of error. A false positive flags human writing as AI-generated; a false negative passes AI-generated text as human. Turnitin describes a false positive as "incorrectly identifying fully human-written text as AI-generated" and states that its false positive rate "is not zero" (Turnitin, 2023).
Independent testing found both kinds of error across the market. A study of 14 detection tools, including Turnitin, concluded that they were "neither accurate nor reliable" and had "a main bias towards classifying the output as human-written rather than detecting AI-generated text" (Weber-Wulff et al., 2023). The same study found that content obfuscation techniques "significantly worsen the performance of tools."
Errors are not spread evenly across writers or texts:
- Non-native English writers. Widely used detectors "consistently misclassify non-native English writing samples as AI-generated" (Liang et al., 2023).
- Short text. OpenAI described its own classifier as "very unreliable on short texts (below 1,000 characters)" before withdrawing it in July 2023 (OpenAI, 2023).
- Edited and paraphrased text. A recursive paraphrasing attack "can significantly reduce detection rates" while only slightly degrading text quality (Sadasivan et al., 2023).
For how the underlying methods work and why each can fail, see AFIP's research on how AI text detection works.
What can an AI detector score prove?
A detector score is a probability produced by a statistical model. It can show that a passage resembles patterns the model associates with machine-generated text. It cannot show who typed the text, whether AI use was permitted, or how much a person edited a draft.
Detector vendors and institutions say the same. Turnitin states that it "does not make a determination of misconduct" and that, because its false positive rate is not zero, instructors "will need to apply your professional judgment, knowledge of your students, and the specific context surrounding the assignment" (Turnitin, 2023). The University of San Diego's law library guide for instructors states that AI detectors "are problematic and not recommended as a sole indicator of academic misconduct" (USD Legal Research Center, 2026).
No detector score should be the only evidence in a decision about someone's work.
How much weight should a detector score get?
AFIP answers that question with a five-grade evidence-weight scale.
AFIP uses a five-grade scale to describe how much a body of evidence supports a finding that text was AI-generated. It is a reasoning framework, not a measured statistic: each grade describes the kinds of evidence present, not a probability.
| Grade | What you have | What it supports | What it cannot support |
|---|---|---|---|
| 1. Lead | One detector score | A reason to look more closely | Any finding or sanction |
| 2. Supported lead | Two detectors using different methods agree | A documented concern worth raising | A conclusion about authorship |
| 3. Contextual support | The text conflicts with the author's known writing or knowledge | A request for drafts and an explanation | A conclusion without hearing the author |
| 4. Process evidence | Drafts, version history or file metadata contradict the claimed writing process | A formal review under your policy | A finding on detector output alone |
| 5. Corroborated | Independent kinds of evidence agree and the author has had a chance to respond | A decision under your written policy | Certainty; record the evidence so the decision can be reviewed |
Most real cases stop at grade 1 or 2. Moving beyond them requires evidence a detector cannot produce, which is why AFIP's methodology treats any single signal as a lead.
What should you do when a detector flags human writing?
- Pause any action. A flag is a grade 1 lead. Do not accuse, grade down or discipline on the score.
- Check the conditions. Note the text length, language and editing history. Short, non-native or heavily edited text is where errors cluster, and when AI use in a group is rare, many flags will be wrong; see why false positives add up at scale.
- Ask for process evidence. Drafts, notes and version history usually settle the question faster than any further scan.
- Hear the author. Ask them to explain how they wrote the piece and why they made specific choices.
- Record the outcome. Write down the evidence on both sides and the reason for the decision, so it can be reviewed later.
When does a second detector help?
A second detector adds information only if it works differently from the first. Two tools trained on similar data can share the same blind spot, so agreement between them counts for less than agreement between different methods. A second opinion is most useful when a first score is borderline, and least useful when the real question is about process, such as whether AI use was allowed.
Free options make a second check easy to run: as checked on October 14, 2026, Walter Writes states that its AI detector checks are free and unlimited. AFIP's guide on telling whether text was written by AI lists widely used detectors and a five-step cross-check procedure. Text that has been through a rewriting tool raises separate problems, covered in detecting AI text run through humanizers.
For related guidance, see how managers should handle AI use in workplace reports and how recruiters should treat AI-drafted resumes and cover letters, or browse all AFIP verification guides.
Frequently asked questions
Do AI detectors really work?
Partly. Detectors pick up statistical patterns that are common in unedited machine text, but independent testing of 14 tools found them neither accurate nor reliable, and accuracy falls on short, edited and non-native English writing. A detector score is useful as a lead to investigate, not as proof of how a text was written.
Why does an AI detector keep saying my writing is AI?
Detectors flag text that looks statistically predictable, and some human writing does; research has found non-native English writing is misclassified as AI-generated more often than native writing. If your own writing is flagged, keep your drafts and version history, which show how the text was produced, and ask for the flag to be reviewed alongside that evidence.
Is it ethical to use an AI checker?
Yes, when the result is treated as one piece of evidence. It becomes unfair when a score alone leads to an accusation or penalty, because every detector produces errors. Ethical use means telling people that detectors are used, weighing scores against other evidence, and giving the author a chance to respond.
Can AI detectors falsely detect AI?
Yes. A false positive is human writing wrongly flagged as AI-generated. Turnitin, one of the most widely used detectors, states that its own false positive rate is not zero, and research has found non-native English writing is misclassified as AI-generated more often than native writing.