How do AI detectors work? Perplexity, burstiness and their limits

That last part is the one to hold on to. A detector never sees how a text was made; it sees only the finished words and asks how much they look like what a model would produce. Every method below is a version of that question, and each one fails in its own predictable way.

How AI detectors work

There are three approaches, and a single tool can combine them.

Perplexity and burstiness

Language models write by choosing a likely next word, again and again. Perplexity measures how surprising a text is to a model: GPTZero, whose first detector was built on it, describes it as "a measure of how likely an AI model would have chosen the exact same set of words as found in the document." Its own example: after "Hi there, I am an AI", the word "assistant" has low perplexity, while "potato" has high perplexity and is more likely human.

Burstiness is the variation around that. GPTZero defines it as "how much writing patterns and text perplexities vary over the entire document." People write a plain sentence, then a strange one, then a short one; models tend to hold a steady level of predictability from start to finish. Low perplexity plus low burstiness is the classic AI signature.

GPTZero calls these "the least computationally expensive of AI detection methods", which is why they spread. They are also crude, and GPTZero itself now says: "As of autumn 2023, GPTZero no longer uses perplexity and burstiness for its AI detection because we migrated to a deep-learning based architecture."

Trained classifiers

A classifier is a model trained on large sets of labelled human and AI text until it learns to tell them apart. It picks up patterns no one wrote down, from word choice to sentence structure, rather than two numbers. Its weakness is that it knows only the writing it was trained on. Each new model writes a little differently, and critics say generative AI development "rapidly outpaces detection tools, allowing easier evasion," as Search Engine Journal reported when OpenAI withdrew its own classifier.

Watermarking

Watermarking works from the other end: the company running the model hides a signal in the text as it is generated. In the best-known research design, the model picks a random set of "green" tokens (words and word pieces) before each one it writes, then nudges its choice toward them, so the text carries a mark that is "invisible to humans but algorithmically detectable from a short span of tokens", without access to the model itself.

A watermark gives a statistical test rather than a style guess, but only for output from a model that applied one. Google's SynthID Detector, which checks images, video and audio for the watermark used by Google and its partners, says as much about itself: "This is not a general AI detector." Text from a model without a watermark carries no mark to find, and even where one exists, the SynthID FAQ warns that "no signal is completely robust."

How accurate are AI detectors?

Less accurate than a single confident number suggests, and least accurate on the cases that matter most.

The clearest data point is a withdrawal. OpenAI launched a classifier for AI-written text in 2023, then added a note to its announcement: "As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy." The company behind ChatGPT could not make its own detector reliable enough to keep offering it.

False positives fall unevenly. A 2023 study of several widely used detectors found that they "consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified." The authors conclude that detectors "may unintentionally penalize writers with constrained linguistic expressions": a smaller vocabulary and simpler sentences are more predictable, which is what perplexity-based detection reads as machine-made. The same study showed the opposite failure too: simple prompting strategies "effectively bypass GPT detectors".

Length matters as well. A few sentences give a detector little to measure, which is why our own tool asks for at least a paragraph and works best on 50 words or more.

What a detector score cannot tell you

A score is an estimate about style, and style is all it measures. It cannot tell you who wrote the text: formal, plain prose by a careful person can score as AI, and a prompted or edited model draft can score as human. It cannot tell you how much was AI-assisted either, because a model draft rewritten by a person and a human draft polished by a model reach the detector as one blended text. And it says nothing about whether the text is any good. Accuracy, originality and usefulness are not in the score; a perfect human score on a page that answers nothing is still a page that answers nothing.

So use a detector the way it can be used: as a prompt to read more closely, never as evidence against a writer. The questions that decide whether a draft deserves to be published are the ones SEO content writing asks, and they hold whether a person, a model or both did the typing. The blog post on AI blogging strategy covers where AI drafting fits in a workflow that keeps those questions in view.

How Structura handles this

Structura offers a free AI detector that runs entirely in your browser: the text you paste is not uploaded or stored. It shows three heuristic signals, each on a 0 to 100 scale where higher reads as more AI-like. "Low perplexity" is a proxy, measured as how varied the vocabulary is; it is not true perplexity computed by a language model. "Uniform rhythm" is burstiness, measured as how much sentence length varies. "Repetitive phrasing" counts repeated sentence openers and repeated word pairs. The headline score comes from a small classifier model when your browser can load one and from these signals when it cannot, and the result says which one you got.

What it does not do matters as much. It does not check for watermarks, it does not compare your text against known AI outputs, and nothing in it was trained on the newest models' output. It is a guide, not a verdict, and the tool says so with every result.

For publishing, the point is not the score. Structura's posts are written for a query and a reader, in an author persona you define, and they go out on your schedule or wait for your approval. If you automate content, content automation covers which parts still need a person.

FAQ