Perplexity
In AI text detection, perplexity measures how predictable a piece of text is to a language model. Lower perplexity often indicates AI-generated text.
Definition¶
Perplexity is a measurement of how well a probability model predicts a sample. In the context of language models, it quantifies how "surprised" a model is by a given sequence of text.
A lower perplexity score means the model found the text highly predictable — the words and structures used are exactly what it would have generated itself. Higher perplexity indicates less predictable, more varied language.
Why It Matters for AI Detection¶
AI detection tools like GPTZero use perplexity as one of their primary signals. The logic is straightforward: text generated by an AI model will naturally have low perplexity when evaluated by a similar model, because both systems are drawing from the same statistical distributions.
Human writing tends to have higher perplexity because humans introduce more idiosyncratic word choices, unexpected metaphors, and structural variations that don't conform to the most probable pattern.
Perplexity in Practice¶
A piece of AI-generated text might score a perplexity of 15–30 when evaluated by a detection model, while a human-written passage on the same topic might score 45–90.
These numbers vary significantly by: - The specific model used to generate the text - The topic and domain (technical writing has naturally lower perplexity) - The writing style and how much it was edited post-generation
Perplexity vs. Burstiness¶
Perplexity alone isn't a reliable detector — it's usually used alongside burstiness, which measures variation in sentence complexity and length. Human writing tends to be both higher-perplexity and more bursty (uneven in complexity and length).
AI text that has been humanized effectively increases perplexity by introducing less predictable word choices and structural variation.