How Red Stet differs from AI-content detectors

The fundamental difference

AI-content detectors analyze the finished text. They look at the prose the reader receives — perplexity against a reference language model, burstiness across sentences, syntactic regularity, word-choice patterns — and produce a probability score that the text was generated by a language model. The signal is the prose itself; the verdict comes from how the prose pattern-matches against generated output.

Red Stet analyzes the writing process. The recording sidecar captures keystroke cadence, paste events, mouse-path geometry, correction rate, and pause distribution while the document is being written, and the verifier scores whether the resulting trace is consistent with hand-typed composition. The signal is the behavior that produced the prose; the verdict comes from how the trace pattern-matches against recorded human writing.

These are different signals applied to the same artifact. An AI detector and Red Stet can read the same document and reach different verdicts because they are reading different evidence. Neither is wrong in those cases — they answer different questions, on different inputs, with different physics behind the answers.

AI-detector failure modes

The published literature on AI-content detectors documents several recurring failure modes. None of these are vendor-marketing complaints; they appear in peer-reviewed work and replicated benchmarks.

False positives on non-native English writers

Liang, Yuksekgonul, Mao, Wu, and Zou (2023), GPT detectors are biased against non-native English writers, published in Patterns (Cell Press), tested seven widely-used GPT detectors against TOEFL essays written by non-native English speakers. The detectors classified more than half the human-written TOEFL essays as AI-generated. The same detectors classified U.S. eighth-graders' essays correctly. The bias source: non-native writing has lower lexical diversity and more regular syntactic patterns, which the detectors read as model output.

False positives on heavily-edited prose

The detectors most sensitive to perplexity also fire on prose that's been through multiple revision passes. Careful word choice, polished sentence structure, and the regularity that comes from copy-editing all lower perplexity in the same direction that model output does. Researchers including Sadasivan and colleagues note this effect in their paraphrasing-attack analyses: editing for clarity moves human prose into the detector's machine band.

Paraphrasing defeats the signal

Sadasivan, Kumar, Balasubramanian, Wang, and Feizi (2023), Can AI-Generated Text Be Reliably Detected?, on arXiv, made the broader theoretical argument: as language models approach human-quality output, the distributional gap detectors rely on closes, and even a moderate paraphrasing attack collapses detection accuracy to near-chance. They demonstrated the effect empirically against then-leading detectors (DetectGPT, OpenAI's classifier, GPTZero) — a single paraphrasing pass dropped accuracy below 50% on most.

Domain sensitivity

Detectors trained predominantly on conversational English produce noisy results on academic writing, technical documentation, legal prose, and structured genres. Wang, Mansurov, and colleagues' 2024 M4 benchmark and follow-up multilingual evaluations show accuracy drops of 10–30 points when detectors trained on one domain are applied to another. The signal is real but corpus-specific.

The humanizer arms race

Dedicated "humanizer" tools — Quillbot, Wordtune, Undetectable.ai, and a long tail of smaller services — paraphrase model output until it lands below detector thresholds. The arms race is well-documented in the academic-integrity literature; Krishna, Song, Karpinska, Wieting, and Iyyer (2023), Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense, at NeurIPS, showed paraphrasing reliably defeats output-text detectors and argued that the only durable detection path is corpus retrieval against the model's training output, not perplexity.

Red Stet's failure modes

The behavioral-fingerprint approach has its own published-and-acknowledged limits. The full treatment lives at Methodology → Known limitations; the short list here is the version a reader comparing tools needs.

Adversarial typing of model output

A writer who generates text in a language model, reads it on a second screen, and types it character-by-character into the editor produces a genuine composition fingerprint. The keystroke cadence is real, the corrections are real, the pauses are real. Red Stet sees a hand-typed document because the document was hand-typed; the authorship question — whether the writer composed the prose or transcribed it — is invisible at the process layer.

Small documents

The statistical signals stabilize only after a few hundred samples. A 50-word note has too little data to score confidently — so the verifier says so: below 500 typed characters it reports "Insufficient evidence — document too short to score" instead of a verdict tier (the composite is shown for transparency, labeled as not-a-verdict), and between 500 and 2,000 characters the tier carries a reduced-confidence note. An integrity board reviewing a 100-word answer gets an honest "insufficient," not a manufactured probability. See Limitations → Short documents for the bands.

Non-keyboard input modalities

Voice dictation, switch input, head tracking, eye-gaze keyboards, and other accessibility input methods produce composition traces that look unlike either typed or pasted text. They are human-authored, and the writer's process is genuine, but it does not match the keyboard-typing reference distribution Red Stet's signals are built on — and under the current scoring, these documents do score below the hand-typed thresholds. The verifier does not yet detect input modality or annotate the verdict's keyboard scope; that routing is open work. A reviewer handling a writer with documented assistive input should treat the composite as not-applicable and lean on the rest of the integrity process. See Limitations → Assistive input modalities, which this page defers to.

Adversarial calibration

A writer who knows the recording is happening, knows what the signals measure, and has time to practice can move their composite score deliberately. Slow-typing model output, breaking pastes into hand-typed fragments, simulating correction rates — the published behavioral-biometrics literature has documented all of these adversarial strategies. Red Stet's response is the multi-signal composite (every signal has to be spoofed in sync) and the per-moment surfacing (the reader sees each questionable window directly), not a claim of unspoofability.

The case for stacking

The failure modes of the two signal classes are largely disjoint. The cases that defeat an AI detector — non-native English, heavy editing, paraphrased model output, conversational content — leave the composition fingerprint untouched. The cases that defeat Red Stet — hand-typed model output, very short documents, non-keyboard input — leave the output-text signal untouched.

A document that passes both signals is both process-consistent and output-pattern-consistent with human authorship. Two independent signals agreeing on the same answer reduce the joint error rate to the product of the individual error rates, not their sum. The joint pass case is much stronger evidence than either signal alone produces.

A document that fails one signal and passes the other is the interesting case — the case an integrity board wants to see surfaced rather than buried under a single percentage. A TOEFL essay flagged by an AI detector but cleared by Red Stet's composition fingerprint reads, on examination, like exactly what the Liang et al. result predicted: a non-native English writer's prose being misclassified by perplexity-based detection, with the keystroke trace showing the writing happened at the keyboard, in real time, with the normal pauses and corrections. The opposite case — composition-clean prose flagged by Red Stet as a non-keyboard pattern — is the writer using accessibility input, and the integrity board should know that before adjudicating.

The practical recommendation that follows from this is: an academic-integrity process that runs both signals on a flagged document, and reads them as independent evidence to be interpreted, has more reliable inputs than either signal alone produces. That is the case for stacking. It is not a Red Stet sales pitch; it is what the disjoint failure-mode profiles imply.

Different questions, different defenses

The two tools answer different questions about a document.

An AI-content detector answers: was this text generated by a model? The output is a probability, the framing is adversarial (the writer is the accused), and the verdict tends to be taken as evidence in disciplinary proceedings even when the published research is clear it shouldn't be.

Red Stet answers: is this document consistent with hand-typed composition? The output is a composite score plus the per-moment evidence that produced it. The framing is affirmative (the writer is presumed to have written the document, and the recording is evidence for that claim). The verdict is interpretive — the reader sees the flagged moments themselves and decides whether the document warrants further conversation, not whether the writer is guilty.

The shape of the defense follows the shape of the question. A "was this AI?" finding lands on the writer as an accusation they have to refute against a vendor's percentage. A "was this hand-typed?" finding lands on the document as evidence the writer can point to in their favor — the recording shows the prose accumulating in real time, with the corrections and the pauses and the re-reads. The first cycle is adversarial and produces the false-positive horror stories the integrity community has been documenting since 2023. The second is interpretive and produces material the integrity process can read.

Adding a third signal — voice profiling

A third class of signal sits outside both of the above and is on Red Stet's future-work list: stylometric voice profiling. The published stylometric literature — Mosteller and Wallace on the Federalist Papers (1964), Burrows' Delta (2002), modern n-gram authorship-attribution work — establishes that a writer's literary voice (word-choice preferences, sentence-rhythm tendencies, syntactic tics, lexical idiosyncrasies) is recoverable from prose alone and is stable across documents. When Red Stet ships a voice-profile capability, it will sit alongside the composition fingerprint and the AI-content signal as a third independent input.

The three questions become: does the process trace look like hand-typed composition (composition fingerprint), does the prose sound like this writer's other prose (voice profile), and does the prose pattern-match against generated output (AI-content detector). A document flagged by all three is high-confidence not the writer's authentic work. A document flagged by only one is worth a closer look but doesn't license a verdict. Three independent signals, three different failure-mode profiles, one more layer of disjoint evidence the integrity process can synthesize. The voice-profile work hasn't started yet; the framing here is where it will land when it does.

Bottom line

Red Stet and AI-content detectors answer different questions about authorship, on different inputs, with different failure modes. Used together they produce more reliable evidence than either alone. Used in isolation, both have documented failure cases that an integrity process should account for. The honest framing for a board chair, a journalist, or a researcher comparing the two tools is: they are complementary signal classes, not competing products, and the published literature on both is the right starting point for deciding which to run, when, and how to read the result.

References on AI-content detection

The detector-side citations from this page. The behavioral-biometrics citations underlying Red Stet's signals live on the per-signal pages and in the full bibliography at Methodology → References.

Liang, Yuksekgonul, Mao, Wu & Zou (2023)

GPT detectors are biased against non-native English writers. Patterns (Cell Press), 4(7), 100779. The result that anchors most published skepticism about output-text detection at scale. doi:10.1016/j.patter.2023.100779

Sadasivan, Kumar, Balasubramanian, Wang & Feizi (2023)

Can AI-Generated Text Be Reliably Detected? arXiv:2303.11156. The theoretical and empirical argument that paraphrasing attacks collapse output-text detector accuracy to near-chance. arxiv.org/abs/2303.11156

Krishna, Song, Karpinska, Wieting & Iyyer (2023)

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems (NeurIPS) 36. The replicated paraphrasing-defeats-detection result, with a proposed corpus-retrieval alternative. arxiv.org/abs/2303.13408

Mitchell, Lee, Khazatsky, Manning & Finn (2023)

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. Proceedings of the 40th International Conference on Machine Learning. The probability-curvature detector benchmarked in most subsequent comparison studies. arxiv.org/abs/2301.11305

Wang, Mansurov, Ivanov, Su, Shelmanov, Tsvigun, Whitehouse, Mohammed Afzal, Mahmoud, Sasaki, Arnold, Aji, Habash, Gurevych & Nakov (2024)

M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection. Proceedings of EACL 2024. The cross-domain benchmark documenting the 10–30 point accuracy drop when detectors trained on one domain are applied to another. arxiv.org/abs/2305.14902

Weber-Wulff, Anohina-Naumeca, Bjelobaba, Foltýnek, Guerrero-Dib, Popoola, Šigut & Waddington (2023)

Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19, 26. The European Network for Academic Integrity's evaluation of 14 detection tools: accuracy across tools was low and inconsistent, with substantial false-positive rates on human-written text. doi:10.1007/s40979-023-00146-z

Sadasivan, Kumar, Balasubramanian, Wang & Feizi (2024)

Follow-up analyses on paraphrasing attacks and watermark-removal strategies in the year following the original arXiv paper. Cited in subsequent integrity-community surveys. (See arXiv:2303.11156 v2 and later.)

Mosteller & Wallace (1964)

Inference and Disputed Authorship: The Federalist. Addison-Wesley. The founding work of statistical stylometry; cited here for the voice-profile direction Red Stet plans to build toward.

Burrows (2002)

'Delta': a Measure of Stylistic Difference and a Guide to Likely Authorship. Literary and Linguistic Computing, 17(3), 267–287. The modern stylometric distance measure most authorship-attribution work descends from. doi:10.1093/llc/17.3.267