AI Detector Accuracy: Slop or Not on the RAID Benchmark
Every AI detector publishes an accuracy figure. Ask where that figure came from and the answer is usually a test set nobody outside the company has seen.
Slop or Not's text detector is now listed on RAID, the public benchmark for machine-generated-text detectors, scored by the benchmark's own maintainers.
AI detector accuracy is easy to claim and hard to check, so we let someone else hold the answer key. Slop or Not catches 99.5% of AI text at a 5% false-positive rate on the full RAID test set, adversarial attacks included. That number sits on a public leaderboard next to dozens of other detectors, scored on the same documents under the same rules.
This post covers the text detector. RAID does not evaluate image detection, and nothing below claims otherwise.
What Is the RAID Benchmark?
RAID is a public benchmark for AI-text detectors, published in an ACL 2024 paper by a University of Pennsylvania-led team. It holds roughly 6 million documents generated by 11 models across 8 domains, run through 11 adversarial attacks and 4 decoding strategies. Anyone can submit a detector. The maintainers score it and publish the result.
The scale is useful. The part that matters more is who holds the answer key. Detector companies publish accuracy figures drawn from private test sets, some of them past 99.9%, and there is no way to check those figures because there is nothing to check them against.
RAID works the other way around. The maintainers score the submission rather than the company being measured, and every detector on the board sat the same exam, which is what makes the numbers comparable across products.
The adversarial attacks are where the field separates. RAID does not only feed detectors clean model output. It also modifies that output in 11 ways built to slip past detection, then re-scores every submission on the altered documents.
AI Detector Accuracy: How Did Slop or Not Score?
Slop or Not scored 99.81% AUROC on the full RAID benchmark and caught 99.52% of AI-generated text at a 5% false-positive rate, with all 11 adversarial attacks in play. Remove the attacks and those figures rise to 99.88% and 99.64%.
| RAID test set | AUROC | Detection rate @ 5% FPR | @ 1% FPR |
|---|---|---|---|
| Full benchmark (11 adversarial attacks) | 99.81% | 99.52% | 97.79% |
| Without adversarial attacks | 99.88% | 99.64% | 99.09% |
The two columns answer different questions. AUROC asks how cleanly the detector separates human writing from AI writing across every possible threshold. The detection rate at a fixed false-positive rate asks something more practical: with the dial set so that 5% of human documents get flagged, how much AI text does the detector actually catch?
The number worth reading is the drop between the two rows. Adding the full attack suite costs Slop or Not 0.12 points of detection rate, from 99.64% down to 99.52%. On the same benchmark, that suite costs Grammarly 0.44 points and GPTZero 1.13 points.
That gap is most of the job, because nobody pastes pristine model output into a checker. Text arrives after a spellcheck, a paraphrase pass, or a deliberate attempt to get around detection. A detector that only performs on clean samples fails on the first hard case.
How Does That Compare to Other Detectors?
As of August 2026, Slop or Not ranks 8th of 39 ranked detectors on RAID's default aggregate: detection rate at a 5% false-positive rate with adversarial attacks included. It leads GPTZero on every aggregate metric RAID reports. QuillBot, one place ahead, edges it by about a tenth of a point.
Leaderboard snapshot, August 2026: detection rate at 5% FPR, adversarial attacks included.
| Rank | Detector | @ 5% FPR |
|---|---|---|
| 1 | Zhuque | 99.80% |
| 2 | MELD | 99.78% |
| 3 | Trinka AI | 99.76% |
| 7 | QuillBot | 99.62% |
| 8 | Slop or Not | 99.52% |
| 9 | Grammarly | 99.47% |
| 16 | GPTZero | 97.24% |
As of August 2026 the board carries 43 submissions, four of them partial and unranked, which leaves 39 ranked detectors with Slop or Not in the top 10. It is also the only one on the board that runs its detection on-device in an iPhone and Mac app.
Against GPTZero, and scoped to RAID, Slop or Not leads on all five aggregate metrics the leaderboard reports: AUROC, plus detection rate at 5% and 1% false-positive rates, with and without attacks. The Slop or Not vs GPTZero comparison covers the rest of the differences.
Against QuillBot, Slop or Not loses. QuillBot catches 99.62% at a 5% false-positive rate to our 99.52%, roughly a tenth of a point ahead on the aggregate detection rate as of August 2026. It is on the same public board as our own score. Hiding it would defeat the point of submitting. The Slop or Not vs QuillBot comparison has the full picture.
The entries that are missing say something too. Winston AI and Pangram both advertise 99.98% accuracy. Neither appears on RAID. An accuracy figure with no public test behind it is a marketing claim, not a measurement.
What Doesn't RAID Tell You?
RAID measures one thing well: how a text detector holds up against the generator output collected for the 2024 dataset, under adversarial pressure. It has nothing to say about image detection, and nothing to say about models released after the dataset was built.
The generators are dated. RAID's 11 models are the ones the 2024 benchmark was built on, and several are older than that: GPT-4, ChatGPT, Llama 2, Mistral, Cohere, and MPT all predate it, and GPT-2 dates to 2019. Writing from a 2026 model is a different distribution, so RAID complements our internal testing against current models rather than replacing it.
RAID is also text-only. Slop or Not's image detection reports 90% accuracy on AI-generated images without provenance data and 100% on images carrying a C2PA or IPTC watermark from Gemini, OpenAI image generation, Microsoft Designer, or Adobe Firefly. Those figures come from our internal tests, and the RAID result does not extend to them.
One more thing about that 5% false-positive rate, because it reads worse than it is. The 5% is the operating point the benchmark reports at. Of 100 human-written passages, about 5 get flagged at that setting, and the benchmark then measures how much AI text the detector catches under that calibration. The 1% column is the same detector run stricter.
No AI detector is 100% accurate, this one included. Accuracy is based on our internal tests where noted above, and results vary with new AI models and with methods built to trick detectors. Treat a verdict as triage, not proof.
What Does This Mean If You Use Slop or Not?
The detector that earned the RAID score is the detector that ships. The same Core ML model runs on the Apple Neural Engine in the iPhone and Mac apps, so the number on the leaderboard describes the check you run on your own device.
Benchmark results often come from a configuration nobody outside the lab can install. This one does not. When you paste an essay into Slop or Not on an iPhone, the model handling it is the model RAID scored.
Where the check runs still differs by platform, and the difference matters. On iPhone and Mac, detection happens on-device and the text never leaves it, so a teacher can check student work and a lawyer can check a draft brief without deciding whether to upload it first. The free online AI text detector covers Windows, Android, Linux, and ChromeOS: the text goes to Numen's own Mac server, gets checked, and is deleted after processing. The apps have no word cap per document or per month. The online checker takes up to 20,000 characters per submission. The free tier limits how many checks you run per day, not how long each document can be.
Every detector claims a number. Ours is on a leaderboard you can open right now.
Run a document through the free online AI text detector in your browser, or download Slop or Not for iPhone and Mac to keep the text on your device. If you are weighing it against a cloud checker, the ZeroGPT alternative guide covers word caps and uploads, and the ChatGPT detector guide for iPhone walks through the on-device workflow.
