magazine.sebastianraschka.comarticleBuilding an AI Text Detector From Scratch
Sebastian Raschka walks through building an AI-generated-text detector from scratch by fine-tuning a DistilBERT classifier that outputs a 0-100 AI-likelihood score, similar to the approach reportedly behind Substack's new AI-detection feature (built on Pangram-style methods). He then uses this detector as a verifier to train a small language model to rewrite text so it scores as human-written, turning the tutorial into a hands-on case study of the detector-evasion cat-and-mouse game and of verifier-based LLM training beyond math/code reasoning tasks.
TL;DR Tutorial fine-tunes a DistilBERT AI-text detector, then trains an SLM against it as a verifier to evade detection — a concrete classifier-evasion exercise.
Also seen from another angle🧠 AI →
Technologies named here
- Substack AI detector
- Pangram
- DistilBERT
- ChatGPT
- Hugging Face
Techniques named here
- Supervised classifiers
- Perturbation-based probability tests
- Perplexity measures
- Watermarking
Related in AI Security
blog.talosintelligence.comtoolIntroducing CAIRN: Frontier tracking for AI-integrated malware (Cisco Talos)
github.comadvisoryFlowise MCP Security Bypass Enables RCE
github.comrepoYesWeHack Claude Kit
jfrog.comarticle