GPT-4o scored 0.82 on non-public booksโwhat does it mean for OpenAIโs GPT training data?
Beyond Public Access in LLM Pre-Training Data is a study by Ilan Strauss that applied the DE-COP membership inference attack to 34 O'Reilly Media books (13,962 paragraphs). GPT-4o scored an AUROC of 0.82 on non-public books, against 0.54 for GPT-3.5 Turbo and 0.56 for GPT-4o Mini, where 0.50 is chance. The author notes a small book sample and possible control-data contamination as limits.
research, training, evalsยท Published and added 3 Nov 2026


























































