Configuration, Not Conscience: A Large-Scale Empirical Study of LLM System Prompts

What it claims

Leaked system prompts read mostly as operational specifications, not value statements. The corpus comes from four community repositories and contains 407 files from 62 vendors. Only 31 of these are vendor-published (Anthropic release-note prompts, 2024-07-12 to 2026-05-28); the other 376 were leaked or reconstructed. The reuse analysis adds two things the summary lacks:

• Copied text is concentrated in a few coding agents. RooCode has 54% of its word 5-grams contained in Cline, and Tencent CodeBuddy's Craft prompt has 50%. The identity sentence in CodeBuddy matches Cline and RooCode except for the assistant name.
• Architectural convergence mostly happens without copying. Devin and Amp score 0.915 on semantic similarity but only 0.014 on literal overlap.

Anthropic's Chrome extension and Perplexity's browser assistant are the one pair that ranks first on literal, semantic and structural similarity at once. The authors say they cannot tell influence from shared convention.

How it works

• Corpus. Pinned commits of four collections: CL4R1T4S (09916a9), System-Prompts (0d04045), system-prompts-and-models-of-ai-tools (570364c) and system prompts leaks (5d3c769), collected 2026-06-17 to 2026-07-11. Tool JSON schemas, README files and Claude Code's bundled tool docs were excluded.
• Deduplication.
• Word 5-gram Jaccard ≥0.85 within each vendor, grouped by connected components.
• A guard blocks merges between files with different model-tier tokens, because an Opus/Sonnet pair can reach Jaccard 0.97.
• Result: 29 clusters covering 66 files, leaving 370 unique documents. Duplicates are tagged, not dropped.
• Labels. A single coder assigned one of six archetypes and one value posture. Postures: 236 operational, 156 caution, 15 truth-seeking. 152 of 155 chat files are caution or truth-seeking. Every coding, autonomous, app-builder and search prompt is operational.
• Block classifier. Paragraph-like chunks are each assigned to one of eight buckets by strongest lexical match. A mixed paragraph counts entirely toward its dominant bucket. Regex detectors cover emphasis, harm categories, injection defense and rot markers.
• Similarity, three ways. TF–IDF cosine (1–2-grams), 5-gram Jaccard with asymmetric containment, and MiniLM (all-MiniLM-L6-v2, 256-token chunks, mean-pooled).
• Structure. A markup-only skeleton (XML tags, headers, code fences, lists) compared by profile cosine. Pairwise analyses use a 95-document subset: the longest file per vendor × archetype pair.

Evidence

• Prompt length by archetype (mean words): app builder 7,153; chat 5,630; autonomous agent 4,590; coding agent 3,878; narrow 1,368; search 989. Standard deviations are often larger than the means.
• Top cross-vendor pairs:

RooCode is a public fork of Cline, which validates that the method finds known reuse.

• The three measures are largely independent. Literal vs. semantic r = 0.175. Structural vs. literal r = 0.14. Structural vs. semantic r = 0.04.
• Strong structural matches with zero literal overlap:
• AWS Kiro spec-mode – Microsoft Word Copilot: 0.73
• Augment – Google AI Studio: 0.68
• Cluely – ElevenLabs: 0.67
• Structure is about which elements are used, not their order. Shuffling tokens barely changes cross-vendor overlap (Jaccard 0.038 ordered vs. 0.031 shuffled).

Limits

• Classifier precision is moderate. Pilot validation scored 35/48 blocks (73%), with 6 blocks per bucket. Tool/protocol and safety policy each reached 4/6, and the catch-all bucket 2/6. Scoring was automated, not independent human coding. The 58%/5% headline therefore depends on a noisy classifier, and the authors call all magnitudes directional.
• Detector and label biases.
• Regexes are English-only.
• They may over-count permissive mentions and under-count paraphrases.
• Archetype labels come from a single coder.
• Unverified sources. The 376 non-official files cannot be checked for fidelity, completeness or dates. Results describe those documents, not vendor policy or deployed model behavior.
• Similarity is not attribution. The authors state that similarity scores do not show copying or collaboration between vendors.
• The fetched text stops partway through Section 4.1. The following appear here only as abstract-level claims, without their supporting numbers or method:
• the 11:1 rule-line ratio
• the five-way comparison of official vs. leaked prompts
• prompt-rot figures (five version chains, diffed at word level)