A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

What it claims

A dermatologist tested GPT-6 Astra and Claude Fable 5.1 against a fine-tuned Inception v3 CNN in October 2026, using 200 biopsy-proven MILK10k lesions. The CNN's advantage was largest on benign lesions, where the LLMs over-called cancer. Each LLM leaned on one favourite diagnosis. The CNN also missed melanomas: it named 13 of 30, against 8 and 11 for the two LLMs.

How it works

The test set has 100 lesions that should not be discharged and 100 that are safe to discharge:

• Should not be discharged: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas (including Bowen's disease) and 20 actinic keratoses.
• Safe to discharge: 50 naevi and 50 seborrhoeic keratoses.

The benign lesions were all biopsied, so the author describes them as harder than typical cases.

Each LLM received the clinical and dermoscopic images and a single prompt at default settings. The prompt asked for the most likely of six diagnoses (or "unsure"), a confidence, a second choice, and a discharge decision. Neither model refused a case.

The CNN is Inception v3, pretrained on everyday photographs. All layers were fine-tuned on 68,011 ISIC Archive images on one H100. Every image of the test lesions and their patients was removed from training and validation data. The CNN scores both images of a lesion and weights the dermoscopic score more heavily.

Evidence

• Top-1 accuracy: the CNN named the right diagnosis for 125 of 200 lesions. The two LLMs named it for 84 and 85.
• Top-2 accuracy (second choice allowed): the CNN reached 173. The two LLMs reached 134 and 138.
• Naevi (out of 50): the CNN named 37, GPT-6 Astra 16 and Claude Fable 5.1 33.
• Seborrhoeic keratoses (out of 50): the CNN named 26, GPT-6 Astra 13 and Claude Fable 5.1 6.
• Where the CNN trailed: it was one short of GPT-6 Astra on basal cell carcinoma and behind both LLMs on actinic keratosis.

The models failed in different ways. GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the 30 melanomas. Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were. The CNN's 17 melanoma misses went toward benign labels: 8 naevus and 6 seborrhoeic keratosis.

A second test posed the question as a patient would. Each LLM saw the clinical photo alone and was asked "do I need to see a doctor?". Both advised seeing a doctor in almost every case, harmless lesions included. On a further 100 biopsy-proven melanomas, they advised a doctor for 99 and 98.

The CNN's discharge AUC is 0.86 on this set. It is 0.96 on ordinary archive images.

How to use it

The CNN runs entirely in a phone browser using WebGPU, and nothing is uploaded. Its random-sample button draws from the same 200 test lesions. One download contains the cases, the prompts, every LLM reply and the scoring code. Scoring was done by script against histology. MILK10k images are under CC BY-NC, which rules out commercial reuse. The page does not state a licence or file size for the CNN weights.

Limits

• Single run: each LLM case was run once, and answers vary between runs.
• Small numbers: 30 melanomas give wide uncertainty, and the author says the sample cannot rank close results.
• Contamination: MILK10k has been public since 2025, so the LLMs may have seen it in training. The author argues this would favour them.
• Weak baseline: Inception v3 is a deliberately weak baseline. The author says state-of-the-art systems use ensembles of image transformers, and regulated products perform considerably better.
• Disclosure: the analysis and first drafts were produced with Claude Code, which runs on a Claude model. One of the tested LLMs is also Anthropic's.