A 10-year-old convolutional neural network beat GPT-6 Astra and Claude Fable 5.1 at diagnosing skin cancer in a 200-lesion test published October 25 by Magnus Lynch, a consultant dermatologist and Mohs surgeon at Guy's Hospital in London. The specialised Inception v3 model named the correct diagnosis as its first choice in 125 of 200 cases, or 62%. Both frontier LLMs landed at 42%, with 84 correct for GPT-6 Astra and 85 for Claude Fable 5.1.
Lynch drew 200 biopsy-proven lesions at random from MILK10k, a public dataset released in 2025, with a clinical photograph and a dermoscopic image for each. Half should not be discharged: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas including Bowen's disease, and 20 actinic keratoses. The other half were safe to discharge, split between 50 benign naevi and 50 seborrhoeic keratoses. Because every lesion was biopsied, benign ones included, Lynch notes these cases are harder to diagnose.
OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1 received both images and no other context. Each was asked for the most likely of six diagnoses or "unsure," a confidence level, a second choice and a discharge decision, using default settings and one run per case. The CNN was Inception v3, pretrained on everyday photos and fine-tuned on 68,011 ISIC Archive images on a single H100 GPU. The test lesions were not used to train the network, choose the model or set its threshold.
Key facts
- 01Inception v3 picked the correct first diagnosis for 125 of 200 biopsy-proven lesions (62%), versus 42% for GPT-6 Astra and Claude Fable 5.1.
- 02On 30 melanomas, the CNN got 13 right, Claude Fable 5.1 got 11 and GPT-6 Astra got 8.
- 03The test drew 200 random lesions from the public MILK10k dataset, each with a clinical photo and a dermoscopic image.
- 04The CNN was fine-tuned on 68,011 ISIC Archive images on a single H100 GPU, with the test lesions held out.
- 05Magnus Lynch, a consultant dermatologist at Guy's Hospital in London, published the results on October 25, 2026.
The gap held when second choices were counted. The correct diagnosis appeared in the top two for 173 lesions with the CNN, 138 with Claude Fable 5.1 and 134 with GPT-6 Astra. On the 30 melanomas, the CNN got 13 right (43%), Claude Fable 5.1 got 11 (37%) and GPT-6 Astra got 8 (27%).
“A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs.”— Magnus Lynch, Consultant Dermatologist, Guy's Hospital
Results varied by lesion type. GPT-6 Astra identified all 25 basal cell carcinomas, one more than the CNN and seven more than Claude Fable 5.1. The CNN led on squamous cell carcinoma with 16 of 25, against 12 for GPT-6 Astra and 5 for Claude. Claude Fable 5.1 was strongest on actinic keratosis with 12 of 20, while the CNN led on benign naevi with 37 of 50 and on seborrhoeic keratoses with 26 of 50.
Each LLM showed a distinct failure pattern. GPT-6 Astra labelled 75 lesions as basal cell carcinoma when only 25 were, including 14 of the 30 melanomas. Claude Fable 5.1 labelled 53 lesions as actinic keratosis against 20 actual cases. Of the CNN's 17 incorrect melanoma calls, 8 were naevi and 6 were seborrhoeic keratoses.
The discharge decisions show a different trade-off. The LLMs were cautious: GPT-6 Astra kept 97 of the 100 lesions that needed follow-up and Claude Fable 5.1 kept 96, but they discharged only 32 and 22 of the 100 safe lesions. At its preset threshold the CNN discharged 76 safe lesions but kept only 77 of the unsafe ones, sending home 18 of the 30 melanomas. With its threshold matched, the CNN kept 98 or 99 unsafe lesions and discharged one or two melanomas.
Lynch also ran a patient-style test, giving the LLMs only the clinical photograph and the question "do I need to see a doctor?" Both models advised seeing a doctor in almost every case, harmless lesions included.
Lynch lists several limits. Two hundred lesions are enough to show the pattern but not to rank close results, and each LLM case was run once even though answers vary between runs. MILK10k has been public since 2025, so the LLMs could have seen it in training. Inception v3 is a baseline rather than state-of-the-art skin cancer AI, scoring an AUC of 0.86 on these biopsy-only cases against 0.96 on ordinary archive images.
Lynch designed the test and made the clinical rulings, while Anthropic's Claude Code produced the analysis and first drafts. Scoring was done by script against histology.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
