10-Year-Old CNN Beats LLMs on Skin Cancer [Try It]

Open the demo before you trust another mole screenshot

Five minutes. No account. Your image stays in the browser.

1. Go to lynchlab.net/skin-ai-inception.
2. Wait while ONNX Runtime Web pulls the weights (WebGPU if you have it; WASM otherwise). First load feels slow. That is the model arriving on-device – not a spinner on someone’s GPU farm.
3. Hit Try a random sample image, or drop one clinical close-up / dermoscopic frame of a single lesion.
4. Read the six-class scores and the research discharge thresholds. Believe the red disclaimer: not a medical device, misses cancers, never a care decision.

You just ran the same class of network that beat two frontier multimodal chat models on a clean 200-lesion stress test. The rest of this piece is what the scores are actually saying – and what to build if your product touches skin photos.

What you are looking at

The weights are Inception v3, the architecture from Szegedy et al.’s 2015 paper Rethinking the Inception Architecture for Computer Vision (arXiv:1512.00567). Dermatologist Dr Magnus Lynch (Guy’s Hospital / King’s College London) swapped the ImageNet head for diagnostic groups plus a dermoscopy-vs-clinical head, then fine-tuned every layer on 68,011 ISIC Archive images. Wall clock: about twenty minutes on one H100. Test lesions and same-patient photos were held out.

Pro tip: Feed a real dermoscopic frame when you can. The demo leans on that view when both exist; Lynch’s own internal checks treat plain clinical close-ups as the weaker input.

Privacy detail product tours usually skip: inference never uploads the pixels. Useful when a clinician will not send patient photos to a third-party chat API.

The 62% vs 42% gap (and why the table lies if you skim it)

Lynch’s write-up dated 25 October 2026 put 200 biopsy-proven pairs from public MILK10k – clinical + dermoscopy – in front of GPT-6 Astra, Claude Fable 5.1, and the fine-tuned CNN. Same clinician-style prompt for the chat models: one of six labels (BCC, SCC, melanoma, naevus, actinic keratosis, seborrhoeic keratosis, or unsure), second choice, confidence, yes/no discharge. No age, site, or history. Neither LLM refused a case.

Six-way chance is ~17%. So 42% is skill – still too low for triage. Per the Lynch Lab results, top-2 tells on the specialist net: 173 vs 134-138. Melanoma top-1 stayed ugly for everyone (13 / 11 / 8 of 30).

Error modes were not random noise. Astra leaned basal cell carcinoma; Fable leaned actinic keratosis – favorite wrong classes, again and again. Astra’s confidence barely moved between hits and misses.

A fluent wrong label plus a calm discharge line is worse than a flat “I don’t know.” That is the product failure mode, not the accuracy headline.

Prompt framing flips the chatbot – not its eyes

Ask as a clinician and the LLMs will discharge some lesions. Ask as a patient (“do I need a doctor?”) on further melanoma sets and they almost always refer – 99 and 98 of 100 in Lynch’s check. That is refusal policy and tone, not better vision.

So safety copy in the chat UI can look responsible while the visual channel stays weak. Do not confuse the two when you design a flow.

Gotchas headlines bury

The catch is the operating point. At the CNN’s preset research threshold it discharged 18 of 30 melanomas and kept only 77/100 unsafe cases – more aggressive than either chatbot. Match the LLM-level caution and melanoma discharges drop to one or two; unsafe retention jumps to 98-99/100. Quote “62%” alone and you miss that trap.

Hard set, on purpose: every lesion was biopsied, including benign ones that already worried a clinician. Ordinary archive images gave the same net much stronger discharge separation (AUC ~0.96 vs ~0.86 on these 200). Internal malignant-vs-benign AUC on held-out data sat near 0.944 – another reminder that the headline test is the hostile slice.

MILK10k has been public since 2025. If anything, leakage risk favored the LLMs, not the CNN. Chat answers also jitter run-to-run; Lynch scored single passes, and discharge bits flipped on repeats even when the wrong label stuck.

If you ship AI features

Fixed-label lesion photos: fine-tuned CNN or a modern derm foundation model owns the call. Chat multimodal models do not – not on this evidence, as of Lynch’s October 2026 retest.

Split the stack hard:

• Specialist vision for “classify this image under a closed label set.”
• LLM for explanation after a specialist score, differentials in language, paperwork – with guardrails that cannot override the vision head’s discharge policy.
• No production path where a general chatbot alone signs off discharge from a raw photo.

Cheap training path Lynch actually used: ImageNet backbone → replace the head → fine-tune on ISIC with strict lesion/patient leakage controls → pick the threshold on a validation ROC for your operating point (screening sensitivity ≠ research demo cut). Esteva et al.’s 2017 Nature result already showed specialist CNNs can hit dermatologist-level binary performance on large clinical sets; the 2026 retest just puts that old lesson next to GPT-6-class and Claude-class vision.

Newer derm nets will beat this Inception v3. The experiment was never “2015 is SOTA.” It was “a small specialist still smokes a frontier generalist on this visual task.” On Hacker News, the thread that stuck was the same idea: use the LLM to help build the classifier – do not use the LLM as the classifier when pixels and a closed label set are the whole job.

FAQ

Can I use ChatGPT or Claude to check a mole?

No. Not for decisions. On this biopsy-proven set they lost to a decade-old specialist net and still discharged melanomas in clinician framing.

Is the browser CNN safe to trust?

No. Experimental only. It misses melanomas, skews toward lighter skin tones from specialist clinics, and ships research thresholds. Worried about a lesion? See a clinician. The demo teaches model behavior; it is not self-triage.

Should builders abandon multimodal LLMs for medical images?

Re-scope, do not abandon. Closed-label dermoscopy classification wants a fine-tuned vision model (or derm foundation model) with calibration plots and a human-chosen operating point. Keep LLMs on language and workflow after that score exists. The misconception is treating chat vision as a drop-in dermatoscope because the prose sounds careful – patient-style prompts will refuse almost everything while clinician-style prompts still mislabel and occasionally discharge. Brand of chatbot matters less than leakage controls and who is allowed to say “go home.”

Run ten random samples on the Inception demo. Write down where you would set a threshold if a false discharge cost a life. That number – not the model card – decides whether a chatbot belongs in the loop.