This is not a compelling analysis.

First, the CNN is not 10 years old. The base is but it was fine tuned for this project on the same dataset the test images were derived from:

> The test: The lesions come from MILK10k, a public dataset released through the International Skin Imaging Collaboration [ISIC] in 2025

> The CNN is deliberately not state of the art. I used Inception v3 pretrained on everyday photographs and then fine-tuned all layers on 68,011 dermoscopic and clinical images from the ISIC Archive on a single H100 GPU

The test images themselves were filtered out at least, but that's not really enough. The CNN has a home field advantage because the test images are going to be in distribution with the training images.

Second, Looking at the code, for frontier models probably you should be giving them image tools and using them in an agent loop and not trying to one shot results (e.g. https://platform.claude.com/docs/en/build-with-claude/prompt...). It is also not meaningful to ask an LLM for a numeric confidence score.

Third, all 200 of the the test images were biopsied by the clinician originally. So when you ask the model "You are assisting a dermatologist" and "Decide whether this lesion can be safely discharged, meaning it needs no treatment, no biopsy and no follow-up" why would you penalize the model for making the same call the physician did even if it was benign in retrospect?

Are you evaluating the model on if it can replace the need for a biopsy itself (in which case the ground truth is whether it in retrospect was harmful or not) or in if it can match physician judgement in determining when a biopsy is needed (in which case the ground truth is if they physician ordered a biopsy)? You're asking the model to do the latter and grading it on the former.

Overall yeah probably specialized models make a ton of sense in a bunch of areas still but idk this does not seem like a good demonstration of that

A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis