Two open challenges toward an AI second opinion on mammograms

In the words of their briefs, classification tells you that something is suspicious, segmentation tells you where.

• Mammography Classification: fine-tune an open-source model to classify mammograms as normal, benign or malignant, or by BI-RADS category, scored on the area under the ROC curve (AUC).
• Mammography Segmentation: produce a mask of each lesion, scored on Dice and IoU, two closely related measures of overlap between the predicted mask and the reference.

Both start from CBIS-DDSM, a curated public collection of mammography images with lesion annotations. Each expects the same four deliverables: a documented dataset, a model with its reported score, the training code, and the model packaged as a callable API. The goal is not a leaderboard trick but a reproducible building block for a clinical validation phase.

Among the Lab’s contributors, Alix and Hedi were the first to take them on.

Breast cancer is the most common cancer in women in 164 of 186 countries. In 2024, an estimated 2.4 million women were diagnosed with it and 694,000 died of it, according to the World Health Organization. By 2050, the International Agency for Research on Cancer projects 3.2 million new cases and 1.1 million deaths a year, weighing disproportionately on countries with a low Human Development Index.

Survival depends on where a woman lives: five-year survival exceeds 90% in high-income countries, against 66% in India and 40% in South Africa, according to the WHO Global Breast Cancer Initiative. So does access to the people who read the images. Low-income countries count around one to two radiologists per million people, against more than 90 in high-income countries, according to a Lancet Oncology Commission. Even wealthy health systems fall short: the United Kingdom lacks 32% of the clinical radiology consultants it needs.

AI support for reading mammograms is one of the most closely studied uses of AI in medicine. In Sweden, the MASAI randomised trial found that AI-supported screening detected more cancers without increasing false positives, and reduced the radiologists’ screen-reading workload by 44%. Its final analysis showed it was non-inferior on interval cancers, those diagnosed between two screening rounds. In Germany, the PRAIM study, covering 463,094 women screened in routine practice, found a 17.6% higher cancer detection rate when radiologists used AI.

Two things matter in these results. In each of them, radiologists still read the images: AI supports them, it does not replace them. And they were obtained with mature commercial systems. Open models have a long way to go: in the RSNA 2023 screening mammography challenge, the median sensitivity of 1,537 submitted algorithms was 27.6%, and the best reached 48.6%.

The conditions for trusting such a tool are well established: a defined task and population, external validation, human oversight, monitoring after deployment. Our article on AI in medical imaging details all five.

MyTwin’s ambition is to make an AI second opinion on mammograms freely and easily accessible through MyTwin for Patients: to every woman who wants one, and first to those living in medical deserts, far from a radiologist. A second opinion here means information for a woman and her doctor, never a diagnosis. Our guide to the second medical opinion explains when and how one helps.

Hospitals that read large volumes of mammograms are the other side of the model: the technology is meant to be offered to them under licence.

The road there has steps that can’t be skipped:

1. open challenges to build reproducible models, the stage we are at today;
2. clinical validation on independent data;
3. certification as a medical device: under European rules, software that supports a cancer diagnosis from images can fall into the highest risk class;
4. availability in MyTwin for Patients, and licences for hospitals.

Each of these steps will have its own news.