Purpose

The aim of this study was to describe the current state of science regarding independent external validation of artificial intelligence (AI) technologies for screening mammography.

Methods

A systematic review was performed across five databases (Embase, PubMed, IEEE Explore, Engineer Village, and arXiv) through December 10, 2020. Studies that used screening examinations from real-world settings to externally validate AI algorithms for mammographic cancer detection were included. The main outcome was diagnostic accuracy, defined by area under the receiver operating characteristic curve (AUC). Performance was also compared between radiologists and either stand-alone AI or combined radiologist and AI interpretation. Study quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 tool.

Results

After data extraction, 13 studies met the inclusion criteria (148,361 total patients). Most studies (77% [n = 10]) evaluated commercially available AI algorithms. Studies included retrospective reader studies (46% [n = 6]), retrospective simulation studies (38% [n = 5]), or both (15% [n = 2]). Across 5 studies comparing stand-alone AI with radiologists, 60% (n = 3) demonstrated improved accuracy with AI (AUC improvement range, 0.02-0.13). All 5 studies comparing combined radiologist and AI interpretation with radiologists alone demonstrated improved accuracy with AI (AUC improvement range, 0.028-0.115). Most studies had risk for bias or applicability concerns for patient selection (69% [n = 9]) and the reference standard (69% [n = 9]). Only two studies obtained ground-truth cancer outcomes through regional cancer registry linkage.

Conclusions

To date, external validation efforts for AI screening mammographic technologies suggest small potential diagnostic accuracy improvements but have been retrospective in nature and suffer from risk for bias and applicability concerns.

In breast cancer, high levels of homeobox protein Hox-B13 (HOXB13) have been associated with disease progression of ER-positive breast cancer patients and resistance to tamoxifen treatment. Since HOXB13 p.G84E is a prostate cancer risk allele, we evaluated the association between HOXB13 germline mutations and breast cancer risk in a previous study consisting of 3,270 familial non-BRCA1/2 breast cancer cases and 2,327 controls from the Netherlands. Although both recurrent HOXB13 mutations p.G84E and p.R217C were not associated with breast cancer risk, the risk estimation for p.R217C was not very precise. To provide more conclusive evidence regarding the role of HOXB13 in breast cancer susceptibility, we here evaluated the association between HOXB13 mutations and increased breast cancer risk within 81 studies of the international Breast Cancer Association Consortium containing 68,521 invasive breast cancer patients and 54,865 controls. Both HOXB13 p.G84E and p.R217C did not associate with the development of breast cancer in European women, neither in the overall analysis (OR = 1.035, 95% CI = 0.859-1.246, P = 0.718 and OR = 0.798, 95% CI = 0.482-1.322, P = 0.381 respectively), nor in specific high-risk subgroups or breast cancer subtypes. Thus, although involved in breast cancer progression, HOXB13 is not a material breast cancer susceptibility gene.

Background

Ductal carcinoma in situ (DCIS) incidence has grown with the implementation of screening and its detection varies across International Cancer Screening Network (ICSN) countries. The aim of this survey is to describe the management of screen-detected DCIS in ICSN countries and to evaluate the potential for treatment related morbidity.

Methods

We sought screen-detected DCIS data from the ICSN countries identified during 2004-2008. We adopted standardised data collection forms and analysis and explored DCIS diagnosis and treatment processes ranging from pre-operative diagnosis to type of surgery and radiotherapy.

Results

Twelve countries contributed data from a total of 15 screening programmes, all from Europe except the United States of America and Japan. Among women aged 50-69 years, 7,176,050 screening tests and 5324 screen-detected DCIS were reported. From 21% to 93% of DCIS had a pre-operative diagnosis (PO); 67-90% of DCIS received breast conservation surgery (BCS), and in 41-100% of the cases this was followed by radiotherapy; 6.4-59% received sentinel lymph node biopsy (SLNB) only and 0.8-49% axillary dissection (ALND) with 0.6% (range by programmes 0-8.1%) being node positive. Among BCS patients 35% received SLNB only and 4.8% received ALND. Starting in 2006, PO and SLNB use increased while ALND remained stable. SLNB and ALND were associated with larger size and higher grade DCIS lesions.

Conclusions

Variation in DCIS management among screened women is wide and includes lymph node surgery beyond what is currently recommended. This indicates the presence of varying levels of overtreatment and the potential for its reduction.

Background

There is concern about detection of ductal carcinoma in situ (DCIS) in screening mammography. DCIS accounts for a substantial proportion of screen-detected lesions but its effect on breast cancer mortality is debated. The International Cancer Screening Network conducted a comparative analysis to determine variation in DCIS detection.

Patients and methods

Data were collected during 2004-2008 on number of screening examinations, detected breast cancers, DCIS cases and Globocan 2008 breast cancer incidence rates derived from national or regional cancer registers. We calculated screen-detection rates for breast cancers and DCIS.

Results

Data were obtained from 15 screening settings in 12 countries; 7,176,050 screening examinations; 29,605 breast cancers and 5324 DCIS cases. The ratio between highest and lowest breast cancer incidence was 2.88 (95% confidence interval (CI) 2.76-3.00); 2.97 (95% CI 2.51-3.51) for detection of breast cancer; and 3.49 (95% CI 2.70-4.51) for detection of DCIS.

Conclusions

Considerable international variation was found in DCIS detection. This variation could not be fully explained by variation in incidence nor in breast cancer detection rates. It suggests the potential for wide discrepancies in management of DCIS resulting in overtreatment of indolent DCIS or undertreatment of potentially curable disease. Comprehensive cancer registration is needed to monitor DCIS detection. Efforts to understand discrepancies and standardise management may improve care.