BACKGROUND: Diagnostic interpretations of melanocytic skin lesions vary widely among pathologists, yet the underlying reasons remain unclear. OBJECTIVE: Identify pathologist characteristics associated with rates of accuracy and reproducibility. METHODS: Pathologists independently interpreted the same set of biopsy specimens from melanocytic lesions on 2 occasions. Diagnoses were categorized into 1 of 5 classes according to the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis system. Reproducibility was determined by pathologists concordance of diagnoses across 2 occasions. Accuracy was defined by concordance with a consensus reference standard. Associations of pathologist characteristics with reproducibility and accuracy were assessed individually and in multivariable logistic regression models. RESULTS: Rates of diagnostic reproducibility and accuracy were highest among pathologists with board certification and/or fellowship training in dermatopathology and in those with 5 or more years of experience. In addition, accuracy was high among pathologists with a higher proportion of melanocytic lesions in their caseload composition and higher volume of melanocytic lesions. LIMITATIONS: Data gathered in a test set situation by using a classification tool not currently in clinical use. CONCLUSION: Diagnoses are more accurate among pathologists with specialty training and those with more experience interpreting melanocytic lesions. These findings support the practice of referring difficult cases to more experienced pathologists to improve diagnostic accuracy, although the impact of these referrals on patient outcomes requires additional research.

Background

Many circulating biomarkers have been reported for the diagnosis of breast cancer, but few, if any, have undergone rigorous credentialing using prospective cohorts and blinded evaluation.

Methods

The NCI Early Detection Research Network (EDRN) has created a prospective, multicenter collection of plasma and serum samples from 832 subjects designed to evaluate circulating biomarkers for the detection and diagnosis of breast cancer. These samples are available to investigators who wish to evaluate their biomarkers using a set of blinded samples. The breast cancer reference set is composed of blood samples collected using a standard operating procedure at four U.S. medical centers from 2008 to 2010 from women undergoing either tissue diagnosis for breast cancer or routine screening mammography. The reference set contains samples from women with incident invasive cancer (n = 190), carcinoma in situ (n = 55), benign pathology with atypia (n = 63), benign disease with no atypia (n = 231), and women with no evidence of breast disease by screening mammography (BI-RADS 1 or 2, n = 276). Using a subset of plasma samples (n = 505) from the reference set, we analyzed 90 proteins by multiplexed immunoassays for their potential utility as diagnostic markers.

Results

We found that none of these markers is useful for distinguishing cancer from benign controls. However, elevated CA-125 does appear to be a candidate marker for estrogen receptor-negative cancers.

Conclusions

Markers that can distinguish benign breast conditions from invasive cancer have not yet been found.

Impact

Availability of prospectively collected samples should improve future validation efforts.

BACKGROUND: Previous studies of second opinions in the diagnosis of melanocytic skin lesions have examined blinded second opinions, which do not reflect usual clinical practice. The current study, conducted in the USA, investigated both blinded and nonblinded second opinions for their impact on diagnostic accuracy. METHODS: In total, 100 melanocytic skin biopsy cases, ranging from benign to invasive melanoma, were interpreted by 74 dermatopathologists. Subsequently, 151 dermatopathologists performed nonblinded second and third reviews. We compared the accuracy of single reviewers, second opinions obtained from independent, blinded reviewers and second opinions obtained from sequential, nonblinded reviewers. Accuracy was defined with respect to a consensus reference diagnosis. RESULTS: The mean case-level diagnostic accuracy of single reviewers was 65.3% (95% CI 63.4-67.2%). Second opinions arising from sequential, nonblinded reviewers significantly improved accuracy to 69.9% (95% CI 68.0-71.7%; P < 0.001). Similarly, second opinions arising from blinded reviewers improved upon the accuracy of single reviewers (69.2%; 95% CI 68.0-71.7%). Nonblinded reviewers were more likely than blinded reviewers to give diagnoses in the same diagnostic classes as the first diagnosis. Nonblinded reviewers tended to be more confident when they agreed with previous reviewers, even with inaccurate diagnoses. CONCLUSION: We found that both blinded and nonblinded second reviewers offered a similar modest improvement in diagnostic accuracy compared with single reviewers. Obtaining second opinions with knowledge of previous reviews tends to generate agreement among reviews, and may generate unwarranted confidence in an inaccurate diagnosis. Combining aspects of both blinded and nonblinded review in practice may leverage the advantages while mitigating the disadvantages of each approach. Specifically, a second pathologist could give an initial diagnosis blinded to the results of the first pathologist, with subsequent nonblinded discussion between the two pathologists if their diagnoses differ.

Early detection of pancreatic ductal adenocarcinoma (PDAC) is key to improving patient outcomes; however, PDAC is usually diagnosed late. Therefore, blood-based minimally invasive biomarker assays for limited volume clinical samples are urgently needed. A novel miRNA profiling platform (Abcam Fireplex-Oncology Panel) was used to investigate the feasibility of developing early detection miRNA biomarkers with 20 μL plasma from a training set (58 stage II PDAC cases and 30 controls) and two validation sets (34 stage II PDAC cases and 25 controls; 44 stage II PDAC cases and 18 controls). miR-34a-5p [AUC = 0.77; 95% confidence interval (CI), 0.66-0.87], miR-130a-3p (AUC = 0.74; 95% CI, 0.63-0.84), and miR-222-3p (AUC = 0.70; 95% CI, 0.58-0.81) were identified as significantly differentially abundant in plasma from stage II PDAC versus controls. Although none of the miRNAs individually outperformed the currently used serologic biomarker for PDAC, carbohydrate antigen 19-9 (CA19-9), combining the miRNAs with CA 19-9 improved AUCs from 0.89 (95% CI, 0.81-0.95) for CA 19-9 alone to 0.92 (95% CI, 0.86-0.97), 0.94 (95% CI, 0.89-0.98), and 0.92 (95% CI, 0.87-0.97), respectively. Gene set enrichment analyses of transcripts correlated with high and low expression of the three miRNAs in The Cancer Genome Atlas PDAC sample set. These miRNA biomarkers, assayed in limited volume plasma together with CA19-9, discriminate stage II PDAC from controls with good sensitivity and specificity. Unbiased profiling of larger cohorts should help develop an informative early detection biomarker assay for diagnostic settings. PREVENTION RELEVANCE: Development of minimally invasive biomarker assays for detection of premalignant disease and early-stage pancreatic cancer is key to improving patient survival. This study describes a limited volume plasma miRNA biomarker assay that can detect early-stage resectable pancreatic cancer in clinical samples necessary for effective prevention and clinical intervention.

Importance

The recently updated American Joint Committee on Cancer (AJCC) classification of cancer staging, the AJCC Cancer Staging Manual, 8th edition (AJCC 8), includes revisions to definitions of T1a vs T1b or greater. The Melanoma Pathology Study database affords a comparison,of pathologists' concordance and reproducibility in the microstaging of melanoma according to both the existing 7th edition (AJCC 7) and the new AJCC 8.

Objective

To compare AJCC 7 and AJCC 8 to examine whether changes to the definitions of T1a and T1b or greater are associated with changes in concordance and reproducibility.

Design setting and participants

In this diagnostic study conducted as part of the national Melanoma Pathology Study across US states, 187 pathologists interpreting melanocytic skin lesions in practice completed 4342 independent case interpretations of 116 invasive melanoma cases. A consensus reference diagnosis and participating pathologists' interpretations were classified into the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis class IV (T1a) or class V ( T1b) using both the AJCC 7 and AJCC 8 criteria.

Main outcomes and measures

Concordance with consensus reference diagnosis, interobserver reproducibility, and intraobserver reproducibility.

Results

For T1a diagnoses, participating pathologists' concordance with the consensus reference diagnosis increased from 44% (95% CI, 41%-48%) to 54% (95% CI, 51%-57%) using AJCC 7 and AJCC 8 criteria, respectively. The concordance for cases of T1b or greater increased from 72% (95% CI, 69%-75%) to 78% (95% CI, 75%-80%). Intraobserver reproducibility of diagnoses also improved, increasing from 59% (95% CI, 56%-63%) to 64% (95% CI, 62%-67%) for T1a invasive melanoma, and from 74% (95% CI, 71%-76%) to 77% (95% CI, 74%-79%) for T1b or greater invasive melanoma cases.

Conclusions and relevance

Melanoma staging in AJCC 8 shows greater reproducibility and higher concordance with a reference standard. Improved classification of invasive melanoma can be expected after implementation of AJCC 8, suggesting a positive impact on patients. However, despite improvement, concordance and reproducibility remain low.