Purpose

Domain knowledge (DK) acquired from prior studies is important for medical diagnosis. This paper leverages the population-level DK using an optimality design criterion to train a deep learning model in an end-to-end manner. In this study, the problem of interest is at the patient level to diagnose a subject with idiopathic pulmonary fibrosis (IPF) among subjects with interstitial lung disease (ILD) using a computed tomography (CT). IPF diagnosis is a complicated process with multidisciplinary discussion with experts and is subject to interobserver variability, even for experienced radiologists. To this end, we propose a new statistical method to construct a time/memory-efficient IPF diagnosis model using axial chest CT and DK, along with an optimality design criterion via a DK-enhanced loss function of deep learning.

Methods

Four state-of-the-art two-dimensional convolutional neural network (2D-CNN) architectures (MobileNet, VGG16, ResNet-50, and DenseNet-121) and one baseline 2D-CNN are implemented to automatically diagnose IPF among ILD patients. Axial lung CT images are retrospectively acquired from 389 IPF patients and 700 non-IPF ILD patients in five multicenter clinical trials. To enrich the sample size and boost model performance, we sample 20 three-slice samples (triplets) from each CT scan, where these three slices are randomly selected from the top, middle, and bottom of both lungs respectively. Model performance is evaluated using a fivefold cross-validation, where each fold was stratified using a fixed proportion of IPF vs non-IPF.

Results

Using DK-enhanced loss function increases the model performance of the baseline CNN model from 0.77 to 0.89 in terms of study-wise accuracy. Four other well-developed models reach satisfactory model performance with an overall accuracy >0.95 but the benefits brought on by the DK-enhanced loss function is not noticeable.

Conclusions

We believe this is the first attempt that (a) uses population-level DK with an optimal design criterion to train deep learning-based diagnostic models in an end-to-end manner and (b) focuses on patient-level IPF diagnosis. Further evaluation of using population-level DK on prospective studies is warranted and is underway.

Background

Idiopathic pulmonary fibrosis (IPF) is a fatal interstitial lung disease characterized by an unpredictable decline in lung function. Predicting IPF progression from the early changes in lung function tests have known to be a challenge due to acute exacerbation. Although it is unpredictable, the neighboring regions of fibrotic reticulation increase during IPF's progression. With this clinical information, quantitative characteristics of high-resolution computed tomography (HRCT) and a statistical learning paradigm, the aim is to build a model to predict IPF progression.

Design

A paired set of anonymized 193 HRCT images from IPF subjects with 6-12 month intervals were collected retrospectively. The study was conducted in two parts: (1) Part A collects the ground truth in small regions of interest (ROIs) with labels of "expected to progress" or "expected to be stable" at baseline HRCT and develop a statistical learning model to classify voxels in the ROIs. (2) Part B uses the voxel-level classifier from Part A to produce whole-lung level scores of a single-scan total probability's (STP) baseline.

Methods

Using annotated ROIs from 71 subjects' HRCT scans in Part A, we applied Quantum Particle Swarm Optimization-Random Forest (QPSO-RF) to build the classifier. Then, 122 subjects' HRCT scans were used to test the prediction. Using Spearman rank correlations and survival analyses, we ascertained STP associations with 6-12 month changes in quantitative lung fibrosis and forced vital capacity.

Conclusion

This study can serve as a reference for collecting ground truth, and developing statistical learning techniques to predict progression in medical imaging.

To translate artificial intelligence (AI) algorithms into clinical practice requires generalizability of models to real-world data. One of the main obstacles to generalizability is data shift, a data distribution mismatch between model training and real environments. Explainable AI techniques offer tools to detect and mitigate the data shift problem and develop reliable AI for clinical practice. Most medical AI is trained with datasets gathered from limited environments, such as restricted disease populations and center-dependent acquisition conditions. The data shift that commonly exists in the limited training set often causes a significant performance decrease in the deployment environment. To develop a medical application, it is important to detect potential data shift and its impact on clinical translation. During AI training stages, from premodel analysis to in-model and post hoc explanations, explainability can play a key role in detecting model susceptibility to data shift, which is otherwise hidden because the test data have the same biased distribution as the training data. Performance-based model assessments cannot effectively distinguish the model overfitting to training data bias without enriched test sets from external environments. In the absence of such external data, explainability techniques can aid in translating AI to clinical practice as a tool to detect and mitigate potential failures due to data shift. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

Background

Idiopathic pulmonary fibrosis (IPF) is a progressive, irreversible, and usually fatal lung disease of unknown reasons, generally affecting the elderly population. Early diagnosis of IPF is crucial for triaging patients' treatment planning into anti-fibrotic treatment or treatments for other causes of pulmonary fibrosis. However, current IPF diagnosis workflow is complicated and time-consuming, which involves collaborative efforts from radiologists, pathologists, and clinicians and it is largely subject to inter-observer variability.

Purpose

The purpose of this work is to develop a deep learning-based automated system that can diagnose subjects with IPF among subjects with interstitial lung disease (ILD) using an axial chest computed tomography (CT) scan. This work can potentially enable timely diagnosis decisions and reduce inter-observer variability.

Methods

Our dataset contains CT scans from 349 IPF patients and 529 non-IPF ILD patients. We used 80% of the dataset for training and validation purposes and 20% as the holdout test set. We proposed a two-stage model: at stage one, we built a multi-scale, domain knowledge-guided attention model (MSGA) that encouraged the model to focus on specific areas of interest to enhance model explainability, including both high- and medium-resolution attentions; at stage two, we collected the output from MSGA and constructed a random forest (RF) classifier for patient-level diagnosis, to further boost model accuracy. RF classifier is utilized as a final decision stage since it is interpretable, computationally fast, and can handle correlated variables. Model utility was examined by (1) accuracy, represented by the area under the receiver operating characteristic curve (AUC) with standard deviation (SD), and (2) explainability, illustrated by the visual examination of the estimated attention maps which showed the important areas for model diagnostics.

Results

During the training and validation stage, we observe that when we provide no guidance from domain knowledge, the IPF diagnosis model reaches acceptable performance (AUC±SD = 0.93±0.07), but lacks explainability; when including only guided high- or medium-resolution attention, the learned attention maps are not satisfactory; when including both high- and medium-resolution attention, under certain hyperparameter settings, the model reaches the highest AUC among all experiments (AUC±SD = 0.99±0.01) and the estimated attention maps concentrate on the regions of interests for this task. Three best-performing hyperparameter selections according to MSGA were applied to the holdout test set and reached comparable model performance to that of the validation set.

Conclusions

Our results suggest that, for a task with only scan-level labels available, MSGA+RF can utilize the population-level domain knowledge to guide the training of the network, which increases both model accuracy and explainability.