Deep learning, data ramping, and uncertainty estimation for detecting artifacts in large, imbalanced databases of MRI images

Abstract

Magnetic resonance imaging (MRI) is increasingly being used to delineate morphological changes underlying neurological disorders. Successfully detecting these changes depends on the MRI data quality. Unfortunately, image artifacts frequently compromise the MRI utility, making it critical to screen the data. Currently, quality assessment requires visual inspection, a time-consuming process that suffers from inter-rater variability. Automated methods to detect MRI artifacts could improve the efficiency of the process. Such automated methods have achieved high accuracy using small datasets, with balanced proportions of MRI data with and without artifacts. With the current trend towards big data in neuroimaging, there is a need for automated methods that achieve accurate detection in large and imbalanced datasets. Deep learning (DL) is the ideal MRI artifact detection algorithm for large neuroimaging databases. However, the inference generated by DL does not commonly include a measure of uncertainty. Here, we present the first stochastic DL algorithm to generate automated, high-performing MRI artifact detection implemented on a large and imbalanced neuroimaging database. We implemented Monte Carlo dropout in a 3D AlexNet to generate probabilities and epistemic uncertainties. We then developed a method to handle class imbalance, namely data-ramping to transfer the learning by extending the dataset size and the proportion of the artifact-free data instances. We used a 34,800 scans (98% clean) dataset. At baseline, we obtained 89.3% testing accuracy (F1 = 0.230). Following the transfer learning (with data-ramping), we obtained 94.9% testing accuracy (F1 = 0.357) outperforming focal cross-entropy (92.9% testing accuracy, F1 = 0.304) incorporated for comparison at handling class imbalance. By implementing epistemic uncertainties, we improved the testing accuracy to 99.5% (F1 = 0.834), outperforming the results obtained in previous comparable studies. In addition, we estimated aleatoric uncertainties by incorporating random flips to the MRI volumes, and demonstrated that aleatoric uncertainty can be implemented as part of the pipeline. The methods we introduce enhance the efficiency of managing large databases and the exclusion of artifact images from big data analyses. HighlightsO_LIWe address the difficulty in automatically detecting artifacts in a large, imbalanced image database. C_LIO_LIWe reproduced high accuracy in detecting artifacts in small balanced datasets. C_LIO_LIWe implemented Monte Carlo (MC) dropout in 3D AlexNet to generate uncertainty metrics. C_LIO_LIWe transfer the learning from a small balanced dataset to a large imbalanced dataset, outperforming focal loss. C_LIO_LIWe outperformed artifact detection in previous studies using comparable large imbalanced datasets C_LIO_LIThe user can select an operating point to consider manual inspection vs. artifact detection error. C_LI