Probabilistic Brain MR Image Transformation Using Generative Models

Abstract

Brain MR image transformation, which is the process of transforming one type of MR image into another, is a critical neuroimaging task that is needed when the target image type is missing or corrupted. Accordingly, several methods have been developed to tackle this problem, with a recent focus on deep learning-based models. In this paper, we investigate the performance of the conditional version of three such probabilistic generative models, including conditional Generative Adversarial Networks (cGAN), Noise Conditioned Score Networks (NCSN), and De-noising Diffusion Probabilistic Models (DDPM). We also compare their performance against a more traditional deterministic U-Net based model. We train and test these models using MR images from publicly available datasets IXI and OASIS. For images from the IXI dataset, we conduct experiments on combinations of transformations between T1-weighted (T1), T2-weighted (T2), and proton density (PD) images, whereas for the OASIS dataset, we consider combinations of T1, T2, and Fluid Attenuated Inversion Recovery (FLAIR) images. In evaluating these models, we measure the similarity between the transformed image and the target image using metrics like PSNR and SSIM. In addition, for the three probabilistic generative models, we evaluate the utility of generating an ensemble of predictions by computing a metric that measures the variance in their predictions and demonstrate that it can be used to identify out-of-distribution (OOD) input images. We conclude that the NCSN model yields the most accurate transformations, while the DDPM model yields variance results that most clearly detect OOD inputs. We also note that while the results for the two diffusion models (NCSN and DDPM) are more accurate than those for the cGAN, the latter was significantly more efficient in generating multiple samples. Overall, our work demonstrates the utility of probabilistic conditional generative models for MR image transformation and highlights the role of generating an ensemble of outputs in identifying OOD input images.