Performance Comparison of Computational Prediction Methods for the Function and Pathogenicity of Non-coding Variants

Abstract

Non-coding variants in the human genome greatly influence some traits and complex diseases by their own regulation and modification effects. Hence, an increasing number of computational methods are developed to predict the effects of variants in the human non-coding sequences. However, it is difficult for users with insufficient knowledge about the performances of computational methods to select appropriate computational methods from dozens of methods. In order to solve this problem, we assessed 12 performance measures of 24 methods on four independent non-coding variant benchmark datasets: (I) rare germline variant from ClinVar, (II) rare somatic variant from COSMIC, (III) common regulatory variant dataset, and (IV) disease associated common variant dataset. All 24 tested methods performed differently under various conditions, indicating that these methods have varying strengths and weaknesses under different scenarios. Importantly, the performance of existing methods was acceptable in the rare germline variant from ClinVar with area under curves (AUCs) of 0.4481 - 0.8033 and poor in the rare somatic variant from COSMIC (AUCs: 0.4984 - 0.7131), common regulatory variant dataset (AUCs: 0.4837 - 0.6472), and disease associated common variant dataset (AUCs: 0.4766 -0.5188). We also compared the prediction performance among 24 methods for non-coding de novo mutations in autism spectrum disorder and found that the CADD and CDTS methods showed better performance. Summarily, we assessed the performances of 24 computational methods under diverse scenarios, providing preliminary advice for proper tool selection and new method development in interpreting non-coding variants.