A Comprehensive Evaluation of Methods for Mendelian Randomization Using Realistic Simulations of Genome-wide Association Studies

Abstract

BackgroundMendelian randomization (MR) has provided major opportunities for understanding the causal relationship among complex traits. Previous studies have often evaluated MR methods based on simulations that do not adequately reflect the data-generating mechanism in GWAS and there are often discrepancies in performance of MR methods in simulations and real datasets. MethodsWe use a simulation framework that generates data on full GWAS for two traits under realistic model for effect-size distribution coherent with heritability, co-heritability and polygenicity typically observed for complex traits. We further use recent data generated from GWAS of 38 biomarkers in the UK Biobank to investigate their causal effects on risk of type-2 diabetes using externally available GWAS summary-statistics. ResultsSimulation studies show that weighted mode and MRMix are the only two methods which maintain correct type-I error rate in a diverse set of scenarios. Between the two methods, MRMix tends to be more powerful for larger GWAS while the opposite being true for smaller sample sizes. Among the other methods, random-effect IVW, MR-Robust and MR-RAPS tend to perform best in maintaining low mean squared error when the InSIDE assumption is satisfied, but can produce large bias when InSIDE is violated. In real data analysis, some biomarkers showed major heterogeneity in estimates of their causal effects on risk of type-2 diabetes across the different methods, with patterns similar to those observed in simulation studies. ConclusionsRelative performance of different MR methods depends heavily on sample sizes of underlying GWAS, proportion of valid instruments and validity of the InSIDE assumption. Key MessagesO_LIMany previous simulations studies to evaluate Mendelian randomization methods do not adequately reflect the data-generating mechanism of genome-wide association studies (GWAS). C_LIO_LIWe use a simulation framework that generates data on full GWASs under realistic model informed by recent studies on effect-size distribution. We also used very recent GWAS data available on a large number of biomarkers to evaluate their causal effect on type-2 diabetes using alternative methods. C_LIO_LIAmong the 10 methods that were compared, relative performance of different methods depends heavily on sample sizes of underlying GWAS, proportion of valid instruments and validity of the InSIDE assumption. C_LIO_LIWeighted mode and MRMix are the only two methods that maintain correct type I error rate in a diverse set of scenarios. C_LI