Predictive Modeling Achieves High Test-Retest Reliability with Resting State Functional Connectivity

Abstract

Recent studies found low test-retest reliability in fMRI, raising serious concerns among researchers, but these studies mostly focused on reliability of individual fMRI features (e.g., individual connections in resting state connectivity maps). Meanwhile, neuroimaging researchers increasingly employ multivariate predictive models that aggregate information across a large number of features to predict outcomes of interest, but the test-retest reliability of predicted outcomes of these models has not previously been systematically studied. Here we apply ten predictive modeling methods to resting state connectivity maps from the Human Connectome Project dataset to predict 61 outcome variables. Compared to mean reliability of individual resting state connections, we find mean reliability of the predicted outcomes of predictive models is substantially higher for all ten modeling methods assessed. Moreover, improvement was consistently observed across all scanning and processing choices (i.e., scan lengths, censoring thresholds, volume-versus surface-based processing). For the most reliable methods, reliability of predicted outcomes was mostly, though not exclusively, in the "good" range (above 0.60). Finally, we identified three mechanisms that help to explain why predicted outcomes of predictive models have higher reliability than individual imaging features. We conclude that researchers can potentially achieve higher test-retest reliability by making greater use of predictive models.