Using experimental data as a voucher for study pre-registration

Abstract

Undisclosed exploitation of flexibility in data acquisition and analysis blurs the important distinction between exploratory and hypothesis-driven findings and inflates false-positive rates1-4. Indeed, recent replication attempts have revealed low levels of replicability, pointing to high rates of false-positives in the literature5-10. A contemporary solution to this problem is pre-registration: commitment to aspects of methods and analysis before data acquisition11. This solution is valid only to the extent that the commitment stage is time-locked to precede data collection. To date, time-locking can only be guaranteed by introducing a third party such as peer reviewers at an early stage, making this solution less appealing for many12. Here we adapt a cryptographic method13 to encode information of study protocol within random aspects of the data acquisition process. This way, the structure of variability in the data time-locks the commitment stage with respect to data acquisition. Being independent of any third party, this method fully preserves scientific autonomy and confidentiality. We provide code for easy implementation and a detailed example from the field of functional Magnetic Resonance Imaging (fMRI).