Increasing discovery rates in preclinical research through optimised statistical decision criteria

Abstract

The success of scientific discovery in preclinical research is based on the different roles of exploration and confirmation. Exploration involves identifying potential effects (high sensitivity), which are then tested more rigorously during confirmation (high specificity). Here, we examine different experimental strategies and their ability to balance sensitivity and specificity to identify relevant effects. In simulations based on empirical data, we specifically compare a conventional p-value based approach with a method based on an a priori determined smallest effect size of interest (SESOI). Using a SESOI increases transition rates from exploration to confirmation and leads to higher detection rates across the trajectory. In particular, specificity in the SESOI trajectory increases if number of true effects are low. We conclude that employing a SESOI is superior to a p-value based approach in many contexts. Based on our findings, we propose a reconsideration of planning and conducting preclinical experiments, especially when the prior probability of true hypotheses is low.