Systematic variability enhances the reproducibility of an ecological study

Abstract

Many scientific disciplines currently are experiencing a \"reproducibility crisis\" because numerous scientific findings cannot be repeated consistently. A novel but controversial hypothesis postulates that stringent levels of environmental and biotic standardization in experimental studies reduces reproducibility by amplifying impacts of lab-specific environmental factors not accounted for in study designs. A corollary to this hypothesis is that the deliberate introduction of controlled systematic variability (CSV) in experimental designs can increase reproducibility. We tested this hypothesis using a multi-laboratory microcosm study in which the same ecological experiment was repeated in 14 laboratories across Europe. Each laboratory introduced environmental and genotypic CSV within and among replicated microcosms established in either growth chambers (with stringent control of environmental conditions) or glasshouses (with more variable environmental conditions). The introduction of genotypic CSV led to lower among-laboratory variability in growth chambers, indicating increased reproducibility, but had no significant effect in glasshouses where reproducibility also was lower. Environmental CSV had little effect on reproducibility. Although there are multiple causes for the \"reproducibility crisis\", deliberately including genetic variation may be a simple solution for increasing the reproducibility of ecological studies performed in controlled environments.