bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/066803

A statistical definition for reproducibility and replicability

Patil, P.·Peng, R.·Leek, J.
View original source ↗Export metadata ↓

Abstract

Everyone agrees that reproducibility and replicability are fundamental characteristics of scientific studies. These topics are attracting increasing attention, scrutiny, and debate both in the popular press and the scientific literature. But there are no formal statistical definitions for these concepts, which leads to confusion since the same words are used for different concepts by different people in different fields. We provide formal and informal definitions of scientific studies, reproducibility, and replicability that can be used to clarify discussions around these concepts in the scientific and popular press.

Explore related subjects

Scientific Communication and EducationPreprint

Keep this discovery

BibTeXRIS

Patil, P., Peng, R., Leek, J.. 2016-07-29. A statistical definition for reproducibility and replicability. https://doi.org/10.1101/066803

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2016

Bayesian Analysis of High Throughput Data

Duplicate or triplicate experimental replicates are commonplace in the high throughput literature. However, it has not been tested whether this is statistically defensible or not. To address this issue, we use probabilistic programming to develop a simple hierarchical model for analyzing high throughput measurement data. With the model and simulated data, we show that a small increase in replicate experiments can quantitatively improve accuracy in measurement. We also provide posterior densities for statistical parameters used in the evaluation of HT data. Finally, we provide an extensible open source implementation that ingests data structured in a simple format and produces posterior densities of estimated measurement and assay evaluation parameters.

Scientific Communication and Education↗
bioRxiv2016

Reproducibility and replicability of rodent phenotyping in preclinical studies

The scientific community is increasingly concerned with cases of published \"discoveries\" that are not replicated in further studies. The field of mouse behavioral phenotyping was one of the first to raise this concern, and to relate it to other complicated methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of the mouse as a model animal for human health and disease mechanisms. In January 2015, researchers from various disciplines including genetics, behavior genetics, neuroscience, ethology, statistics and bioinformatics gathered in Tel Aviv University to discuss these issues. The general consent presented here was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, were agreed by all to have a key role to play in identifying specific problems and promoting effective solutions. As replicability is related to validity and may also affect generalizability and translation of findings, the implications of the present discussion reach far beyond the issue of replicability of mouse phenotypes but may be highly relevant throughout biomedical research.

Scientific Communication and Education↗
bioRxiv2016

Identification of Successful Mentoring Communities using Network-based Analysis of Mentor-Mentee Relationships across Nobel Laureates

Skills underlying scientific innovation and discovery generally develop within an academic community, often beginning with a graduate mentors laboratory. In this paper, a network analysis of doctoral student-dissertation advisor relationships in The Academic Tree is used to identify successful mentoring communities in high-level science, as measured by number of Nobel laureates within the community. Nobel laureates form a distinct group in the network with greater numbers of Nobel laureate ancestors, descendants, mentees/grandmentees, and local academic family. Subnetworks composed entirely of Nobel laureates extend across as many as four generations. Successful historical mentoring communities were identified centering around Cambridge University in the latter 19th century and Columbia University in the early 20th century. The current practice of building web-based academic networks, extended to include a wider variety of measures of academic success, would allow for the identification of modern successful scientific communities and should be promoted.

Scientific Communication and Education↗