A simple proposal for the publication of journal citation distributions

Vincent Lariviere; Veronique Kiermer; Catriona J MacCallum; Marcia McNutt; Mark Patterson; Bernd Pulverer; Sowmya Swaminathan; Stuart Taylor; Stephen Curry

doi:10.1101/062109

bioRxiv · 10.1101/062109

A simple proposal for the publication of journal citation distributions

Vincent Lariviere·Veronique Kiermer·Catriona J MacCallum·Marcia McNutt·Mark Patterson·Bernd Pulverer·Sowmya Swaminathan·Stuart Taylor·Stephen Curry

View original source ↗Export metadata ↓

Abstract

Although the Journal Impact Factor (JIF) is widely acknowledged to be a poor indicator of the quality of individual papers, it is used routinely to evaluate research and researchers. Here, we present a simple method for generating the citation distributions that underlie JIFs. Application of this straightforward protocol reveals the full extent of the skew of distributions and variation in citations received by published papers that is characteristic of all scientific journals. Although there are differences among journals across the spectrum of JIFs, the citation distributions overlap extensively, demonstrating that the citation performance of individual papers cannot be inferred from the JIF. We propose that this methodology be adopted by all journals as a move to greater transparency, one that should help to refocus attention on individual pieces of work and counter the inappropriate usage of JIFs during the process of research assessment.

Explore related subjects

Scientific Communication and Education Preprint

Keep this discovery

BibTeX RIS

Vincent Lariviere, Veronique Kiermer, Catriona J MacCallum, Marcia McNutt, Mark Patterson, Bernd Pulverer, Sowmya Swaminathan, Stuart Taylor, Stephen Curry. 2016-07-05. A simple proposal for the publication of journal citation distributions. https://doi.org/10.1101/062109

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2016

Bayesian Analysis of High Throughput Data

Duplicate or triplicate experimental replicates are commonplace in the high throughput literature. However, it has not been tested whether this is statistically defensible or not. To address this issue, we use probabilistic programming to develop a simple hierarchical model for analyzing high throughput measurement data. With the model and simulated data, we show that a small increase in replicate experiments can quantitatively improve accuracy in measurement. We also provide posterior densities for statistical parameters used in the evaluation of HT data. Finally, we provide an extensible open source implementation that ingests data structured in a simple format and produces posterior densities of estimated measurement and assay evaluation parameters.

Scientific Communication and Education↗

bioRxiv2016

Reproducibility and replicability of rodent phenotyping in preclinical studies

The scientific community is increasingly concerned with cases of published \"discoveries\" that are not replicated in further studies. The field of mouse behavioral phenotyping was one of the first to raise this concern, and to relate it to other complicated methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of the mouse as a model animal for human health and disease mechanisms. In January 2015, researchers from various disciplines including genetics, behavior genetics, neuroscience, ethology, statistics and bioinformatics gathered in Tel Aviv University to discuss these issues. The general consent presented here was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, were agreed by all to have a key role to play in identifying specific problems and promoting effective solutions. As replicability is related to validity and may also affect generalizability and translation of findings, the implications of the present discussion reach far beyond the issue of replicability of mouse phenotypes but may be highly relevant throughout biomedical research.

Scientific Communication and Education↗

bioRxiv2016

Identification of Successful Mentoring Communities using Network-based Analysis of Mentor-Mentee Relationships across Nobel Laureates

Skills underlying scientific innovation and discovery generally develop within an academic community, often beginning with a graduate mentors laboratory. In this paper, a network analysis of doctoral student-dissertation advisor relationships in The Academic Tree is used to identify successful mentoring communities in high-level science, as measured by number of Nobel laureates within the community. Nobel laureates form a distinct group in the network with greater numbers of Nobel laureate ancestors, descendants, mentees/grandmentees, and local academic family. Subnetworks composed entirely of Nobel laureates extend across as many as four generations. Successful historical mentoring communities were identified centering around Cambridge University in the latter 19th century and Columbia University in the early 20th century. The current practice of building web-based academic networks, extended to include a wider variety of measures of academic success, would allow for the identification of modern successful scientific communities and should be promoted.

Scientific Communication and Education↗