Hood, A. S. C. | bioRxiv Science

bioRxiv2020

The data-index: an author-level metric that values impactful data and incentivises data sharing

Author-level metrics are a widely used measure of scientific success. The h-index, and its variants, measure publication output (number of publications) and impact (number of citations), and these are often used to allocate funding or jobs. Here we argue that the emphasis on publication output and impact hinders progress in the fields of ecology and evolution as it disincentivises two fundamental practices: generating long-term datasets and sharing data. We describe a new author-level metric, the data-index, which values dataset output and impact and promotes generating and sharing data as a result. It is designed to complement other metrics of scientific success, as scientific contributions are diverse and our value system should reflect that. Future work should focus on designing alternative metrics that value our wider merits, such as communicating our research, informing policy, mentoring other scientists, and providing open-access code and tools.

scientific communication and education↗

bioRxiv2020

Dynamic meta-analysis: a method of using global evidence for local decision making

Meta-analysis is often used to make generalizations across all available evidence at the global scale. But how can these global generalizations be used for evidence-based decision making at the local scale, if only the local evidence is perceived to be relevant to a local decision? We show how an interactive method of meta-analysis -- dynamic meta-analysis -- can be used to assess the local relevance of global evidence. We developed Metadataset (www.metadataset.com) as an example of dynamic meta-analysis. Using Metadataset, we show how evidence can be filtered and weighted, and results can be recalculated, using dynamic methods of subgroup analysis, meta-regression, and recalibration. With an example from agroecology, we show how dynamic meta-analysis could lead to different conclusions for different subsets of the global evidence. Dynamic meta-analysis could also lead to a rebalancing of power and responsibility in evidence synthesis, since evidence users would be able to make decisions that are typically made by systematic reviewers -- decisions about which studies to include (e.g., critical appraisal) and how to handle missing or poorly reported data (e.g., sensitivity analysis). We suggest that dynamic meta-analysis could be scaled up and used for subject-wide evidence synthesis in several scientific disciplines (e.g., agroecology and conservation biology). However, the metadata that are used to filter and weight the evidence would need to be standardized within disciplines.

ecology↗