bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/100081

An index of the quality of evidence in meta-analysis of randomized controlled trials

Brewer, D. D.
View original source ↗Export metadata ↓

Abstract

The quality of evidence in meta-analysis of randomized controlled trials is the degree to which the estimated effect represents the \"truth.\" Current approaches to assessing the quality of evidence focus on trial design and methods. I describe a new quality of evidence index composed of four sub-indexes that measure pre-registration, independent replication, data availability, and trial design and methods, respectively. This index is systematic, objective, and quantitative. I illustrate the index with an empirical example and provide a spreadsheet for easy calculation.\n\nO_QD\"...when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.\"\n\n-Lord Kelvin (William Thompson), 1824-1907\n\nC_QD

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

BibTeXRIS

Brewer, D. D.. 2017-01-13. An index of the quality of evidence in meta-analysis of randomized controlled trials. https://doi.org/10.1101/100081

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2018

Fair allocation of healthcare research funds by the European Union?

This study aimed to investigate the distribution of European Union (EU) healthcare research grants across EU countries, and to study the effect of the potential influencing factors on grant allocation. We analysed publicly available data on healthcare research grants from the 7th Framework Programme and the Horizon 2020 Programme allocated to beneficiaries between 2007 and 2016. Grant allocation was analysed at the beneficiary-, country-, and country group-level (EU-15 versus newer Member States, defined as EU-13). The investigated country-level explanatory variables included GDP per capita, population size, overall disease burden, and healthcare research excellence. Grant amounts per 100,000 inhabitants was used as an outcome variable in the regression analyses.\n\nResearch funds were disproportionally allocated to EU-15 versus the EU-13, as 96.9% of total healthcare grants were assigned to EU-15 countries. At the beneficiary level, EU funding was positively influenced by participating in previous grants. The average grant amount per beneficiary was higher for EU-15 organizations. In univariate regression analyses at the country level, higher GDP per capita (p<0.001) and better medical research excellence (p<0.001) were associated with more EU funding, and a higher disease burden was associated with less EU funding (p=0.003). In the multiple regression analysis GDP per capita (p=0.002) and research excellence (p<0.001) had a significant positive association with EU funding. Population size had an inverted U-shaped relationship with EU funding for healthcare research, having the largest per capita funding in second and the third quartiles (p=0.03 and p=0.02).\n\nThe uneven allocation of healthcare research funds across EU countries was influenced by GDP per capita, medical research excellence and population size. Wealthier countries with an average population size and strong research excellence in healthcare had more EU funding for healthcare research. Higher disease burden apparently was not associated with more EU research funding.

scientific communication and education↗
bioRxiv2018

Authorship inflation and author gender in pulmonology research.

IntroductionHonorary authorship and equal gender representation are two pressing matters in scientific research. Honorary authorship is the inclusion of authors who do not meet the criteria established by the International Committee of Medical Journal Editors (ICMJE) authorship guidelines. The inclusion of honorary authors in the medical literature has led to an increase of the number of authors on studies and a decrease in single author studies in various fields.\n\nMethodsOur primary objective was to assess authorship trends in two major pulmonology journals (selected on the basis of Google Scholar rankings): Thorax and American Journal of Respiratory and Critical Care Medicine. We reviewed all articles published in both journals in the years 1994, 2004, and 2014 using Web of Science and extracted data such as number of authors and gender of the first and last authors.\n\nResultsThe total number of authors steadily increased from 1994 to 2014. The median number of authors grew from about four in 1994 to nearly seven in 2014, which is approximately a 75% increase. When we compiled all the data, we found the percentage of female authors from both journals had increased from 17% to 29.9% during the study period.\n\nDiscussionWe found an increase in the average number of authors on pulmonology publications between 1994 and 2014 as well as an increase in the number of females with a lead or main author position. This may be due to a variety of factors, such as increased team science. However, our data in conjunction with data from other areas of medicine, indicate that honorary authorship may be contributing to the trends we identified.

scientific communication and education↗