Biology subjects

Puljak, L.

Publications and source records attributed to Puljak, L..

3 recordsLinked to original sources

bioRxiv2018

Judgments of other bias in Cochrane systematic reviews of interventions are highly inconsistent and thus hindering use and comparability of evidence

BackgroundClinical decisions are made based on Cochrane systematic reviews (CSRs), but implementation of results of evidence syntheses such as CSRs is problematic if the evidence is not prepared consistently. All systematic reviews should assess risk of bias (RoB) in included studies, and in CSRs this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study we aimed to analyze types and judgments of other bias in the RoB tool in CSRs of interventions.\n\nMethodsWe analyzed CSRs that included randomized controlled trials (RCTs) and extracted data regarding other bias from the RoB table and accompanying support for the judgment. We categorized different types of other bias.\n\nResultsWe analyzed 768 CSRs that included 11369 RCTs. There were 602 (78%) CSRs that had other bias domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 CSRs for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 CSRs that described various sources of other bias there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. Furthermore, we found numerous other inconsistencies in reporting of sources of other bias in CSRs.\n\nConclusionCochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Furthermore, discrepant and erroneous judgments of bias in evidence synthesis will inevitably hinder implementation of evidence in routine clinical practice and reduce confidence of practitioners in otherwise trustworthy sources of information.

epidemiology↗

bioRxiv2018

Risk of bias in Cochrane systematic reviews: assessments of risk related to attrition bias are highly inconsistent

BackgroundAn important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews (CSRs) are considered golden standard regarding systematic review methodology, but Cochranes instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in CSRs of interventions published in the Cochrane Database of Systematic Reviews (CDSR).\n\nMethodsWe analyzed CSRs published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many CSRs had different judgments for the same supporting explanations.\n\nResultsIn the main analysis we included 10292 judgments and supporting explanations for attrition bias from 729 CSRs. We categorized supporting explanations for those judgments into four categories and we found that most of the supporting explanations were unclear. Numerical indicators for percent of attrition, as well as statistics related to attrition were judged very differently. One third of CSR authors had more than one category of supporting explanation; some had up to four different categories. Inconsistencies were found even with the number of judgments, names of risk of bias domains and different judgments for the same supporting explanations in the same CSR.\n\nConclusionWe found very high inconsistency in methods of appraising risk of attrition bias in recent Cochrane reviews. Systematic review authors need clear guidance about different categories they should assess and judgments for those explanations. Clear instructions about appraising risk of attrition bias will improve reliability of the Cochranes risk of bias tool, help authors in making decisions about risk of bias and help in making reliable decisions in healthcare.

epidemiology↗

bioRxiv2018

Judgments of risk of bias associated with random sequence generation in trials included in Cochrane systematic reviews are frequently erroneous

BackgroundPurpose of this study was to analyze adequacy of judgments about risk of bias (RoB) for random sequence generation in Cochrane systematic reviews (CSRs) of randomized controlled trials (RCTs).\n\nMethodsInformation was extracted from RoB tables of CSRs using automated data scraping. We categorized all comments provided as supports for judgments for RoB related to randomization. We analyzed number and type of various supporting comments and assessed adequacy of RoB judgment for randomization in line with recommendations from the Cochrane Handbook.\n\nResultsWe analyzed 10527 RCTs that were included in 729 CSRs. For 5682 RCTs randomization was not described; for the others it was indicated randomization was done using computer/software/internet (N=2886), random number table (N=888), mechanic method (N=366), or it was incomplete/inappropriate (N=303).\n\nOverall, 1194/10125 trials (12%) had erroneous RoB judgment about randomization. The highest proportion of errors was found for trials with high RoB (28%), followed by those with low (19%), or unclear (3%). Therefore, one in eight judgments for the analyzed domain in CSRs was erroneous, and one in three if the judgment was \"high risk\".\n\nConclusionCochrane systematic reviews cannot be necessarily trusted when it comes to judgments for risk of bias related to randomized sequence generation.

epidemiology↗