Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews

Table 3 Characteristics of included Cochrane and non-Cochrane reviews—evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials

Publication characteristics	100 Cochrane reviews (100 %)	100 non-Cochrane reviews (100 %)	P value*
Intervention
Pharmacologic	55 (55 %)	29 (29 %)	0.020
Non-pharmacologic	45 (45 %)	71 (71 %)	0.061
Review has ≥1 meta-analysis
Yes	85 (85 %)	45 (45 %)	0.0065
Included trials
Number of randomized clinical trials in total	1242	1249
-Low^a risk of bias	74 (6 %)	25 of 424^e (6 %)	1.00
-Unclear^b risk of bias	407 (33 %)	226 of 424^e (53 %)	0.0001
-High^c risk of bias	761 (61 %)	173 of 424^e (41 %)	0.0001
Reviews with ≥1 low risk of bias trial and ≥1 high risk of bias trial	26 (26 %)	6 of 18^f (33 %)	0.60
Reviews with ≥1 low risk of bias trial and ≥1 high or unclear risk of bias trial	32 (32 %)	8 of 18^f (44 %)	0.47
Number of randomized clinical trials included^d in a review
-One to five	39 (39 %)	38 (38 %)	1.00
-Six to ten	23 (23 %)	26 (26 %)	0.75
->Ten	38 (38 %)	36 (36 %)	0.89

*P values were calculated with Fisher’s two-tailed exact test
^aIf a trial had all standard domains (not including the “other bias” domain) judged as “low” risk of bias, we defined the trial as “low risk of bias”
^bIf a trial had at least one of the standard domains (not including the “other bias” domain) judged as “unclear” risk of bias and no domains judged as “high” risk of bias, we defined the trial as “unclear risk of bias.” The judgement of at least one standard risk of bias domain (not including the “other bias” domain) as “unclear” was found in 1103 of 1242 included randomized clinical trials (89 %)
^cIf a trial had at least one of the six standard domains (not including the “other bias” domain) judged as “high” risk of bias, we defined the trial as “high risk of bias”
^dWe only included systematic reviews with one or more randomized clinical trials included in their analyses
^eIt was only possible to assess whether a trial was judged as “low,” “unclear” or “high” risk of bias in 18 non-Cochrane reviews (which provided information on risk of bias judgements for all six standard domains (not including the “other bias” domain) for individual trials via a “risk of bias graph/summary” or “characteristics of studies” section)
^fThe 18 non-Cochrane reviews included 424 randomized clinical trials in total

ISSN: 2046-4053