Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews

Table 6 Use of additional non-standard domains and the “other bias” domain in the Cochrane and non-Cochrane reviews that applied the Cochrane risk of bias tool for randomized clinical trials

Use of additional domains and “other bias”	100 Cochrane reviews (100 %)	31 non-Cochrane reviews (100 %)^c	P value*
Additional domains
Any additional domain(s)	11 (11 %)	6 (19 %)	0.37
-Adds “baseline imbalance”	6 of 11 (55 %)	2 of 6 (33 %)	1.00
-Adds “funding” or “conflicts of interest”	5 of 11 (45 %)	1 of 6 (17 %)	0.62
-Adds “intention to treat”	2 of 11 (18 %)	2 of 6 (33 %)	0.62
-Adds “compliance”	2 of 11 (18 %)	1 of 6 (17 %)	1.00
-Adds “follow up”	3 of 11 (27 %)	2 of 6 (33 %)	1.00
-Adds “timing of outcome assessment”	2 of 11 (18 %)	1 of 6 (17 %)	1.00
-Adds “overall risk of bias”	1 of 11 (9 %)	4 of 6 (67 %)	0.14
-Adds other additional domain^a	6 of 11 (55 %)	2 of 6 (33 %)	1.00
Other bias^b
Includes the “other bias” domain	73 (73 %)	17 (55 %)	0.41
-Used for “baseline imbalance”	33 of 73 (45 %)	0 of 17 (0 %)	0.0059
-Used for “funding” or “conflicts of interest”	23 of 73 (32 %)	2 of 17 (12 %)	0.24
-Used for “intervention differed between groups”	16 of 73 (22 %)	0 of 17 (0 %)	0.069
-Used for “unclear reporting by trial publication author”	15 of 73 (21 %)	0 of 17 (0 %)	0.12
-Used for “trial design”	11 of 73 (15 %)	0 of 17 (0 %)	0.20

*P values were calculated with Fisher’s two-tailed exact test
^aAll of the following other additional domains appeared ones in review samples: Cochrane reviews: “co-interventions avoided or similar,” “confounding variables,” “definition of incomplete response,” “definition of local recurrence,” “method of follow up” and “size”; non-Cochrane reviews: “co-intervention” and “double blinding”
^b“Other bias”—comments were interpreted and categorised (e.g. the “other bias” comment “There were baseline differences between groups.” was categorised as “baseline imbalance”). The five most used “other bias”—categories are listed
^c31 of 100 non-Cochrane reviews used the Cochrane risk of bias tool for randomized clinical trials (i.e. the tool) and were compared to the 100 Cochrane reviews that used the tool for randomized clinical trials

ISSN: 2046-4053