Skip to main content

Table 3 Extract of results from survey of Cochrane review authors who had used the risk of bias tool

From: Evaluation of the Cochrane Collaboration’s tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation

Survey questions

n (%)

Number of reviews respondent used RoB tool in (Q1)

 

    One

80 (42)

    Two or three

75 (40)

    More than three

33 (18)

Used RoB tool to update an existing review (Q2)

102 (54)

Time taken to complete RoB assessment for one study (Q3)

 

    Up to 10 minutes

23 (12)

    10 to 20 minutes

81 (43)

    20 minutes to 1 hour

69 (37)

    More than 1 hour

14 (8)

Time taken is acceptable (Q4)

156 (83)

Used pilot testing (Q5)

62 (33)

Modified the RoB tool when used for randomized trials (Q7)

56 (31)

Used the RoB tool for non-randomized studies (Q6)

39 (21)

    Modified the RoB tool when used for non-randomized studies (Q6a)

31

RoB assessments incorporated in conclusion/analysis: (Q9)a

 

    Sensitivity analysis by RoB judgement

76 (40)

    Included a narrative summary

104 (55)

    Not at all

26 (14)

Used direct quotes to support judgement (Q10)

 

    Always or nearly always

76 (41)

    Often

59 (32)

Feel confident in their RoB assessments (Q12)

 

    Very confident

61 (32)

    Somewhat confident

111 (59)

Tool is better than previous Cochrane practice (Q13)

165 (87)

Features respondents most liked (Q14)a

 

    Ability to provide information (for example quotes)

140 (74)

    Standardized approach

153 (81)

Features respondents least liked (Q15)a

 

    Judgement options (Yes/No/Unclear) confusing

69 (36)

    Time taken to complete

56 (29)

Encountered problems with assessing sequence generation (Q17)

82 (44)

Encountered problems with assessing allocation concealment (Q18)

90 (50)

Encountered problems with assessing blinding (Q19)

94 (52)

Encountered problems with assessing incomplete outcome data (Q20)

122 (67)

Encountered problems with assessing selective outcome reporting (Q21)

110 (60)

Encountered problems with assessing other bias (Q22)

107 (58)

Other bias domain is helpful (Q23)

108 (61)

Use standard ‘other sources of bias’ (Q24)

53 (29)

  1. Based on 190 respondents, authors who have used the RoB tool. Only the most frequent responses shown in the table, and some response options have been grouped to fewer categories. Not all respondents answered each question. Question numbers refer to the survey question in Additional file1: Appendix 1. For full details of questions and responses, see Additional file1: Appendix 1. a Respondents were allowed to select multiple answers for this question. RoB, risk of bias.