Skip to main content

Table 3 Extract of results from survey of Cochrane review authors who had used the risk of bias tool

From: Evaluation of the Cochrane Collaboration’s tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation

Survey questions n (%)
Number of reviews respondent used RoB tool in (Q1)  
    One 80 (42)
    Two or three 75 (40)
    More than three 33 (18)
Used RoB tool to update an existing review (Q2) 102 (54)
Time taken to complete RoB assessment for one study (Q3)  
    Up to 10 minutes 23 (12)
    10 to 20 minutes 81 (43)
    20 minutes to 1 hour 69 (37)
    More than 1 hour 14 (8)
Time taken is acceptable (Q4) 156 (83)
Used pilot testing (Q5) 62 (33)
Modified the RoB tool when used for randomized trials (Q7) 56 (31)
Used the RoB tool for non-randomized studies (Q6) 39 (21)
    Modified the RoB tool when used for non-randomized studies (Q6a) 31
RoB assessments incorporated in conclusion/analysis: (Q9)a  
    Sensitivity analysis by RoB judgement 76 (40)
    Included a narrative summary 104 (55)
    Not at all 26 (14)
Used direct quotes to support judgement (Q10)  
    Always or nearly always 76 (41)
    Often 59 (32)
Feel confident in their RoB assessments (Q12)  
    Very confident 61 (32)
    Somewhat confident 111 (59)
Tool is better than previous Cochrane practice (Q13) 165 (87)
Features respondents most liked (Q14)a  
    Ability to provide information (for example quotes) 140 (74)
    Standardized approach 153 (81)
Features respondents least liked (Q15)a  
    Judgement options (Yes/No/Unclear) confusing 69 (36)
    Time taken to complete 56 (29)
Encountered problems with assessing sequence generation (Q17) 82 (44)
Encountered problems with assessing allocation concealment (Q18) 90 (50)
Encountered problems with assessing blinding (Q19) 94 (52)
Encountered problems with assessing incomplete outcome data (Q20) 122 (67)
Encountered problems with assessing selective outcome reporting (Q21) 110 (60)
Encountered problems with assessing other bias (Q22) 107 (58)
Other bias domain is helpful (Q23) 108 (61)
Use standard ‘other sources of bias’ (Q24) 53 (29)
  1. Based on 190 respondents, authors who have used the RoB tool. Only the most frequent responses shown in the table, and some response options have been grouped to fewer categories. Not all respondents answered each question. Question numbers refer to the survey question in Additional file1: Appendix 1. For full details of questions and responses, see Additional file1: Appendix 1. a Respondents were allowed to select multiple answers for this question. RoB, risk of bias.