Skip to main content

Table 5 Inter-rater agreement

From: A comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2

ROBIS question No. of studies Gwet’s AC1/Gwet’s AC2 95% CI
Domain 1: study eligibility criteria
 1.1 30 0.62 0.38, 0.85
 1.2 31 0.70 0.56, 0.84
 1.3 31 0.69 0.56, 0.82
 1.4 31 0.61 0.48, 0.74
 1.5 31 0.56 0.37, 0.74
Domain 1 Concerns regarding specification of study eligibility criteria 31 0.45 0.22, 0.67
Domain 2: identification and selection of studies
 2.1 31 0.53 0.41, 0.65
 2.2 30 0.53 0.35, 0.71
 2.3 31 0.62 0.47, 0.77
 2.4 31 0.41 0.20, 0.62
 2.5 29 0.59 0.30, 0.88
Domain 2 Concerns regarding methods used to identify and/or select studies 31 0.36 0.17, 0.55
Domain 3: data collection and study appraisal
 3.1 29 0.88 0.68, 1.00
 3.2 31 0.66 0.51, 0.82
 3.3 31 0.65 0.51, 0.78
 3.4 31 0.77 0.61, 0.93
 3.5 30 0.73 0.48, 0.98
Domain 3 Concerns regarding methods used to collect data and appraise studies 31 0.55 0.35, 0.76
Domain 4: synthesis and findings
 4.1 31 0.60 0.46, 0.74
 4.2 29 0.48 0.28, 0.68
 4.3 31 0.77 0.66, 0.88
 4.4 31 0.18 − 0.02, 0.37
 4.5 30 0.22 0.02, 0.43
 4.6 31 0.39 0.17, 0.62
Domain 4 Concerns regarding the synthesis and findings 31 0.17 − 0.03, 0.37
Risk of bias in the review
 A 31 0.28 0.09, 0.47
 B 31 0.64 0.54, 0.75
 C 31 0.45 0.31, 0.60
ROB 31 0.45 0.24, 0.66