Table 12 Inter-rater agreement

 1.1

16

0.57

0.17, 0.96

0.008

 1.2

16

0.71

0.55, 0.87

< 0.001

 1.3

16

0.76

0.61, 0.91

< 0.001

 1.4

16

0.71

0.54, 0.87

< 0.001

 1.5

16

0.49

0.20, 0.77

0.002

 Domain 1Concerns regarding specification of study eligibility criteria

16

0.30

− 0.03, 0.63

0.072

 2.1

16

0.54

0.34, 0.73

< 0.001

 2.2

16

0.64

0.37, 0.92

< 0.001

 2.3

16

0.57

0.34, 0.81

< 0.001

 2.4

16

0.50

0.27, 0.73

< 0.001

 2.5

14

0.61

0.18, 1.00

< 0.001

 Domain 2Concerns regarding methods used to identify and/or select studies

16

0.43

0.13, 0.73

0.008

 3.1

14

0.82

0.51, 1.00

< 0.001

 3.2

16

0.70

0.44, 0.96

< 0.001

 3.3

16

0.72

0.52, 0.92

< 0.001

 3.4

16

0.92

0.83, 1.00

< 0.001

 3.5

15

0.66

0.21, 1.00

0.007

 Domain 3Concerns regarding methods used to collect data and appraise studies

16

0.61

0.32, 0.89

< 0.001

 4.1

16

0.65

0.45, 0.86

< 0.001

 4.2

16

0.42

0.11, 0.73

0.011

 4.3

16

0.73

0.58, 0.88

< 0.001

 4.4

16

0.23

− 0.02, 0.48

0.072

 4.5

16

0.40

0.22, 0.57

< 0.001

 4.6

16

0.55

0.32, 0.77

< 0.001

 Domain 4Concerns regarding the synthesis and findings

16

0.17

− 0.17, 0.50

0.305

 A

16

0.47

0.28, 0.65

0.015

 B

16

0.69

0.55, 0.82

< 0.001

 C

16

0.54

0.37, 0.72

< 0.001

 ROB

16

0.47

0.17, 0.77

0.004