Table 8 Inter-rater agreement

 1.1

14

0.73

0.46, 1.00

< 0.001

 1.2

15

0.70

0.45, 0.95

< 0.001

 1.3

15

0.62

0.39, 0.84

< 0.001

 1.4

15

0.54

0.32, 0.76

< 0.001

 1.5

15

0.64

0.40, 0.88

< 0.001

 Domain 1Concerns regarding specification of study eligibility criteria

15

0.61

0.29, 0.92

0.001

 2.1

15

0.53

0.36, 0.69

< 0.001

 2.2

14

0.42

0.16, 0.69

0.005

 2.3

15

0.72

0.53, 0.92

< 0.001

 2.4

15

0.31

− 0.08, 0.70

0.110

 2.5

15

0.56

0.14, 0.99

0.013

 Domain 2Concerns regarding methods used to identify and/or select studies

15

0.29

0.03, 0.55

0.031

 3.1

15

0.95

0.66, 1.00

< 0.001

 3.2

15

0.65

0.47, 0.84

< 0.001

 3.3

15

0.57

0.40, 0.74

< 0.001

 3.4

15

0.55

0.23, 0.88

0.003

 3.5

15

0.81

0.51, 1.00

< 0.001

 Domain 3Concerns regarding methods used to collect data and appraise studies

15

0.52

0.19, 0.83

0.004

 4.1

15

0.55

0.33, 0.77

< 0.001

 4.2

13

0.55

0.29, 0.81

0.001

 4.3

15

0.80

0.62, 0.98

< 0.001

 4.4

15

0.13

− 0.19, 0.45

0.405

 4.5

14

− 0.10

− 0.52, 0.33

0.633

 4.6

15

0.23

− 0.17, 0.64

0.235

 Domain 4Concerns regarding the synthesis and findings

15

0.18

− 0.08, 0.44

0.154

 A

15

0.10

− 0.25, 0.44

0.552

 B

15

0.61

0.40, 0.83

< 0.001

 C

15

0.39

0.01, 0.76

0.009

 ROB

15

0.43

0.10, 0.77

0.015