Table 10 Risk of bias of meta-analyses: ROBIS assessments of individual items, domains and overall risk

1. Study eligibility criteria

 1.1 Did the review adhere to predefined objectives and eligibility criteria? (protocol)

Probably Yes

Probably No

Probably Yes

Probably No

Yes

Yes

 1.2 Were the eligibility criteria appropriate for the review question?

Probably Yes

Probably No

Probably No

Probably Yes

Yes

Yes

 1.3 Were eligibility criteria unambiguous?

Probably Yes

Probably Yes

Yes

No

Yes

Yes

 1.4 Were all restrictions in eligibility criteria based on study characteristics appropriate?

Yes

Yes

Probably No

No

Yes

Yes

 1.5 Were any restrictions in eligibility criteria based on sources of information appropriate?

Yes

Yes

Yes

Yes

Probably Yes

Probably Yes

 1.6 Concerns? (low / high / unclear)

Low

High

High

High

Low

Low

2. Identification and selection of studies

 2.1 Did the search include an appropriate range of databases/electronic sources for published and unpublished reports?

Yes

Probably Yes

Yes

Yes

Yes

Yes

 2.2 Were methods additional to database searching used to identify relevant reports?

Yes

Yes

Yes

Yes

Yes

Yes

 2.3 Were the terms and structure of the search strategy likely to retrieve as many eligible studies as possible?

Probably No

Probably Yes

No Information

Probably No

Yes

Probably Yes

 2.4 Were restrictions based on date, publication format, or language appropriate?

Yes

Yes

Yes

Yes

Yes

Yes

 2.5 Were efforts made to minimise error in selection of studies?

Yes

No

Probably No

Probably No

Probably No

Probably No

 2.6 Concerns? (low / high / unclear)

Unclear

High

Unclear

High

Low

Low

3. Data collection and study appraisal

 3.1 Were efforts made to minimise error in data collection?

Yes

No

Yes

Yes

Yes

Yes

 3.2 Were sufficient study characteristics available for both review authors and readers to be able to interpret the results?

Yes

Yes

Probably Yes

No

Yes

Yes

 3.3 Were all relevant study results collected for use in the synthesis?

Yes

Yes

Yes

Yes

Yes

Yes

 3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria?

Probably Yes

Probably Yes

Probably Yes

Probably Yes

Yes

Yes

 3.5 Were efforts made to minimise error in risk of bias assessment?

Yes

No

Probably No

Probably No

Probably Yes

Probably Yes

 3.6 Concerns? (low / high / unclear)

Low

Unclear

Unclear

High

Low

Low

4. Synthesis and findings

 4.1 Did the synthesis include all studies that it should?

Yes

Probably Yes

Probably Yes

No

Yes

Probably Yes

 4.2 Were all predefined analyses reported or departures explained?

Probably Yes

Probably No

Probably Yes

No

Probably Yes

Yes

 4.3 Was the synthesis appropriate, given the nature and similarity in the research questions, study designs, and outcomes across included studies?

Yes

Probably No

Probably Yes

No

Yes

Yes

 4.4 Was between-study variation (heterogeneity) minimal or addressed in the synthesis?

Yes

No

Yes

Probably Yes

Yes

Yes

 4.5 Were the findings robust, for example, as demonstrated through funnel plot or sensitivity analyses?

Yes

No

Probably Yes

No Information

Yes

No

 4.6 Were biases in primary studies minimal or addressed in the synthesis?

Yes

Yes

Probably No

No

Yes

Yes

 Concerns? (low / high / unclear)

Low

High

Unclear

High

Low

Low

Risk of bias in the review

 A. Did the interpretation of findings address all of the concerns identified in Domains 1 to 4?

Yes

Probably No

Probably Yes

No

Yes

Yes

 B. Was the relevance of identified studies to the review's research question appropriately considered?

Yes

Yes

Probably No

Probably No

Yes

Probably Yes

 C. Did the reviewers avoid emphasising results on the basis of their statistical significance?

Yes

Yes

Yes

No

Yes

Yes

 Risk of bias in the review (low / high / unclear)

Low

High

High

High

Low

Low