Table 1 A comparison of the content of the two tools (AMSTAR-2 and ROBIS)

Eligibility criteria

1. Did the research questions and inclusion criteria for the review include the components of PICO?

2. Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?

3. Did the review authors explain their selection of the study designs for inclusion in the review?

1.1 Did the review adhere to pre-defined objectives and eligibility criteria?

1.2 Were the eligibility criteria appropriate for the review question?

1.3 Were eligibility criteria unambiguous?

1.4 Were all restrictions in eligibility criteria based on study characteristics appropriate (e.g., date, sample size, study quality, outcomes measured)?

1.5 Were any restrictions in eligibility criteria based on sources of information appropriate (e.g., publication status or format, language, availability of data)?

Study selection and Data extraction

5. Did the review authors perform study selection in duplicate?

6. Did the review authors perform data extraction in duplicate?

2.5 Were efforts made to minimise error in selection of studies?

3.1 Were efforts made to minimise error in data collection?

3.3 Were all relevant study results collected for use in the synthesis?

Literature search

4. Did the review authors use a comprehensive literature search strategy?

2.1 Did the search include an appropriate range of databases/electronic sources for published and unpublished reports?

2.3 Were the terms and structure of the search strategy likely to retrieve as many eligible studies as possible?

2.4 Were restrictions based on date, publication format, or language appropriate?

Grey literature

NA

2.2 Were methods additional to database searching used to identify relevant reports?

List of studies

7. Did the review authors provide a list of excluded studies and justify the exclusions?

N/A

Characteristics of studies

8. Did the review authors describe the included studies in adequate detail?

3.2 Were sufficient study characteristics available for both review authors and readers to be able to interpret the results?

Quality assessment

9. Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?

3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria?

3.5 Were efforts made to minimise error in risk of bias assessment?

Synthesis of the findings

N/A

N/A

11. If meta-analysis was performed did the review authors use appropriate methods for statistical combination of results?

12. If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?

4.1 Did the synthesis include all studies that it should?

4.2 Were all pre-defined analyses reported or departures explained?

4.3 Was the synthesis appropriate given the nature and similarity in the research questions, study designs and outcomes across included studies?

4.6 Were biases in primary studies minimal or addressed in the synthesis?

Heterogeneity

14. Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review? 15. If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?

4.4 Was between-study variation (heterogeneity) minimal or addressed in the synthesis?

4.5 Were the findings robust, e.g., as demonstrated through funnel plot or sensitivity analyses?

Interpretation

13. Did the review authors account for RoB in individual studies when interpreting/ discussing the results of the review?

A. Did the interpretation of findings address all of the concerns identified in Domains 1 to 4?

B. Was the relevance of identified studies to the review’s research

question appropriately considered?

C. Did the reviewers avoid emphasising results on the basis of their statistical significance?

Conflict of interest

10. Did the review authors report on the sources of funding for the studies included in the review?

16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?

N/A

N/A