Table 4 Specification of eligibility criteria

Becker 2008 [1]; Cooper 2012 [32]; Ioannidis 2009 [38]; JBI 2015 [40, 41]; Li 2012 [44]; Thomson 2013 [59]

1.2.1 Select only SRs that are similar (or narrower) in scope to the overview PICO elements (i.e. exclude SRs that include out-of-scope interventions/populations in addition to the intervention/population addressed by the overview)

Becker 2008 [1]; Cooper 2012 [32]; Foisy 2011 [34]; JBI 2015 [40, 41]; Robinson 2016 [48,49,50,51,52]; Ryan 2009 [53, 54]; Thompson 2013 [59]

1.2.2 Select all SRs that address the PICO elements, including those broader in scope than the overview (i.e. SRs that include the intervention/ population addressed by the overview, plus other out-of-scope interventions/ populations). This may involve selecting: (i) any SR, irrespective of whether separate data are available for the subgroup of interest or (ii) limiting to SRs that present separate data for the subgroup of interest

Becker 2008 [1]; Cooper 2012 [32]; Kramer 2009 [43]; Ryan 2009 [53, 54]; Thompson 2013 [59]; Whitlock 2008 [48,49,50,51,52]

1.3.1 Include all outcomes reported in included SRs

Becker 2008 [1]; Hartling 2012 [35]; Ryan 2009 [53, 54]; Thomson 2013 [59]

▪ Map the outcomes to a taxonomy (Ryan 2009 [53, 54])

1.3.2 Select one or more outcomes using pre-specified criteria, for example: (i) outcomes judged important by subject specialists (e.g. consumers, policy makers), (ii) primary outcomes, and (iii) outcomes common to more than one SR

Caird 2015 [31]; Hartling 2014 [37]; Ioannidis 2009 [38]; JBI 2015 [40, 41]; Smith 2011 [57]; Thomson 2013 [59]

▪ Report only those outcomes common to more than one SR (Caird 2015 [31]; Hartling 2014 [37])

1.3.3 Select one or more outcomes using pre-specified decision rules (e.g. combine selection criteria in an algorithm)

Inferred method

1.4.1 Include all SRs that meet the PICO criteria (i.e. no methodological criteria applied)

Caird 2015 [31]

1.4.2 Select SRs that meet minimum quality criteria or take a particular methodological approach.

Minimum criteria include:

(i) meets definition of an SR, (e.g. explicit search)

(ii) up-to-date

(iii) quality of the SR (e.g. based on selected criteria; cutoffs derived from AMSTAR score)

(iv) use of best practice methods (e.g. specific RoB tools; Cochrane or AHRQ’s EPC methods)

(v) free of conflicts of interest (e.g. no industry funding)

(vi) reports sufficient primary study characteristics to interpret results (e.g. PICO elements, RoB assessment)

Methodological approaches include:

(vii) type of included primary studies

(viii) type of data

(ix) type of synthesis (e.g. meta-analysis, narrative)

Becker 2008 [1]; Chen 2014 [2]; Cooper 2012 [32]; Foisy 2011 [34]; Hartling 2013 [36]; James 2014 [39]; JBI 2015 [40, 41]; Robinson 2016 [48,49,50,51,52]; Smith 2011 [57]; Thompson 2013 [59]

1.5.1 Include all SRs that meet the PICO, irrespective of overlap

Cooper 2012 [32]; Whitlock 2008 [48,49,50,51,52]

1.5.2 Select one SR from multiple addressing the same question using pre-specified methodological criteria as outlined in 1.4.2

Cooper 2012 [32]; Pieper 2014 [46]; Robinson 2015 [48,49,50,51,52]

▪ Select the highest quality SR (Cooper 2012 [32])

1.5.3 Select one SR from multiple addressing the same question using pre-specified decision rules (e.g. combine one or more eligibility criteria in an algorithm)

Cooper 2012 [32]

▪ Select the SR with the most complete information, and if these are equivalent, the M-A with the greatest number of primary studies (Cooper 2012 [32])

1.5.4 Exclude SRs that do not contain any unique primary studies, when there are multiple SRs

Pieper 2014 [46]

1.6.1 Do not include primary studies

Becker 2008 [1]; Caird 2015 [31]; Thompson 2013 [59]; Whitlock 2008 [48,49,50,51,52]

1.6.2 Include primary studies if pre-specified eligibility criteria are met, for example: (i) when a SR is not up-to-date, (ii) when a SR is inconclusive (i.e. new studies may overturn the findings of a SR), (iii) when the included SRs provide incomplete coverage of evidence in relation to the overview PICO (e.g. missing one or more interventions, population subgroup, study design), and (iv) when there are concerns about the methods SRs used to identify and select studies

Baker 2014 [29]; Caird 2015 [31]; Cooper 2012 [32]; Pieper 2014 [46]; 2014 [47]; Thompson 2013 [59]; White 2009 [48,49,50,51,52]

▪ Include primary studies if the evidence in the SRs is inconclusive (e.g. when addition of a new primary study may overturn the findings) (Pieper 2014 [47])

▪ Include primary studies if the SRs are assessed as low quality (Pieper 2014 [47])

1.6.3 Include primary studies using pre-specified decision rules to determine eligibility (e.g. combine one or more eligibility criteria in an algorithm for selection)

Pieper 2014 [47]

2.1.1 Independent screening all stages by 2 or more authors

Becker 2008 [1]; Chen 2014 [2]; Hartling 2012 [35]; Li 2012 [44]; Pieper 2012 [6, 45]; 2014 [47]; Smith 2011 [57]

2.1.2 One author screening at all stages

Hartling 2012 [35]; Li 2012 [44]; Pieper 2014 [47]

2.1.3 One author screening titles/abstracts, 2 or more screening full text

Hartling 2012 [35]

2.1.4 One screened at all stages, 2nd confirmed

Hartling 2012 [35]

2.1.5 One screened at all stages, 2nd confirms if uncertainty

Hartling 2012 [35]