Table 2 Complexity of different inclusion scenarios

Full inclusion scenario (include all Cochrane and non-Cochrane SRs)

Some overlapping primary studies included in non-Cochrane SRs were identified by, but excluded from, the Cochrane SRs for being outside the scope or for having methodological deficiencies.

Overlapping SRs sometimes presented the same or similar outcome data in different ways.

Overlapping SRs sometimes had discordant results for the same outcomes.

Data extraction from non-Cochrane SRs was sometimes difficult due to deficiencies in conduct and reporting.

First restricted scenario (include only Cochrane SRs)

Input from a clinical expert was often required to determine whether the Cochrane SRs comprehensively examined all relevant intervention comparisons.

Cochrane SRs sometimes overlapped.

Second restricted scenario (include all non-overlapping SRs and selects the Cochrane SR for groups of overlapping SRs)

Not all groups of overlapping SRs included a Cochrane SR.

Data extraction from non-Cochrane SRs was sometimes difficult due to deficiencies in conduct and reporting.

Third restricted scenario (include all non-overlapping SRs and select the most recent SR for groups of overlapping SRs)

Overlapping SRs were sometimes “tied” for most recent year of publication.

Search dates were not comprehensively or consistently reported in all SRs.

Data extraction from non-Cochrane SRs was sometimes difficult due to deficiencies in conduct and reporting.

Fourth restricted scenario (include all non-overlapping SRs and select the highest quality SR for groups of overlapping SRs)

Overlapping SRs were sometimes “tied” for highest quality.

Conducting quality assessments was challenging and time-intensive.

Data extraction from non-Cochrane SRs was sometimes difficult due to deficiencies in conduct and reporting.