Table 1 Methodological features of exemplar overviews

Type of overview

Cochrane overview of reviews

Cochrane overview of reviews

Cochrane overview of reviews

Overview of reviews of diagnostic test accuracy

Mixed method overview of reviews

Aim/question of overview

To synthesise systematic reviews of interventions to improve upper limb function after stroke

To synthesise Cochrane reviews of conservative interventions for the prevention or treatment of female urinary incontinence

To summarise the evidence in Cochrane reviews of the effectiveness and safety of red cell transfusions for treatment or prevention of complications experienced by people with sickle cell disease

To summarise evidence from studies of the accuracy of diagnostic tests of brief cognitive assessments for identifying dementia in primary care

Overview of workplace health promotion interventions

Inclusion criteria: participants, interventions, outcomes

Participants: stroke

Participants: female adults with urinary incontinence

Participants: people with sickle cell disease

Participants: all adults aged 18 years or over recruited from a primary care or general practice population

Participants: any

Interventions: any interventions delivered in a workplace setting

Interventions: any

Interventions: any conservative intervention (definition provided)

Interventions: red cells transfusions

Index test: brief cognitive assessments

Outcomes: upper limb function, impairment or activity of daily living

Outcomes: any

Outcomes: mortality, sickle-cell related complications, adverse events, quality of life, red cell transfusion requirements, and hospital admissions

Outcomes: diagnostic test result (positive/negative)

Outcomes: healthcare or wellbeing outcomes

Other: diagnostic test accuracy information, other reported test outcomes

Inclusion criteria: type of study

Systematic reviews including randomised controlled trials

Cochrane systematic reviews

Cochrane systematic reviews

Systematic reviews of diagnostic test accuracy

Systematic reviews in which the search strategy, inclusion criteria and quality assessment methods are described

Published from 1995 to present; in English;

Search strategy

CDSR, DARE, PROSPERO

CDSR

CDSR

CDSR, EMBASE, MEDLINE, PsycInfo

MEDLINE, DARE, Cochrane Library, PsycInfo, DOPHER, ASSIA, ABI Inform, Scopus, Business Source Premier

Reported line by line for CDSR and DARE

Reported line by line

Reported line by line

Pubmed search reported line by line

Details of the search strategy are available from the reviewers

Updating searches of out of date reviews

–

–

Searches of some included reviews were updated, in order to identify additional primary research studies, and update the original review to include these. The authors determined which reviews to update by using a process of judgement which systematically considered the search dates, presence of included or ongoing trials and relevance of the review topic

–

–

Selection of reviews

Two authors independently apply inclusion criteria

Two authors independently apply inclusion criteria

Two authors independently apply inclusion criteria

Two authors independently apply inclusion criteria

Two reviewers independently apply inclusion criteria

Data extraction

Characteristics of reviews; results of meta-analyses

Characteristics of reviews; results of meta-analyses

Characteristics of reviews; results of meta-analyses

Characteristics of reviews; results of meta analyses; diagnostic test accuracy information

Characteristics of reviews; results of meta-analyses; data from process evaluations

Assessment of quality of reviews

Modified AMSTAR: each of the original 11 AMSTAR questions/criteria were broken down into a series of dichotomous questions, and the answers (yes, no or unclear) to these dichotomous questions were used to determine the answer to the original AMSTAR questions

ROBIS

AMSTAR

AMSTAR and ROBIS

AMSTAR

(Additional comments were added to each of the AMSTAR criteria to clarify the decision making process the authors had made)

(Note: employed both tools as there was an absence of guidance relating to the optimal tool for assessing reviews of DTA and these were the available options. Both tools were reported to provide comparable assessment results, but the ROBIS tool was considered to be easier to apply, and arguably more relevant, as this tool could be specifically tailored to reviews of DTA)

(Note: the AMSTAR scores were categorised into low (score 0–3), medium (score 4–7) or high (8–11))

(Note: this assessment was completed prior to publication and availability of the ROBIS tool)

Assessment of quality of evidence within reviews

GRADE approach, with use of algorithm (details presented and discussed elsewhere [10, 37])

GRADE approach, with use of algorithm, supplemented with input from key stakeholders (clinicians) and statisticians

GRADE approach, applied based on assessments provided in the included reviews, with two independent overview authors applying GRADE levels of evidence and any differences resolved through discussion or third party adjudication

No quality assessment was undertaken. Quality assessment measures from included systematic reviews reported narratively. Results showed that 2/13 reviews made no reference to quality assessment; 2/13 reviews referred to STARD criteria [38] as quality assessment; 2/13 did not report any details on methods or tool used; 6/13 used QUADAS-2 [39] or QUADAS; 1/13 review used a series of tools, including the United States Preventative Services Task Force’s design-specific quality criteria [40], National Institute for Health and Care Excellence (previously National Institute for Health and Clinical Excellence) methodology checklist [41], AMSTAR [42] and the Newcastle-Ottowa Scale for observational studies [43]

No quality assessment was undertaken: the objective of this overview was to provide a ‘systematic map’ of relevant reviews which aimed to identify characteristics associated with effectiveness

Data synthesis

Grouped according to GRADE, for intervention versus control, for each outcome explored. Highlighting evidence of beneficial effect, harm or no effect

Data grouped and presented according to type of urinary incontinence. Tabulated summary of systematic review evidence relating to all conservative interventions for urinary incontinence, signposting which systematic reviews address which interventions, with summary of details of the population of participants, comparisons, volume and quality of evidence

Reported using the GRADE approach, with presentation of statistical outcome data as determined by data available in the reviews

Synthesis of results across included studies using sensitivity and specificity measures, presented narratively and within tables

Framework synthesis to synthesise evidence from systematic reviews with other types of evidence (research of stakeholders’ experiences and perspectives; key policy documents). Data relating to moderator analyses to identify ‘successful’ intervention characteristics were tabulated

Statistical analysis

No statistical analysis was planned or completed

1. Network maps, created using Stata software, to illustrate the number of trial and participants within trials

No statistical analysis was planned or completed

No statistical analysis was completed

No statistical analysis was planned or completed

(Note: additional analysis is ongoing relating to the assessment of test administration times, analysis of DTA across the most widely used brief cognitive assessments, and comparing diagnostic test performance across all systematic reviews)

A visual plot of the effect size relating to the effect of interventions on the primary outcome was produced (reproducing data from the included reviews)

2. Visual plot of effect size for comparisons of conservative intervention versus control, placebo or standard care for outcomes of condition-specific quality of life and symptomatic cure or improvement

Summary of key findings

Two key summary tables were produced; each of these summarised the available evidence relating to 19 identified different interventions, highlighting the GRADE of the evidence, the direction of the effect (where GRADE was high or moderate), and key implications for clinical practice and research (see Additional file 1)

A ‘Summary of results’ table was planned, clearly indicating where there is evidence of an effect of conservative interventions, for each relevant intervention comparison and for both primary and secondary outcome of interest (see Additional file 2)

‘Overview of reviews’ table in a format similar to the ‘Summary of findings’ table as recommended in chapter 22 of the Cochrane Handbook for Systematic Reviews of Interventions [33]. These will include the primary outcomes of this overview for each intervention (see Additional file 2)

A summary of findings table was produced displaying review reference, component study reference, reference standard, index test, threshold, time taken, sample size and key demographic data, target condition, condition prevalence, accuracy data reported and notes. Additional summary tables were produced for comparisons at review level, non-DTA data reported, and individual brief cognitive assessments

Structured summary

Executive summary

Table of ‘key characteristics identified from effectiveness reviews, views studies and policy documents’

A summary table in which information about each of the included reviews was provided, including the AMSTAR rating of the review. The direction of effect for different intervention types was also summarised, categorising evidence according to whether there was a statistically significant beneficial effect, non-statistically significant beneficial effect, no difference between control and intervention, non-significant detrimental effect or statistically significant detrimental effect (see Additional file 3)

Publication status

Published (Cochrane Library) [19]

Ongoing (protocol published in Cochrane Library) [21]

Ongoing (protocol published in Cochrane Library) [20]

Ongoing (protocol published on PROSPERO ref. CRD42015022078) [22]

Published (EPPI-Centre website) [23]