Skip to main content

Table 1 Methodological features of exemplar overviews

From: Selecting and implementing overview methods: implications from five exemplar overviews

  Pollock [19] McClurg [21] Estcourt [20] Hunt [22] Brunton [23]
Type of overview Cochrane overview of reviews Cochrane overview of reviews Cochrane overview of reviews Overview of reviews of diagnostic test accuracy Mixed method overview of reviews
Aim/question of overview To synthesise systematic reviews of interventions to improve upper limb function after stroke To synthesise Cochrane reviews of conservative interventions for the prevention or treatment of female urinary incontinence To summarise the evidence in Cochrane reviews of the effectiveness and safety of red cell transfusions for treatment or prevention of complications experienced by people with sickle cell disease To summarise evidence from studies of the accuracy of diagnostic tests of brief cognitive assessments for identifying dementia in primary care Overview of workplace health promotion interventions
Inclusion criteria: participants, interventions, outcomes Participants: stroke Participants: female adults with urinary incontinence Participants: people with sickle cell disease Participants: all adults aged 18 years or over recruited from a primary care or general practice population Participants: any
Interventions: any interventions delivered in a workplace setting
Interventions: any Interventions: any conservative intervention (definition provided) Interventions: red cells transfusions
Index test: brief cognitive assessments
Outcomes: upper limb function, impairment or activity of daily living Outcomes: any Outcomes: mortality, sickle-cell related complications, adverse events, quality of life, red cell transfusion requirements, and hospital admissions Outcomes: diagnostic test result (positive/negative) Outcomes: healthcare or wellbeing outcomes
Other: diagnostic test accuracy information, other reported test outcomes
Inclusion criteria: type of study Systematic reviews including randomised controlled trials Cochrane systematic reviews Cochrane systematic reviews Systematic reviews of diagnostic test accuracy Systematic reviews in which the search strategy, inclusion criteria and quality assessment methods are described
Published from 1995 to present; in English;
Search strategy CDSR, DARE, PROSPERO CDSR CDSR CDSR, EMBASE, MEDLINE, PsycInfo MEDLINE, DARE, Cochrane Library, PsycInfo, DOPHER, ASSIA, ABI Inform, Scopus, Business Source Premier
Reported line by line for CDSR and DARE Reported line by line Reported line by line Pubmed search reported line by line
Details of the search strategy are available from the reviewers
Updating searches of out of date reviews Searches of some included reviews were updated, in order to identify additional primary research studies, and update the original review to include these. The authors determined which reviews to update by using a process of judgement which systematically considered the search dates, presence of included or ongoing trials and relevance of the review topic
Selection of reviews Two authors independently apply inclusion criteria Two authors independently apply inclusion criteria Two authors independently apply inclusion criteria Two authors independently apply inclusion criteria Two reviewers independently apply inclusion criteria
Data extraction Characteristics of reviews; results of meta-analyses Characteristics of reviews; results of meta-analyses Characteristics of reviews; results of meta-analyses Characteristics of reviews; results of meta analyses; diagnostic test accuracy information Characteristics of reviews; results of meta-analyses; data from process evaluations
Assessment of quality of reviews Modified AMSTAR: each of the original 11 AMSTAR questions/criteria were broken down into a series of dichotomous questions, and the answers (yes, no or unclear) to these dichotomous questions were used to determine the answer to the original AMSTAR questions ROBIS AMSTAR AMSTAR and ROBIS AMSTAR
(Additional comments were added to each of the AMSTAR criteria to clarify the decision making process the authors had made) (Note: employed both tools as there was an absence of guidance relating to the optimal tool for assessing reviews of DTA and these were the available options. Both tools were reported to provide comparable assessment results, but the ROBIS tool was considered to be easier to apply, and arguably more relevant, as this tool could be specifically tailored to reviews of DTA) (Note: the AMSTAR scores were categorised into low (score 0–3), medium (score 4–7) or high (8–11))
(Note: this assessment was completed prior to publication and availability of the ROBIS tool)
Assessment of quality of evidence within reviews GRADE approach, with use of algorithm (details presented and discussed elsewhere [10, 37]) GRADE approach, with use of algorithm, supplemented with input from key stakeholders (clinicians) and statisticians GRADE approach, applied based on assessments provided in the included reviews, with two independent overview authors applying GRADE levels of evidence and any differences resolved through discussion or third party adjudication No quality assessment was undertaken. Quality assessment measures from included systematic reviews reported narratively. Results showed that 2/13 reviews made no reference to quality assessment; 2/13 reviews referred to STARD criteria [38] as quality assessment; 2/13 did not report any details on methods or tool used; 6/13 used QUADAS-2 [39] or QUADAS; 1/13 review used a series of tools, including the United States Preventative Services Task Force’s design-specific quality criteria [40], National Institute for Health and Care Excellence (previously National Institute for Health and Clinical Excellence) methodology checklist [41], AMSTAR [42] and the Newcastle-Ottowa Scale for observational studies [43] No quality assessment was undertaken: the objective of this overview was to provide a ‘systematic map’ of relevant reviews which aimed to identify characteristics associated with effectiveness
Data synthesis Grouped according to GRADE, for intervention versus control, for each outcome explored. Highlighting evidence of beneficial effect, harm or no effect Data grouped and presented according to type of urinary incontinence. Tabulated summary of systematic review evidence relating to all conservative interventions for urinary incontinence, signposting which systematic reviews address which interventions, with summary of details of the population of participants, comparisons, volume and quality of evidence Reported using the GRADE approach, with presentation of statistical outcome data as determined by data available in the reviews Synthesis of results across included studies using sensitivity and specificity measures, presented narratively and within tables Framework synthesis to synthesise evidence from systematic reviews with other types of evidence (research of stakeholders’ experiences and perspectives; key policy documents). Data relating to moderator analyses to identify ‘successful’ intervention characteristics were tabulated
Statistical analysis No statistical analysis was planned or completed 1. Network maps, created using Stata software, to illustrate the number of trial and participants within trials No statistical analysis was planned or completed No statistical analysis was completed No statistical analysis was planned or completed
(Note: additional analysis is ongoing relating to the assessment of test administration times, analysis of DTA across the most widely used brief cognitive assessments, and comparing diagnostic test performance across all systematic reviews)
A visual plot of the effect size relating to the effect of interventions on the primary outcome was produced (reproducing data from the included reviews)
2. Visual plot of effect size for comparisons of conservative intervention versus control, placebo or standard care for outcomes of condition-specific quality of life and symptomatic cure or improvement
Summary of key findings Two key summary tables were produced; each of these summarised the available evidence relating to 19 identified different interventions, highlighting the GRADE of the evidence, the direction of the effect (where GRADE was high or moderate), and key implications for clinical practice and research (see Additional file 1) A ‘Summary of results’ table was planned, clearly indicating where there is evidence of an effect of conservative interventions, for each relevant intervention comparison and for both primary and secondary outcome of interest (see Additional file 2) ‘Overview of reviews’ table in a format similar to the ‘Summary of findings’ table as recommended in chapter 22 of the Cochrane Handbook for Systematic Reviews of Interventions [33]. These will include the primary outcomes of this overview for each intervention (see Additional file 2) A summary of findings table was produced displaying review reference, component study reference, reference standard, index test, threshold, time taken, sample size and key demographic data, target condition, condition prevalence, accuracy data reported and notes. Additional summary tables were produced for comparisons at review level, non-DTA data reported, and individual brief cognitive assessments Structured summary
Executive summary
Table of ‘key characteristics identified from effectiveness reviews, views studies and policy documents’
A summary table in which information about each of the included reviews was provided, including the AMSTAR rating of the review. The direction of effect for different intervention types was also summarised, categorising evidence according to whether there was a statistically significant beneficial effect, non-statistically significant beneficial effect, no difference between control and intervention, non-significant detrimental effect or statistically significant detrimental effect (see Additional file 3)
Publication status Published (Cochrane Library) [19] Ongoing (protocol published in Cochrane Library) [21] Ongoing (protocol published in Cochrane Library) [20] Ongoing (protocol published on PROSPERO ref. CRD42015022078) [22] Published (EPPI-Centre website) [23]