Skip to main content

Table 6 Data extraction

From: Toward a comprehensive evidence map of overview of systematic review methods: paper 1—purpose, eligibility, search and data extraction

Step Sub-step Methods/approaches Sources
▪ Examples
1.0 Plan the data elements to extract
  1.1 Determine the data to extract on the characteristics of SRsa Becker 2008 [1]; Caird 2015 [31]; JBI 2015 [40, 41]; Li 2012 [44]; Ryan 2009 [53, 54]
  1.2 Determine the data required to assess which SRs address the overview question and allow assessment of the overlap across SRsa Smith 2011 [57]
  1.3 Determine data to extract about the results from the SRs for each relevant primary outcome
   1.3.1 Extract M-A results Becker 2008 [1]; Caird 2015 [31]; Hartling 2012 [35]; Smith 2011 [57]
   1.3.2 Extract numeric trial results Thomson 2013 [59]
   1.3.3 Extract narrative results Bolland 2014 [30]; JBI 2015 [40, 41]; Li 2012 [44]; Ryan 2009 [53, 54]
   1.3.4 Extract a combination of 1.3.1–1.3.3  
   1.3.5 Extract risk of bias assessment (overall assessment, or domain/item level data, or both) and certainty of the evidence Becker 2008 [1]; Hartling 2012 [35]; JBI 2015 [40, 41]; Li 2012 [44]; Ryan 2009 [53, 54]
  1.4 Determine the data to extract from primary studiesa
   1.4.1 Extract numerical trial results Caird 2015 [31]
   1.4.2 Extract data required to assess risk of bias for each domain or item Hartling 2012 [35]
  1.5 Develop a data extraction forma Becker 2008 [1]; Cooper 2012 [32]; Hartling 2012 [35]; JBI 2015 [40, 41]; Singh 2012 [56]
2.0 Plan the data extraction process
  2.1 Determine the sources where data will be obtained from
   2.1.1 SRs Becker 2008 [1]; Bolland 2014 [30]; Caird 2015 [31]; CMIMG 2012 [4]; Hartling 2014 [37]; JBI 2015 [40, 41]; Pieper 2012 [6, 45]
   2.1.2 Primary studies Caird 2015 [31]; Salanti 2011 [3]; Thomson 2013 [59]; Whitlock 2008 [48,49,50,51,52]
   2.1.3 Registry entries (for SRs and/or trials) Inferred method
   2.1.4 A combination of the above Caird 2015 [31]; Salanti 2011 [3]; Thomson 2013 [59]; Whitlock 2008 [48,49,50,51,52]
  2.2 Determine how overlapping information across SRs will be handled
   2.2.1 Extract information from all SRs Bolland 2014 [30]; Caird 2015 [31]; CMIMG 2012 [4]; Cooper 2012 [32]; Hartling 2014 [37]; JBI 2015 [40, 41]; Pieper 2014 [46]; White 2009 [48,49,50,51,52]
   2.2.2 Extract information from only one SR based on a priori eligibility criteria Cooper 2012 [32]; CMIMG 2012 [4]; Foisy 2011 [34]; Hartling 2014 [37]; Pieper 2012 [6, 45]; Pieper 2014 [47]; Thomson 2013 [59]
▪ SR with the greatest number of trials (Cooper 2012 [32])
▪ Most recent SR (Pieper 2014 [47]; Cooper 2012 [32])
  2.3 Determine how discrepant data across SRs will be handled in data extraction
   2.3.1 Extract all data, recording discrepancies Becker 2008 [1]; Bolland 2014 [30]; Caird 2015 [31]; Kovacs 2014 [42]; Pieper 2012 [6, 45]; Pieper 2014 [46]; Smith 2011 [57]; Thomson 2010 [58]
   2.3.2 Extract data from only one SR based on a priori eligibility criteria Cooper 2012 [32]; Pieper 2014 [47]
▪ Most recent SR and SR of the highest quality (Pieper 2014 [47])
▪ Highest quality SR (Cooper 2012 [32])
   2.3.3 Extract data element (e.g. effect estimates, quality assessments) from the SR which meets decision rule criteria Bolland 2014 [30]; Cooper 2012 [32]
▪ SR that reports the most complete information on effect estimates (Bolland 2014 [30])
   2.3.4 Reconcile discrepancies through approaches outlined in 2.4 Bolland 2014 [30]; Caird 2015 [31]; Flodgren 2011 [33]; JBI 2015 [40, 41]; Salanti 2011 [3]; Thomson 2010 [58]; Whitlock 2008 [48,49,50,51,52]
  2.4 Determine additional steps to deal with missing data from SRs, or when there is variation in information reported across SRs
   2.4.1 Retrieve reports of the primary studies Bolland 2014 [30]; Caird 2015 [31]; CMIMG 2012 [4]; Flodgren 2011 [33]; Pieper 2012 [6, 45]; Pieper 2014 [47]; Salanti 2011 [3]; Thomson 2010 [58]; White 2009 [49,50,51]
   2.4.2 Contact SR or trial authors, or both, for missing info and/or clarification Bolland 2014 [30]; Flodgren 2011 [33]; JBI 2015 [40, 41]; Whitlock 2008 [49,50,51]
   2.4.3 Search SR or trial registry entries for information Inferred method
   2.4.4 A combination of the above approaches Bolland 2014 [30]; Caird 2015 [31]; Salanti 2011 [3]; Thomson 2010 [58]; Whitlock 2008 [48,49,50,51,52]
   2.4.5 Do not take additional steps to deal with missing data or discrepancies Becker 2008 [1]; Caird 2015 [31]; Foisy 2011 [34]; JBI 2015 [40, 41]
  2.5 Pilot the data extraction forma Cooper 2012 [32]; JBI 2015 [40, 41]
  2.6 Determine the number of overview authors required to extract dataa
   2.6.1 Single, double, or more Becker 2008 [1]; Bolland 2014 [30]; Hartling 2012 [35]; JBI 2015 [40, 41]; Li 2012 [44]; White 2009 [48,49,50,51,52]
   2.6.2 Data extraction versus data checking Becker 2008 [1]; CMIMG 2012 [4]; Singh 2012 [56]; Whitlock 2008 [48,49,50,51,52]
▪ Evaluate a random sample of primary studies to ensure that data abstraction is accurate and reproducible (Whitlock 2008 [48,49,50,51,52])
  2.7 Determine if authors (co-)authored one or several of the reviews included in the overview, and if yes, plan safeguards to avoid bias in data extraction Buchter 2015 [60]
▪ Overview authors do not extract data from their co-authored SRs
  1. CMIMG Comparing Multiple Interventions Methods Group, JBI Joanna Briggs Institute, M-A meta-analysis, SRs systematic reviews
  2. aAdaption of the step from SRs to overviews. No methods evaluation required, but special consideration needs to be given to unique issues that arise in conducting overviews