Skip to main content

Table 1 Examples of meta-strategic links and SRDR tools uses within the evidence analysis process

From: Metacognition and evidence analysis instruction: an educational framework and practical experience

  Type of meta-strategic skill
Step Synchronic
(strategies for weighing alternatives)
(assessing current activities in light of downstream goals)
(evaluating current results based on previous activities)
Formulate question/conceptual framework (logic model) Discriminate among question types Relevance of question to practice Identify need for preliminary background reading
Discriminate among question components (PICO, PICOT, PIOS, etc.) Availability of evidence to answer the question
Identify evidence Alternative data sources Will available study designs answer the question? Do too many or too few results indicate that the question was inadequately formulated?
How will different methods of reporting outcomes affect the way the question can be answered? Are search terms adequate to capture comparisons made at the analytic step? If current SRMAs exist, how does the question for this SRMA provide new insight?
SRDR: Abstrakr facilitates consensus among project members for source selection.
Extract and analyze Alternative platforms or extraction tools, basis for choosing among them What design, sample, or intervention/exposure characteristics are necessary for later analyses or conclusions? Do presence of common confounders suggests that the conceptual framework was misspecified?
What methods of analyzing data are available? What are their relative benefits? SRDR: tabular structure scaffolds analytic framework for data extraction and a priori subgroup analyses. Do available outcome measures reported address the question asked?
Are outcome measures commensurate? Outcome definition wizard motivates planning for type of analysis. SRDR: Customizable questions allows for revision of logic model.
SRDR: Customizable fields force planning at two levels: (1) information to be gathered and (2) structure of fields (multiple choice, numerical entry, free text)
Synthesize evidence What methods of synthesis are available? What are their relative benefits and drawbacks? How might the synthesis plan need to change in light of available data? Does observed heterogeneity suggest that important extraction categories were missed?
What are alternative methods of reporting outcomes? Are sources of heterogeneity relevant for application identified? SRDR, OMA: high heterogeneity may indicate important moderator conditions missed in data extraction.
SRDR: OMA wizard helps students identify appropriate method of meta-analysis. SRDR: OMA facilitates post-hoc exploration of sources of heterogeneity.
Evaluate evidence What are the various threats to confidence in the findings?   What aspects of analyses condition the application of findings?
Were patterns between outcomes and study characteristics identified and analyzed?
  1. Examples of use of SRDR suite of tools for evidence analysis instruction in italics
  2. PICO, PICOT, PIOS question formulation heuristics comprising a combination of the following: population, intervention, comparator, outcome, time, or study design, OMA OpenMeta[Analyst], SRDR Systematic Review Data Repository, SRMA systematic review/meta-analysis