Table 1 Examples of meta-strategic links and SRDR tools uses within the evidence analysis process
From: Metacognition and evidence analysis instruction: an educational framework and practical experience
| Â | Type of meta-strategic skill | ||
|---|---|---|---|
Step | Synchronic (strategies for weighing alternatives) | Diachronic (assessing current activities in light of downstream goals) | Iterative (evaluating current results based on previous activities) |
Formulate question/conceptual framework (logic model) | Discriminate among question types | Relevance of question to practice | Identify need for preliminary background reading |
Discriminate among question components (PICO, PICOT, PIOS, etc.) | Availability of evidence to answer the question | ||
Identify evidence | Alternative data sources | Will available study designs answer the question? | Do too many or too few results indicate that the question was inadequately formulated? |
How will different methods of reporting outcomes affect the way the question can be answered? | Are search terms adequate to capture comparisons made at the analytic step? | If current SRMAs exist, how does the question for this SRMA provide new insight? | |
SRDR: Abstrakr facilitates consensus among project members for source selection. | |||
Extract and analyze | Alternative platforms or extraction tools, basis for choosing among them | What design, sample, or intervention/exposure characteristics are necessary for later analyses or conclusions? | Do presence of common confounders suggests that the conceptual framework was misspecified? |
What methods of analyzing data are available? What are their relative benefits? | SRDR: tabular structure scaffolds analytic framework for data extraction and a priori subgroup analyses. | Do available outcome measures reported address the question asked? | |
Are outcome measures commensurate? | Outcome definition wizard motivates planning for type of analysis. | SRDR: Customizable questions allows for revision of logic model. | |
SRDR: Customizable fields force planning at two levels: (1) information to be gathered and (2) structure of fields (multiple choice, numerical entry, free text) | |||
Synthesize evidence | What methods of synthesis are available? What are their relative benefits and drawbacks? | How might the synthesis plan need to change in light of available data? | Does observed heterogeneity suggest that important extraction categories were missed? |
What are alternative methods of reporting outcomes? | Are sources of heterogeneity relevant for application identified? | SRDR, OMA: high heterogeneity may indicate important moderator conditions missed in data extraction. | |
SRDR: OMA wizard helps students identify appropriate method of meta-analysis. | SRDR: OMA facilitates post-hoc exploration of sources of heterogeneity. | ||
Evaluate evidence | What are the various threats to confidence in the findings? | Â | What aspects of analyses condition the application of findings? |
Were patterns between outcomes and study characteristics identified and analyzed? | |||