Table 4 Alignment between typical systematic review processes and a QCA process

Identification of cases to include

Formalized process involving a replicable literature search strategy and study inclusion/exclusion criteria defined by dimensions of population, intervention, comparator, outcomes, timing, and setting.

Non-mechanistic, researcher-directed process that selects cases that share enough background similarity yet offer heterogeneity with respect to explanatory conditions and the outcome.

Good.

Data collection

Information from included studies abstracted and put into structured evidence tables. Information abstracted typically includes study setting, population, intervention description, and outcome estimates.

No standard approach, varies by study, and dependent on the research question and nature of data being used.

Adequate, but could be inefficient if QCA is not planned from the start.

Study/case assessment

Risk of bias: based on researcher assessment of study design and study execution using standard assessment domains.

Other elements: reviewer-assigned study attributes based on assessment of information in published study description or provided from authors in response to a query.

Calibration rubric guides process; process should be transparent and replicable based on rubric.

Not well aligned. Current review processes may need strengthening to support a robust calibration process.

Analysis

Qualitative synthesis—narrative summary with strength of evidence grade(s), sometimes stratified for large or diverse bodies of evidence.

Quantitative synthesis—meta-analytic methods to produce summary effect estimates for direct and sometimes indirect comparisons. Meta-regression to explore heterogeneity of effect.

Configurational based on non-correlational analysis of set relationships. Includes components involving logical minimization of truth table but also narrative summary exploring cases identified in the solutions generated.

Requires separate steps but should be coordinated such that the analyses complement each other.

Presentation of findings

Typically involves text summary, along with supporting detailed evidence tables and figures supporting quantitative synthesis (e.g., forest plots, meta-regression figures) organized using the key questions of the review’s analytic framework.

Typically involves presentation of solutions using symbolic notation and Boolean operators in addition to narrative description of findings. In some cases, Venn diagrams, or X-Y plots can be used to convey findings.

Unclear. Need additional experience to determine appropriate way to present and integrate QCA findings in a typical evidence report.