Table 1 Comparison of key features of traditional systematic reviews and CIS

Research question

• A research question is developed a priori and is not changed during the course of the research study.

• A compass question is developed, which can evolve over the course of the research study, in a transparent manner.

Literature search

• Exhaustive literature search

• Exhaustive literature search but can purposively sample literature outside of original search parameters

Article selection

• Static inclusion and exclusion criteria are applied to search results;

• Inclusion and exclusion criteria are iteratively refined to reflect the evolving compass questions (often through successive pilot testing of the criteria using small batches of references with the final criteria applied to all search results);

• Included articles are empirical studies and often consist only of a small number of specific methodological designs (e.g., RCT, CCT, observational studies); and

• Included articles can consist of both empirical and non-empirical (e.g., editorials, essays) analyses that offer insight into the topic under investigation (i.e., articles are not limited to those that meet certain design or quality criteria); and

• All relevant articles meeting inclusion criteria and minimum quality standard are included in the analysis.

• A purposive sample of eligible articles is typically selected based on the unique insights they can provide to the topic under investigation.

Quality rating of articles

• Quality ratings are usually applied to all articles included in a systematic review and used to help interpret the findings.

• Relevance ratings are applied to articles, not quality ratings (i.e., articles are rated on their ability to answer the compass question).

Synthesis

• Data is extracted from all studies meeting the inclusion criteria using an a priori approach; and

• Data is extracted from a purposive sample of studies using an analytic framework where the constructs are iteratively revised and reapplied to the included papers to ensure the constructs are grounded in the data; and

• Findings are synthesized either quantitatively (i.e., in a meta-analysis) or qualitatively, which typically involves reporting findings from studies according to the pre-defined outcomes of interest.

• The analytical constructs that emerged from the iterative analysis are used to derive an explanatory framework.