Skip to main content
Download PDF

Improving the usefulness of evidence concerning the effectiveness of implementation strategies for knowledge products in primary healthcare: protocol for a series of systematic reviews

Systematic Reviews volume 9, Article number: 112 (2020) Cite this article

Abstract

Background

The literature on the implementation of knowledge products is extensive. However, this literature is still difficult to interpret for policymakers and other stakeholders when faced with choosing implementation strategies likely to bring about successful change in their health systems. This work has the particularity to examine the scope of this literature, and to clarify the effectiveness of implementation strategies for different knowledge products. Consequently, we aim to (1) determine the strengths and weaknesses of existing literature overviews; (2) produce a detailed portrait of the literature on implementation strategies for various knowledge products; and (3) assess the effectiveness of implementation strategies for each knowledge product identified and classify them.

Methods

We will use a three-phase approach consisting of a critical analysis of existing literature overviews, a systematic review of systematic reviews, and a series of systematic reviews and meta-analyses. We will follow the Cochrane Methodology for each of the three phases. Our eligibility criteria are defined following a PICOS approach: Population, individuals or stakeholders participating in healthcare delivery, specifically, healthcare providers, caregivers, and end users; Intervention, any type of strategy aiming to implement a knowledge product including, but not limited to, a decision support tool, a clinical practice guideline, a policy brief, or a decision-making tool, a one-pager, or a health intervention; Comparison, any comparator will be considered; Outcomes, phases 1 and 2—any outcome related to implementation strategies including, but not limited to, the measures of adherence/fidelity to the use of knowledge products, their acceptability, adoption, appropriateness, feasibility, adaptability, implementation costs, penetration/reach and sustainability; phase 3—any additional outcome related to patients (psychosocial, health behavioral, and clinical outcomes) or healthcare professionals (behavioral and performance outcomes); Setting, primary healthcare has to be covered. We will search MEDLINE (Ovid), EMBASE, Web of Science, PsycINFO, CINAHL, and the Cochrane Library from their inception onwards. For each phase, two reviewers will independently perform the selection of studies, data extraction, and assess their methodological quality. We will analyze extracted data, and perform narrative syntheses, and meta-analyses when possible.

Discussion

Our results could inform not only the overviews’ methodology but also the development of an online platform for the implementation strategies of knowledge products. This platform could be useful for stakeholders in implementation science.

Systematic review registration

Protocol registered on Open Science Framework, https://osf.io/eb8w2/.

Peer Review reports

Background

Description of the condition

Having an evidence-based knowledge product does not mean that it will be systematically implemented with success and be beneficial for patients. Balas et al. [1] estimated that it takes up to 17 years to translate 14% of original research into practices for the benefit of patients. Given this gap between the production of health evidence and its application in routine practice, increasing attention has been devoted to knowledge translation (KT), and more specifically, to KT [2, 3] and implementation strategies [3,4,5]. Knowledge products considered in implementation strategies include clinical practice guidelines, shared decision-making tools, decision support tools, policy briefs, a one-pager (simple, iconographic, infographic), and a health intervention (technological, pharmacological, behavioral, or management). As a result, there has been a substantial increase in studies examining their effectiveness.

Description of the implementation

Implementation can be defined as an actively planned and deliberatively initiated effort with the intention to translate a given knowledge product into practices in a particular setting and context [6, 7]. Implementation requires characterizing the knowledge product to be implemented as well as the context and setting of implementation [7]. It is also important to (1) determine an adapted process framework or theory; (2) clarify the different stages of the implementation process; (3) identify stakeholders to be involved; (4) recognize facilitators and barriers to implementation using an appropriate framework; (5) identify the corresponding implementation strategies; and (6) target the relevant outcomes and tools to evaluate them [7]. Implementation strategies have been explored extensively in the literature. They can be defined as techniques or methods aiming to improve or optimize the uptake and routine application of complex interventions in clinical care [5]. To be better characterized, implementation strategies have to be named, defined, and specified [5]. The Cochrane Effective Practice and Organization of Care (EPOC) grouped existing implementation strategies in categories targeting respectively healthcare organizations, healthcare professionals, specific types of practice, condition, or setting [3].

How the knowledge product implementation might work

The implementation of a given knowledge product should be guided by one or more frameworks/models/theories depending on its complexity. As suggested by Nilsen [8], an implementation process, identification of its determinants, and evaluation of its outcomes may be supported by a framework/model/theory. Indeed, frameworks/models/theories permit stakeholders to better explain and clarify different steps of the implementation process, i.e., a description of the main steps to translate a given knowledge product into practices followed by the evaluation of outcomes and the sustainability of its use (e.g., the knowledge to action framework [9]). Frameworks/models/theories can also be used to better address the various levels of factors that potentially influence implementation outcomes (e.g., the Consolidated Framework for Implementation Research, CFIR [7]). Finally, there are frameworks/models/theories that have been developed to optimally address the various levels of outcomes that can both explain how different steps of the implementation process took place as well as indicate the timeline for their evaluation [10, 11].

Why it is important to do a series of systematic reviews

According to our preliminary literature search, we found that the literature on the implementation of knowledge products is large. In an effort to further synthesize, several overviews of these systematic reviews [4, 12,13,14,15,16,17,18,19,20,21] have been published in the last 20 years. Overviews attempt to systematically retrieve and summarize the results of multiple systematic reviews on the effectiveness of interventions or treatments for a given condition or public health problem [22]. Their purpose is to contrast the findings of several systematic reviews on the same subject, thus offering decision-makers a broader understanding of the evidence. While, in theory, this type of study has the potential to overview the evidence, its execution is fraught with several challenges. There are guidelines for conducting this type of research [23,24,25,26]; however, several experts who have examined the methodology regarding overviews of systematic reviews over the past decade agreed that existing directives and recommendations are not sufficiently robust and there is great room for improvement [27,28,29].

This literature remains difficult to interpret for policymakers and other stakeholders when faced with having to select implementation strategies likely to bring about successful change in their health systems. Indeed, it remains a challenge to determine what strategy is effective for the implementation of a given knowledge product. Examining the scope of this literature and further clarifying the effectiveness of implementation strategies for different knowledge products could serve to identify original and relevant research questions for future systematic reviews. In turn, these latter may allow us to quantitatively assess the effectiveness of implementation strategies for specific knowledge products. The general aim of this project is to identify, for each category of knowledge product, the most effective implementation strategies for their uptake into healthcare professionals’ clinical practice.

We aim to address the following specific objectives in three distinct phases:

  1. 1.

    Phase 1—to perform a critical analysis of existing literature overviews to determine their strengths and weaknesses. This analysis will not only help us better define our research question for subsequent phases but will also highlight many of the methodological challenges we may encounter.

  2. 2.

    Phase 2—to conduct a systematic review of systematic reviews while taking into account the weaknesses of previously published overviews, and provide a detailed portrait of the literature on various implementation strategies for knowledge.

  3. 3.

    Phase 3—to perform a series of meta-analyses that evaluate the effectiveness of implementation strategies for the different types of knowledge products, using individual randomized controlled trials (RCTs) from systematic reviews included in phase 2. These analyses will provide answers to specific research questions with the most robust experimental study designs, and more detailed ones, in which contextual elements, often absent in overviews, are going to be taken into consideration.

Methods

This study protocol has been registered within the Open Science Framework platform (Registration ID: https://osf.io/eb8w2/). We are reporting it in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) statement (See PRISMA-P checklist in Additional file 2) [30]. We will conduct this project following the recommendations in the Cochrane Handbook for systematic reviews [31].

We will use a three-phase approach (see Fig. 1 for details) including a critical analysis of existing literature overviews, a systematic review of systematic reviews, and a series of systematic reviews and meta-analyses.

Fig. 1
figure 1

Three-phase approach used for the present review

Full size image

Eligibility criteria

In the first two phases of the project, according to a PICOS (Population, Intervention, Comparison, Outcomes, Study design) with knowledge products, studies that meet the following criteria will be considered for inclusion:

Population

The population of interest is individuals or stakeholders participating in healthcare delivery, specifically, healthcare providers, caregivers, and end users (i.e., patients or clients with any conditions).

Intervention

We will consider any type of strategy aiming to implement a knowledge product. The EPOC taxonomy will be used to classify implementation strategies identified (e.g., audit and feedback, educational meetings, interprofessional education) [3].

Knowledge product

As suggested by Pinnock et al. [32], a clear distinction has been made between the implementation strategies and the knowledge products implemented. A knowledge product could be, but not limited to, a decision support tool, a clinical practice guideline, a policy brief, or a decision-making tool, a one-pager (simple, iconographic, infographic), or a health intervention (technological, pharmacological, behavioral, or management). Only implementation studies in which both are reported will be considered in our project.

Comparison

Any comparator will be considered. This includes studies in which two or more implementation strategies for knowledge products are compared (head to head); and studies in which there is no comparator.

Outcome(s)

In phases 1 and 2, we will mainly consider outcomes related to implementation strategies including, but not limited to, the measures of adherence/fidelity to the use of knowledge products, their acceptability, adoption, appropriateness, feasibility, adaptability, implementation costs, penetration/reach, and sustainability [11]. As for phase 3, we will additionally consider the outcomes related to patients (psychosocial, health behavioral, and clinical outcomes) or healthcare professionals (psychosocial, health behavioral, and performance outcomes) that the knowledge products aim to improve.

Study design

This eligibility criterion will be specific to each phase of the project:

  • Phase 1. We will consider any review of reviews on implementation strategies for knowledge products (i.e., a knowledge synthesis in which included studies are any literature reviews).

  • Phase 2. We will consider only systematic reviews (i.e., any study in which authors performed a narrative and/or quantitative synthesis of experimental studies on implementation strategies for knowledge products using a comprehensive and reproducible approach).

  • Phase 3. Only RCTs will be considered.

Setting

Any health domain addressed in primary healthcare will be considered. A review will be included if primary healthcare is covered.

Information sources and search strategy

We will perform comprehensive searches within five electronic databases: MEDLINE (Ovid), EMBASE, Web of Science, PsycINFO, CINAHL, and the Cochrane Library from their inception onwards. The search strategy will be designed by an information specialist using “knowledge transfer” and “implementation strategies” headings (See draft search strategy for MEDLINE/Ovid in Additional file 1). For efficiency considerations, we will not build a new search strategy to identify RCTs, rather, we plan in identifying them from the systematic reviews that we will select in our overview. However, to avoid missing any recently published RCTs in the field, we will update our search strategy by looking for primary studies published after the most recent included reviews. Searches will be conducted within the same electronic databases as in earlier phase. Study designs other than RCTs, for example systematic reviews, quasi-experimental studies and observational studies, will be excluded.

There will be neither language nor literature search date restrictions. The references listed in all eligible studies will also be manually searched in order to identify additional relevant ones.

Data management

We will merge the citations identified from five electronic databases mentioned above in the EndNote software. Then, we will identify and remove the duplicates. The unique citations will be considered for the selection process.

Selection of studies

Two assessors will independently contribute to all steps of the selection process. References identified from relevant electronic databases will be merged, and duplicates removed to obtain a database including unique citations for the study selection process. Assessors will discuss the inclusion criteria to ensure mutual understanding, and pilot test on five percent of unique citations identified to confirm that the evaluation process is reliable. The pilot section will be considered conclusive if the kappa statistic referring to the agreement between assessors is greater than 0.7 [33].

Pairs of assessors will then independently screen for titles and abstracts based on inclusion criteria specific to each phase outlined above. In case of doubt, the citations will be included and considered for full-text reading. Assessors will then independently screen the full texts of references retained at the first step. Any disagreement will be resolved by consensus or by involving a third assessor.

Assessors will include overviews for phase 1 and systematic reviews for phase 2, based on the eligibility criteria mentioned above. For phase 3, the following procedure will be adopted for the identification of relevant RCTs from each review considered: first, we will go through the list including primary studies to identify RCTs that are clearly reported as such by the review’s authors. In the case, RCTs are not, we will select the complete list of primary articles included. Removal of duplicates and screening based only on the study design will be done by two independent assessors.

Data extraction

For each phase, a pilot data extraction will be done on 5% of selected articles. The data extraction will be pursued only when the pilot is conclusive. This process will be independently performed by two assessors. Discrepancies between them will be resolved by a third author.

Phase 1

The following data will be extracted: the first author’s name; the year of publication; the focus of the overview, based on the scope of PICO elements; the number and name of databases consulted; the period of the literature search; the strategies used to update the literature; the type and number of reviews that are included; the type of reviews (Cochrane, non-Cochrane, both); the strategies to deal with the overlap between reviews; the strategies to deal with conflicting results of reviews; the quality assessment of reviews; the type of synthesis performed (narrative, meta-analysis or both); the assessment of the quality of evidence, and the limitations reported by overview authors.

Phase 2

We will extract data concerning the following elements:

  • Review details. First author’s name, year of publication, type, objectives, registration information.

  • Literature search details. Number and names of electronic databases, search for gray literature, search period, restrictions, design and number of primary studies.

  • Characteristics of participants. Name, profile, total number, age (mean, median, and/or range), gender (percentage of men or women).

  • Characteristics of implementation strategies. Components described according to the EPOC taxonomy, knowledge products implemented (e.g., clinical guidelines, health intervention).

  • Characteristics of outcomes. Name, type of measurement tools used (self-administered, objective, both). We will use Proctor et al. taxonomy to guide the classification of outcomes [11].

  • Assessment of study quality. Name of the tool used, result.

  • Type of synthesis. Narrative, meta-analysis, or both.

  • Assessment of publication bias. Method used (funnel plots, statistical tests, both), and treatment of it if present.

  • Assessment of the quality of evidence. Name of the tool used, level of evidence.

  • Conclusion of reviews. Information reported by authors when they conclude (e.g., main results and limitations).

Phase 3

For each RCT considered in this phase, we will extract data concerning the following elements:

  • Characteristics of the study. First author’s name, year of publication, country, language, healthcare domain, setting, study design, sampling, and recruitment method.

  • Characteristics of participants. Size of eligible population for both healthcare providers and end users, total sample size, sample sizes in implementation strategy, and control groups, unit of allocation, mean age or range, sex, race/ethnicity (percentage of Caucasians), socioeconomic level of the sampled population, and profile.

  • Characteristics of implementation strategies (description of the intervention). Components (according to the EPOC taxonomy), underlying framework, content, number of participating centers, mode of delivery (e.g., in person and online), actual duration (e.g., number of hours), actual frequency (e.g., number of sessions), methods used (e.g., interactive or didactic techniques, web applications), course materials used (e.g., web capsules, textbooks), adherence/fidelity, and content of the intervention offered to control group.

  • Characteristics of knowledge products implemented. Name, format and content.

  • Characteristics of outcomes and analysis methods used. Names of interest outcomes, measurement tools, range of scale and analysis methods used, unit of analysis, intention to treat analysis or not, retention rates, and effect measures of implementation strategies.

  • Effects of implementation strategies. Any information useful to estimate the effect size and its 95% confidence interval will be extracted. For continuous outcomes, we will consider the standardized mean difference as effect size requiring the sample size, mean score, and standard deviation in each group studied. For dichotomous outcomes, we will consider the odds ratio as effect size requiring the sample size and number of events in each group studied. We will also collect the value of effect size (crude and adjusted) and its confidence interval estimated in the studies. Information about confounding factors will as well be extracted.

When necessary, corresponding or first authors will be contacted to obtain information about missing data in their studies.

Quality assessment of studies

The quality of overviews retrieved in phase 1 will not be assessed since there is no methodological tool designed for this purpose. Moreover, the objective of this phase is not to draw any conclusions on the effectiveness of implementation strategies from these overviews.

For phase 2, we will assess the methodological quality of selected systematic reviews using the AMSTAR 2 tool [34]. Those meeting the criteria of good or medium quality will be retained for the rest of the process, i.e., will be considered for the next steps of the review of reviews and for the identification of RCTs in phase 3.

As for phase 3, RCTs extracted from systematic reviews will be assessed with the Cochrane Risk of Bias (RoB) 2.0 tool [35].

Both the quality assessment of systematic reviews as the assessing risk of bias for RCTs will be performed independently by two authors. In a pilot assessment of two or four studies included, they will agree on a common understanding of the definitions, criteria, and guidelines provided in the tool to achieve a more objective assessment. The assessment of all studies included will do once the pilot test is conclusive. When needed, a third reviewer will be invited to help reach consensus.

Data synthesis

Phase 1

We will perform a descriptive analysis using counts (number and percentage) to summarize data collected followed by carrying out a critical analysis on the latter considering the following methodological elements: literature search, methodological limitations reported in overviews, overlap between reviews included in overviews, types of data synthesis, quality of reviews included in overviews, and level of evidence.

Phase 2

We will produce a descriptive Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart of the selection process for all included reviews [36]. We will then describe the characteristics of the latter, populations, implementation strategies, and outcomes assessed. We will also analyze the data on publication bias and quality of evidence assessment. Finally, evaluation data of included reviews from the AMSTAR 2 tool [34] will be presented using a graph and table.

Phase 3

First, we will classify RCTs according to the type of knowledge product being implemented. These may include but not limited to clinical guidelines, decision support tools, research summaries, one-pagers (simple, iconographic, infographic), or other health interventions (technological, pharmacological, behavioral, or management).

Second, we will present a descriptive flowchart of the RCTs included in each meta-analysis, according to the PRISMA guidelines [36]. We will describe by frequencies (number and percentage) the characteristics of studies, populations, implementation strategies, and outcomes. Risk of bias levels will also be described for all included RCTs.

Finally, for each outcome and knowledge product, we will determine if there are sufficient data to perform a meta-analysis. If not, we will conduct a narrative synthesis on the effects of implementation strategies, and include in tables and figures to aid in data presentation. If so, we will do a meta-analysis. For each outcome and knowledge product, the random effect model will be used to estimate the pooled effect size of an implementation strategy and its 95% confidence interval, as we anticipate heterogeneity among the RCTs concerning the types of implementation intervention and population [31, 37]. For dichotomous data, the effect size will be expressed as a risk ratio or odds ratio; for continuous data, it will be provided as a standardized mean difference if different measures are used for the same outcome. For cluster RCTs, the analysis will be adjusted for clustering to avoid unit-of-analysis errors [31]. The influence of clinical and methodological heterogeneity on the observed effects will be discussed. Subgroup analyses will be carried out if necessary according to characteristics of the studies, the populations and the implementation interventions mentioned above. Statistical heterogeneity will be assessed using the Higgins’ I square statistic [38, 39]. A funnel plot will be generated to assess publication bias if 10 or more studies are included in the meta-analysis [40]. Statistical tests for funnel plot asymmetry (e.g., Egger’s regression, Begg’s test, and Harbord’s test) will be performed where appropriate [31, 40].

Sensitivity analyses will be carried out excluding the RCTs with high risk of bias from pooled effect size estimates. For each outcome and knowledge product, we will also explore the influence of each RCT by removing its individual effect size from the pooled estimation. These different analyses will allow us to evaluate the robustness of our results.

Assessment of the quality of evidence

The quality of evidence for each outcome and knowledge product will be assessed with the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) to reduce the misinterpretation of our review’s results [41]. The GRADE tool is based on five criteria for each individual study, namely risk of bias, indirectness of evidence, data heterogeneity, imprecision of effect size estimates, and risk of publication bias [41]. Overall assessment will be rated very low, low, moderate, and high for each outcome [41].

Discussion

Our project aims to improve the usefulness of evidence on implementation strategies for knowledge products in primary healthcare. Our expected findings will have both methodological and practice implications. First, findings from the present project, mainly the ones related to the critical analysis on methodology, will contribute to methodological aspects for designing and conducting literature overviews. To this end, we will identify methodological limitations in overviews, and formulate corresponding recommendations with our collaborative team of methodologists. These recommendations will be applied to our own overview in order to determine their applicability. Second, our findings will also serve to provide a list of implementation strategies for various knowledge products. This list will include established effective implementation strategies, those potentially effective, and the non-effective ones for each knowledge product identified. These findings will inform the development of an online platform on implementation strategies for knowledge products in primary healthcare.

Apart from the methodological challenges already taken into account in our methods section, we anticipate other limitations in the realization of our project. First, there may be discrepancies between what we have planned in this protocol and what we will finally do. So, we plan for justifying them and documenting important protocol amendments. Second, doing a series of systematic reviews can take a long time, so it could be challenging to keep update the literature search of interest. To save time, we plan to create parallel review teams that will simultaneously work at the steps of study selection, data extraction, and/or study quality assessment for the critical analysis of existing overviews and the systematic reviews of systematic reviews. For each systematic review of RCTs on specific research questions, we plan to systematically update our literature search to avoid missing any recently published RCTs.

We will use passive and active strategies to disseminate our findings.

Passive strategies

Findings from this project will be published in leading journals in the field intended for knowledge users and stakeholders in implementation science. We plan to produce a series of manuscripts for publication in peer-reviewed journals organized in the following:

  • A manuscript as a preamble in which we will report results from our critical analysis of existing overviews

  • A paper describing results from our overview and our original methodology

  • At least four manuscripts corresponding to each specific meta-analysis or narrative synthesis we will perform. In these papers, we will identify effective implementation strategies for each type of knowledge product

The findings will also be presented at local, national, and international conferences (number equal to four or six). Specialized conferences whose themes related to knowledge translation in health, implementation science, or primary healthcare will be privileged (e.g., the Annual Scientific Meeting of Knowledge Translation Canada, the Annual Conference on the Science of Dissemination and Implementation in Health, or the Annual Meeting of North American Primary Care Research Group). In these cases, we will solicit our expert partners on knowledge translation to participate as credible messengers. Finally, we will design a database on our website (http://unitesoutiensrapqc.ca/) on the effectiveness of implementation strategies stratified according to the types of outcomes and knowledge products. This database will be useful for anyone who wants to design an implementation study on a knowledge product.

Active strategies

Findings from this project are of critical relevance to each of the following knowledge platforms or teams: the Knowledge Translation Components of Strategy for Patient-Oriented Research (SPOR) - Support for People and Patient-Oriented and Trials (SUPPORT) Units, the Réseau provincial de recherche en adaptation-réadaptation (REPAR), and the Centre de recherche interdisciplinaire en réadaptation (CRIR). One of the mandates from these organizations is to guide the application of evidence-based knowledge in primary care clinical practices, as well as in rehabilitation. Since our findings will be the product of a systematic approach, we will integrate them into the range of services we offer, including training workshops (e.g., for conferences or as interactive online training with a follow-up) targeting specific audiences such as health administrators/decision-makers/health professionals and patients. We will also develop a platform on implementation strategies for knowledge products in primary healthcare. A video capsule in plain language will be created to support the use of this platform and to train the potential users including healthcare professionals, health system decision-makers, patients, and researchers involving in implementation science. We will use various communication channels to publicize our platform: social media (e.g., Facebook, Twitter), our bimonthly newsletter (more than 500 subscribers), and clients’ emails through the database, particularly those who accepted to be contacted for our advertisements.

Availability of data and materials

All data and materials used at this step are available in our protocol. There are currently no additional data or materials. However, all data and materials used during the review will be available from the corresponding author.

Abbreviations

AMSTAR:

A MeaSurement Tool to Assess systematic Reviews

CFIR:

Consolidated Framework for Implementation Research

CINHAL:

Cumulative Index to Nursing and Allied Health Literature

CIRR:

Centre de recherche interdisciplinaire en réadaptation

EMBASE:

Excerpta Medica dataBASE

EPOC:

Effective Practice and Organization of Care

GRADE:

Grading of Recommendations, Assessment, Development and Evaluation

KT:

Knowledge translation

MEDLINE:

Medical Literature Analysis and Retrieval System Online

REPAR:

Réseau provincial de recherche en adaptation-réadaptation

RCTs:

Randomized controlled trials

PICO:

Population, Intervention, Comparison, Outcomes

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PRISMA-P:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols

SPOR:

Strategy for Patient-Oriented Research

SUPPORT:

Support for People and Patient-Oriented and Trials

References

  1. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med. 2000;160(3):301–8.

    CAS  PubMed  Google Scholar 

  2. LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012;12:751.

    PubMed  PubMed Central  Google Scholar 

  3. Effective Practice and Organisation of Care (EPOC). EPOC taxonomy. 2015.

  4. Pantoja T, Opiyo N, Lewin S, Paulsen E, Ciapponi A, Wiysonge CS, et al. Implementation strategies for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst Rev. 2017;9:Cd011086.

    PubMed  Google Scholar 

  5. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139.

    PubMed  PubMed Central  Google Scholar 

  6. Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, et al. Making sense of complexity in context and implementation: the context and implementation of complex interventions (CICI) framework. Implement Sci. 2017;12(1):21.

    PubMed  PubMed Central  Google Scholar 

  7. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    PubMed  PubMed Central  Google Scholar 

  8. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.

    PubMed  PubMed Central  Google Scholar 

  9. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

    PubMed  Google Scholar 

  10. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    PubMed  Google Scholar 

  12. Brouwers MC, Garcia K, Makarski J, Daraz L. The landscape of knowledge translation interventions in cancer control: what do we know and where to next? A review of systematic reviews. Implement Sci. 2011;6:130.

    PubMed  PubMed Central  Google Scholar 

  13. Chan WV, Pearson TA, Bennett GC, Cushman WC, Gaziano TA, Gorman PN, et al. ACC/AHA special report: clinical practice guideline implementation strategies: a summary of systematic reviews by the NHLBI implementation science work group: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol. 2017;69(8):1076–92.

    PubMed  Google Scholar 

  14. Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J. Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Syst Rev. 2012;1:36.

    PubMed  PubMed Central  Google Scholar 

  15. Durieux P, Ravaud P, Dosquet P, Durocher A. Effectiveness of clinical guideline implementation strategies: systematic review of systematic reviews. Gastroenterologie clinique et biologique. 2000;24(11):1018–25.

    CAS  PubMed  Google Scholar 

  16. Durieux PR, P. Efficacité des méthodes de mise en œuvre des recommandations médicales. Haute Autorité de Santé, professionnelles Sdbp; 2014.

  17. Goldner EM, Jenkins EK, Fischer B. A narrative review of recent developments in knowledge translation and implications for mental health care providers. Canadian Journal of Psychiatry Revue Canadienne de Psychiatrie. 2014;59(3):160–9.

    PubMed  PubMed Central  Google Scholar 

  18. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Medical Care. 2001;39(8 Suppl 2):Ii2–45.

    CAS  PubMed  Google Scholar 

  19. Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care--effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open. 2015;5(12):e009993.

    PubMed  PubMed Central  Google Scholar 

  20. Lewin S, Lavis JN, Oxman AD, Bastias G, Chopra M, Ciapponi A, et al. Supporting the delivery of cost-effective interventions in primary health-care systems in low-income and middle-income countries: an overview of systematic reviews. Lancet (London, England). 2008;372(9642):928–39.

    Google Scholar 

  21. Prior M, Guerin M, Grimmer-Somers K. The effectiveness of clinical guideline implementation strategies--a synthesis of systematic review findings. Journal of evaluation in clinical practice. 2008;14(5):888–97.

    PubMed  Google Scholar 

  22. Pieper D, Buechter R, Jerinic P, Eikermann M. Overviews of reviews often have limited rigor: a systematic review. Journal of clinical epidemiology. 2012;65(12):1267–73.

    PubMed  Google Scholar 

  23. Becker LA OA. Chapter 22: Overviews of reviews. Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons; 2008.

  24. Biondi-Zoccai G. Umbrella reviews evidence synthesis with overviews of reviews and meta-epidemiologic studies. Cham: Springer; 2016. Available from: Accès via SpringerLink http://dx.doi.org/https://doi.org/10.1007/978-3-319-25655-9 http://ariane.ulaval.ca/cgi-bin/recherche.cgi?qu = a2547826.

    Google Scholar 

  25. Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International journal of evidence-based healthcare. 2015;13(3):132–40.

    PubMed  Google Scholar 

  26. Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11(1):15.

    PubMed  PubMed Central  Google Scholar 

  27. Pieper D, Antoine SL, Morfeld JC, Mathes T, Eikermann M. Methodological approaches in conducting overviews: current state in HTA agencies. Research synthesis methods. 2014;5(3):187–99.

    PubMed  Google Scholar 

  28. Pieper D, Pollock M, Fernandes RM, Buchter RB, Hartling L. Epidemiology and reporting characteristics of overviews of reviews of healthcare interventions published 2012-2016: protocol for a systematic review. Systematic reviews. 2017;6(1):73.

    PubMed  PubMed Central  Google Scholar 

  29. McKenzie JE, Brennan SE. Overviews of systematic reviews: great promise, greater challenge. Syst Rev. 2017;6(1):185.

    PubMed  PubMed Central  Google Scholar 

  30. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

    PubMed  PubMed Central  Google Scholar 

  31. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA. Cochrane handbook for systematic reviews of interventions version 6.0 (Updated July 2019). Cochrane Available from wwtrainingcochraneorg/handbook. 2019.

  32. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI): explanation and elaboration document. BMJ open. 2017;7(4):e013318.

    PubMed  PubMed Central  Google Scholar 

  33. Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;33(2):363–74.

    CAS  PubMed  Google Scholar 

  34. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358:j4008.

    Google Scholar 

  35. Sterne JAC, Savovic J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ (Clinical research ed). 2019;366:l4898.

    Google Scholar 

  36. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ (Clinical research ed). 2009;339:b2700.

    Google Scholar 

  37. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta-analysis: John Wiley & Sons; 2011.

  38. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.

    Google Scholar 

  39. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ (Clinical research ed). 2003;327(7414):557–60.

    PubMed Central  Google Scholar 

  40. Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ (Clinical research ed). 2011;343:d4002.

    Google Scholar 

  41. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical research ed). 2008;336(7650):924–6.

    PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Mr. Dominique Fournier, a scientific English language editor, for his kind help with the manuscript.

Funding

This review is funded by the Quebec Strategy for Patient-Oriented Research (SPOR) - Support for People and Patient-Oriented and Trials (SUPPORT) Unit. The Unit is supported by the Canadian Institutes of Health Research (CIHR) and provincial partners including the Ministère de la Santé et des Services sociaux (MSSS) du Québec and the Fonds de recherche du Québec – Santé (FRQ-S). The funders have no role in developing the protocol.

Author information

Authors and Affiliations

  1. Health and Social Services Systems, Knowledge Translation and Implementation Component of the Quebec SPOR SUPPORT Unit, Laval University, Quebec, Quebec, Canada

    Hervé Tchala Vignon Zomahoun, José Massougbodji, Claude Bernard Uwizeye, Nathalie Rheault, Ali Ben Charif, Sébastien Tchoubi, Serigne Abib Gaye & France Légaré

  2. Centre intégré universitaire de santé et de services sociaux de la Capitale-Nationale (CIUSSS-CN), Quebec, Québec, Canada

    Hervé Tchala Vignon Zomahoun, José Massougbodji, Claude Bernard Uwizeye, Nathalie Rheault, Ali Ben Charif, Ella Diendéré, Léa Langlois & France Légaré

  3. Centre de recherche sur les soins et les services de première ligne de l’Université Laval (CERSSPL-UL), Pavillon Landry-Poulin – 2525, Chemin de la Canardière, Québec, G1J 0A4, Canada

    Hervé Tchala Vignon Zomahoun, José Massougbodji, Claude Bernard Uwizeye, Ali Ben Charif, Ella Diendéré, Léa Langlois & France Légaré

  4. Department of Social and Preventive Medicine, Laval University, Quebec, Quebec, Canada

    Hervé Tchala Vignon Zomahoun & José Massougbodji

  5. School of Physical and Occupational Therapy, Faculty of Medicine, McGill University, Montreal, Quebec, Canada

    André Bussières & Aliki Thomas

  6. Centre de recherche interdisciplinaire en réadaptation du Montréal métropolitain (CRIR), Montreal, Quebec, Canada

    André Bussières, Aliki Thomas & Dahlia Kairy

  7. Réseau provincial de recherche en adaptation-réadaptation (REPAR), Montreal, Quebec, Canada

    André Bussières, Aliki Thomas & Dahlia Kairy

  8. Département chiropratique, Université du Québec à Trois-Rivières, Trois-Rivières, Quebec, Canada

    André Bussières

  9. School of Rehabilitation, Université de Montréal, Montreal, Quebec, Canada

    Dahlia Kairy

  10. Canada Research Chair in Shared Decision Making and Knowledge Translation, Laval University, Quebec, Quebec, Canada

    Ali Ben Charif, Ella Diendéré & France Légaré

  11. Department of Family Medicine and Emergency Medicine, Laval University, Quebec, Quebec, Canada

    Ali Ben Charif & France Légaré

Authors
  1. Hervé Tchala Vignon Zomahoun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Massougbodji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. André Bussières
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aliki Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dahlia Kairy
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Claude Bernard Uwizeye
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nathalie Rheault
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ali Ben Charif
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ella Diendéré
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Léa Langlois
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Sébastien Tchoubi
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Serigne Abib Gaye
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. France Légaré
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HTVZ, JM, AB, AT, DK, and FL contributed to the conception and design of the project. HTVZ drafted the protocol with support of JM. All authors provided a critical review on the initial protocol and its subsequent versions. All authors read and approved the final manuscript. Only the authors are responsible for the information provided or views expressed in this protocol.

Authors’ information

No additional information.

Corresponding author

Correspondence to France Légaré.

Ethics declarations

Ethics approval and consent to participate

For this project, the ethical approval is not required because we will only use the literature as the data source. Also, there is no requirements of an inform consent.

Consent for publication

Our manuscript contains no individual person’s data in any form including any individual details, images, or videos.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Detailed search strategy in Ovid Medline.

Additional file 2.

PRISMA-P 2015 Checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zomahoun, H.T.V., Massougbodji, J., Bussières, A. et al. Improving the usefulness of evidence concerning the effectiveness of implementation strategies for knowledge products in primary healthcare: protocol for a series of systematic reviews. Syst Rev 9, 112 (2020). https://doi.org/10.1186/s13643-020-01382-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-020-01382-x

Keywords

Systematic Reviews

ISSN: 2046-4053