Implementation of evidence-based practice for alcohol and substance use disorders: protocol for systematic review
Systematic Reviews volume 9, Article number: 25 (2020)
Whilst effective treatments exist for substance use and alcohol use disorders, they are not commonly practised. Studies have shown that only a small percentage of services provide evidence-based treatments such as addiction medications or psychosocial therapies. Although there is a growing body of literature on evidence-based treatment, no synthesis of research on the implementation of evidence-based addiction treatment exists. This proposed systematic review will synthesise and evaluate the effectiveness of implementation programmes in the treatment of patients with drug and alcohol problems using the Consolidated Framework for Implementation Research (CFIR) framework.
We will search (from inception onwards) PubMed/MEDLINE, Cochrane Library, PsycINFO, Web of Science and CINAHL. Eligible studies will be clinical trials (e.g. randomised controlled trials, non-randomised controlled trials) and observational studies (e.g. before-and-after studies, interrupted time series) evaluating strategies used to implement evidence-based psychosocial treatments for alcohol and substance use disorders. The primary outcomes will be related to the implementation, service system, or clinical practice (e.g. acceptability, implementation costs, feasibility). Two researchers will independently screen all citations, full-text articles and abstract data. Risk of bias of individual studies will be appraised using appropriate tools. A narrative synthesis will be provided.
This project aims to provide evidence to help guide the design of translational research programmes to improve implementation of evidence-based care in drug and alcohol settings. Findings from the study will specify effective strategies for domains of influence including (1) intervention characteristics (e.g. evidence strength and quality, adaptability), (2) outer setting (e.g. patient needs and resources, external policies and incentives), (3) inner setting (e.g. implementation climate, readiness for implementation), (4) individuals involved (e.g. self-efficacy, knowledge and beliefs about the intervention) and (5) the implementation process (e.g. engaging members of the organisation, executing the innovation). Identified gaps in knowledge will guide further study.
Systematic review registration
The existence of a gap between clinical evidence and clinical practice is well established in the health services literature [1, 2]. Studies from the USA, the Netherlands, Britain, Canada and Australia suggest that 30 to 40% of patients do not receive evidence-based treatments, and up to 25% of patients receive treatments that are either inappropriate or even harmful [3, 4]. The general trends in the gap between research and practice are equally evident in the treatment of addiction where evidence-based treatments are not widely practised [5,6,7]. Specifically, it has been estimated that no more than 25% of community-based services provide evidence-based treatments such as addiction medications, psychosocial therapies, or integrated services for clients with substance use disorders (SUDs) or co-occurring mental health disorders . Whilst effective treatments exist for SUDs, they are not commonly practised [7, 9, 10].
A complex range of barriers operating at multiple levels of healthcare delivery must be addressed in order to bridge this gap, such as the patient level, the provider team or group level, the organisational level and the market or policy level . Historically, there has been a much stronger focus on identifying evidence-based interventions than there has been on developing evidence-based implementation strategies . Interventions that are validated under controlled conditions may not be effective when implemented in substance use disorder practice settings, whereby some features impact the acceptability, uptake and appropriateness of innovations and demand a level of flexibility in the approach that may compromise the efficacy of the intervention [13, 14]. For an intervention to be successful, it must therefore be feasible within the context it is intended so that the fidelity of the intervention can be reproduced and maintained .
The field of implementation science emerged accordingly as an organised and resourced approach to the study of implementation, allowing for the accumulation of knowledge about how to ensure that evidence-based practices are delivered to the necessary patients and thus bridging the gap between evidence and practice . Without implementation science and the knowledge that it provides about the effectiveness of the implementation process, it is not possible to tell whether poor treatment outcomes are due to the intervention or the way it was implemented [16, 17]. The addictions field is grossly underrepresented within implementation science , despite high disease burden  and large gap between treatment and evidence. Thus, there is a particular need to apply this science to the implementation of evidence-based treatment of SUDs.
Various theories, models and frameworks have been proposed to synthesise the knowledge base derived from implementation research, to describe the process of translating research into practice, to understand the determinants of implementation outcomes and to evaluate implementation . The Consolidated Framework for Implementation Research  has been recommended as a taxonomy that may be appropriate for use in the context of SUD research . The Consolidated Framework for Implementation Research (CFIR) consolidates the plethora of terms and concepts generated by implementation researchers into five domains of influence: (1) intervention characteristics (e.g. evidence strength and quality, adaptability), (2) outer setting (e.g. patient needs and resources, external policies and incentives), (3) inner setting (e.g. implementation climate, readiness for implementation), (4) individuals involved (e.g. self-efficacy, knowledge and beliefs about the intervention) and (5) the implementation process (e.g. engaging members of the organisation, executing the innovation). It has been suggested that the CFIR could assist with differentiating the core components from the adaptive components of the intervention [7, 21], utilising formative evaluation in implementation research and developing and testing models that can assist with the prediction of implementation outcomes and sustainability in specific contexts . Additionally, the CFIR has been used to assess the comprehensiveness of strategies employed to implement evidence-based practice in health and mental health settings [22, 23]. When compared to other theoretical approaches, it has been categorised as a ‘determinant framework’ because it is comprised of domains of determinants that are related to implementation outcomes . Amongst the determinant frameworks included in this category, the CFIR provides the most comprehensive approach to synthesising implementation research. The CFIR has therefore been selected as the evaluation framework for this review given its appropriateness, as well as its utility in previous reviews and the SUD context in particular.
The number of empirical evaluations testing implementation strategies in mental health settings is ‘dwarfed’ by the broader field of health care . This argument is particularly pertinent in the SUD setting, where implementation research is lacking . Existing reviews have tended to focus on prevention (e.g. [25, 26]), treatment efficacy (e.g. [27, 28]) and specific interventions (e.g. ). Those that have specifically identified implementation strategies have addressed specific factors (e.g. ) or relationships between factors (e.g. ) influencing the implementation but have not provided a comprehensive account of implementation effectiveness. We now aim to synthesise information on implementation strategies utilised in the SUD field in general and using the CFIR framework to guide analysis.
The objectives of this systematic review are to synthesise and evaluate the effectiveness of implementation programmes in the treatment of patients with drug and alcohol problems.
We aim to evaluate implementation effectiveness with regards to the five domains of influence outlined by the CFIR framework. By providing this overview, we aim to provide evidence to help guide the design of translational research programmes to improve implementation of evidence-based care in drug and alcohol settings.
The present review protocol is being reported in accordance with the reporting guidance the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) statement  (see PRISMA-P checklist in Additional file 1). This review protocol was registered within the International Prospective Register of Systematic Reviews (PROSPERO) (registration number CRD42019123812).
Criteria for considering studies for this review are classified by the following:
In order to meet inclusion criteria, studies must involve an evaluation of implementation strategies used to transfer an evidence-based psychosocial treatment or treatment guideline into clinical practice in SUD settings. Psychosocial treatments include any attempt to affect change in patients’ substance use through behaviour, cognition, affect, interpersonal relationships or environment (e.g. employment, housing). Participants in these studies may include any clinician providing psychosocial interventions to SUD patients accessing outpatient or inpatient drug and alcohol services. ‘Clinician’ is defined as an individual employed to implement change in SUD patients’ substance use using psychosocial treatments exclusively. As such, studies will be excluded from the review if they focus on the development of psychometric instruments, drugs in sport, harm prevention or community awareness.
The psychosocial intervention must be evidence-based and provide clear recommendations for practice. Studies will be excluded if they involve physiological, pharmacological or education-based interventions. Information including the nature of desired change, strategies employed, source of the intervention, mode of delivery (individual or group), identification of who delivered the intervention and the timing, duration, and frequency of the intervention needs to be stated clearly. Only ethically approved studies will be considered.
Comparator and study design
Only studies with a comparison group will be included. Comparisons may be made before and after the administration of the intervention, between two or more forms of intervention, or between different types of intervention(s) (or no intervention). We will include randomised controlled trials (RCTs), non-randomised controlled trials, observational studies including before-and-after studies, and time series analyses.
Primary study outcomes will be related to the implementation, service system or clinical practice. Specifically, outcomes will include infidelity, attitudes toward or satisfaction with the intervention (acceptability), and adoption, appropriateness of the intervention to the target population, implementation costs, the feasibility of the intervention within the setting, and the sustainability of the intervention after implementation. The length of post-intervention follow-up period must be specified and any possible ceiling effects identified. Outcomes will be related to the effectiveness of the implementation process, as distinct from the efficacy of the intervention itself.
Since SUD inpatient and outpatient treatment settings that provide counselling services to patients are the focus of the review, settings such as primary care, criminal justice or those investigating cross-cultural factors will be excluded from the review.
The following electronic databases will be searched (from inception onwards): PubMed/MEDLINE, Cochrane Library, PsycINFO, Web of Science and CINAHL. Additionally, we will conduct reference searches of relevant reviews and articles. Similarly, a grey literature search will be done with help of Google and the Grey Matters tool which is a checklist of health-related sites organised by topic. The tool is produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) .
The search will include all relevant peer-reviewed studies. The search will be conducted across 4 relevant concepts (see draft strategy in Additional file 2): (1) implementation, (2) evidence-based practices, (3) substance use service setting/drug and alcohol service setting and (4) eligible research designs. Specific terms used to search these concepts will be adapted from a recent systematic review conducted in the mental health field . The MEDLINE draft search strategy is available in Additional file 2.
Selection and data extraction
Two reviewers will screen all articles identified from the search independently. First, titles and abstracts of articles returned from initial searches will be screened based on the eligibility criteria outlined above. Second, full texts will be examined in detail and screened for eligibility. Third, references of all considered articles will be hand-searched to identify any relevant report missed in the search strategy by two reviewers independently. Any disagreement between reviewers will be resolved by discussion to meet a consensus. EndNote version X9 (Clarivate Analytics) will be used to manage all records.
Two researchers will extract data and organise it into variables based on the Cochrane Effective Practice and Organisation of Care Review Group (EPOC) Data Abstraction Form (e.g. clinical interventions, strategies, outcomes and results), the conceptual model of Proctor et al.  (implementation, service system and clinical outcomes), information about any specific implementation frameworks used, and a checklist of items aligned with the domains and subdomains of the CFIR (i.e. subdomains associated with intervention characteristics, outer setting, inner setting, characteristics of individuals and the implementation process). This method was used effectively in two previous reviews [23, 34] as a means of categorising the types of implementation strategies addressed by each of the studies included in the review.
Risk of bias of individual studies
All included studies will be critically evaluated by two researchers independently using the Revised Cochrane risk-of-bias tool (RoB 2) . The RoB 2 provides a systematic assessment across five domains of bias (the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome and selection of the reported results) to assess quality of the article per outcome. For cluster-randomised studies, an additional domain must be used when assessing the randomization process. Any non-randomised studies of interventions will be assessed using the ROBINS-I tool is a risk of bias tool .
We do not anticipate having sufficient studies for a meta-analysis and therefore plan to perform a narrative synthesis. The main methods of synthesis will most likely involve tabulation, textual descriptions, qualitative synthesis of themes and vote-counting . We will synthesise the findings from the included articles using the CFIR framework. If any issues requiring sensitivity analyses arise during the review process, such analyses will be conducted.
Meta-biases such as outcome reporting bias will be evaluated by ensuring that a protocol was published prior to the recruitment of participants. Trial registries will also be checked to determine the integrity of reported outcome measures and statistical methods. The grey literature search may also assist with identifying publication bias.
Confidence in cumulative evidence
Strength of the evidence will be graded according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach . In the event that there is disagreement between the two reviewers, a third researcher will be consulted.
This manuscript provides a detailed account of this systematic review protocol used to advance effective components of implementation programmes in treating addiction. To our knowledge, this is the first systematic review of implementation studies for drug and alcohol settings for a range of evidence-based approaches. There has been one previous review of implementation of SUD treatment  that specifically focused on one type of service provision, integrated care. We wish to look at implementation of evidence-based practice more broadly and to utilise an implementation science framework to synthesise studies. A review of this kind is an essential component of bridging the gap between clinical research and practice for drug and alcohol services. This review will provide a summary of the implementation research that has been conducted in SUD settings to date, provide recommendations and will identify areas that are lacking research evidence. This review will provide a foundation from which to develop innovative methods for implementing evidence-based practice in drug and alcohol healthcare settings.
Given the complex, multi-level nature of implementing evidence-based practice, the results of this review will inform several stakeholders. For instance, those making decisions at the economic or policy level, organisational leadership, provider group management and clinicians being required to implement changes will be interested in the most effective means of translating evidence into practice. Researchers seeking to differentiate the efficacy of treatment outcomes from implementation outcomes will also benefit from the findings of this review, and several future research questions will be identified.
Potential limitations at the study and review level include the diversity of implementation strategies, types of clinical interventions, study conditions and outcomes found in the included articles. This will create challenges for the reviewers with regard to accurately synthesising and comparing the information. This may become particularly evident as the authors are attempting to ascertain which components of a particular intervention are the most effective means of achieving certain desired outcomes, or in situations that call for an understanding of how a specific strategy might influence the effectiveness of the implementation. On the other hand, the use of the EPOC data form, the conceptual model of Proctor et al.  and the CFIR  may assist with teasing out these inconsistencies. In addition, it is also possible that many outcomes relevant to the implementation may be underutilised or absent in the studies captured. Any protocol amendments will be documented in a protocol addendum and in the final manuscript of the systematic review.
Despite the challenges that may be encountered, this review will make a valuable contribution to the growing body of literature on the implementation of evidence-based practice in SUD settings. Importantly, it will provide a summary of the most effective means of increasing the use of evidence-based treatments amongst drug and alcohol clinicians and subsequently assist with reducing the burden of disease associated with SUDs. This information will make a significant contribution to understanding how to improve the quality of treatment offered for SUD in specialised settings and in turn to improve the outcomes of people seeking treatment for these disorders.
Availability of data and materials
Consolidated Framework for Implementation Research
Cochrane Effective Practice and Organisation of Care Review Group
Grade of Recommendations Assessment, Development and Evaluation approach
Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols
Randomised controlled trial
Substance use disorder
Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, et al. Toward evidence-based quality improvement: evidence (and its limitations) of the effectiveness of guideline dissemination and implementation startegies 1966-1998. J Gen Int Med. 2006;21:S14–20.
Timmermans S, Mauck A. The promises and pitfalls of evidence-based medicine. Health Affairs. 2005;24(1):18–28.
Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Medical Care. 2003;39:46–54.
Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Millbank Q. 2003;76:517–63.
Carroll KM. Dissemination of evidence-based practices: how far we've come, and how much further we’ve got to go. Addiction. 2012;107:1031–3.
Finney JW, Hagedorn HJ. Introduction to a special section on implementing evidence-based interventions for substance use disorders. Psychol Addict Behav. 2011;25:191–3.
Garner B. Research on the diffusion of evidence-based treatments within substance abuse treatment: a systematic review. J Subst Abuse Treat. 2009;36(4):376–99.
Saunders EC, Kim U, McGovers MP. Substance abuse treatment implementation research. J Subst Abuse Treat. 2014;44(1):1–3.
McLellan AT, Carise D, Kleber HD. Can the national addiction treatment infrastructure support the public's demand for quality care? J Subst Abuse Treat. 2003;25(1):117.
Ducharme LJ, Mello HL, Roman PM, Knudsen HK, Johnson JA. Service delivery in substance abuse treatment: reexamining “comprehensive” care. J Behav Health Serv Res. 2007;34(2):121–36.
Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q. 2001;79:281–315.
Proctor E, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, Methodological, and Training challenges. Adm Policy Ment Health. 2009;36:24–34.
Isett KR, Burnam MA, Comeman-Beattie B, Hyde PS, Morrissey JP, Magnabosco J, et al. The state policy context of implementation issues for evidence-based practices in mental health. Psychiatric Serv. 2007;58(7):914–21.
Cain M, Mittman R. Diffusion of Innovation in health Care. Institute for the Future: Oaklannd; 2002.
Glisson C, Schoenwald SK. The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatements. Mental Health Serv Res. 2005;7(4):243–59.
Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86:811–24.
Damschroder LJ, Hagedorn HJ. A guiding framework and approach for implementation research in substance use disorders treatement. Psychol Addict Behav. 2011;25(2):194–205.
Whiteford HA, Dagenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, Charlson FJ, Norman RE, Flaxman AD, Johns N, Burstein R, Murray CJL, Vos TV. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575–86.
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.
Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2008;27:379–87.
Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: a synthesis of the literature; 2005.
Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffer RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57.
Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported mental health interventions. Res Soc Work Pract. 2014;24(2):192–212.
Ducharme LJ, Chandler RK, Harris AHS. Implementing effective substance abuse treatments in general medical settings: mapping the research terrain. J Subst Abuse Treat. 2015;60:110–8.
C. TTPS, M-TGFATJCRB. Interventions integrating health and academic interventions to prevent substance use and violence: a systematic review and synthesis of process evaluations. . Syst Rev 2018;7(1):227.
Champion KE, Newton NC, Barrett EL, Teesson M. A systematic review of school-based alcohol and other drug prevention programs facilitated by computers or the internet. . Drug Alcohol Rev 2013;32(2):115-123.
Deady M, Teesson M, Kay-Lambkin FJ. Treatments for co-occurring depression and substance use in young people: a systematic review. Curr Drug Abuse Rev. 2014;7(1):3–17.
Leske S, Harris MG, Charlson FJ1, Ferrari AJ, Baxter AJ, Logan JM, Toombs M, Whiteford H.. Systematic review of interventions for Indigenous adults with mental and substance use disorders in Australia, Canada, New Zealand and the United States. Aust N Z J Psychiatry 2016;50(11):1040-1054.
Savic M, Best D, Manning V, Lubman DI. Strategies to facilitate integrated care for people with alcohol and other drug problems: a systematic review. Subst Abuse Treat Prev Policy 2017;12(1):19.
Henderson J, Milligan K, Niccols A, Thabane L, Sword W, Smith A, Rosenkranz S. Reporting of feasibility factors in publications on integrated treatment programs for women with substance abuse issues and their children: a systematic review and analysis. Health Res Policy Syst 2012;10:37.
Kelly P, Hegarty J, Barry J, Dyer KR, Horgan A. A systematic review of the relationship between staff perceptions of organizational readiness to change and the process of innovation adoption in substance misuse treatment programs. J Subst Abuse Treat. 2017;80:6–25.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. PLoS Med. 2009;3(3):e123–30.
CADTH. Grey Matters: a practical tool for searching health-related grey literature Internet. 2018 (cited 2019 Feb 22).
Williams EC, Johnson ML, Lapham GT, Caldeiro RM, Chew L, Fletcher GS, et al. Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review. Psychol Addict Behav. 2011;25(2):206–14.
HM SJAC, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JPT. ROBINS-I: a tool for assessing risk of bias in non-randomized studies of interventions. BMJ. 2016;355:i4919.
Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rogers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews: a product from the ESRC Methods Programme. In: Research IfH, editor. Lancaster. 2006.
Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4(1):38.
Savic M, Best D, Manning V, Lubman DI. Strategies to facilitate integrated care for people with alcohol and other drug problems: a systematic review. Subst Abuse Treat Prev Policy. 2017;12(1):19.
This study was supported by a Research Training Program Scholarship (EL) and a NSW Health Translational Research Fellowship (KM).
There is no dedicated funding.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Louie, E., Barrett, E.L., Baillie, A. et al. Implementation of evidence-based practice for alcohol and substance use disorders: protocol for systematic review. Syst Rev 9, 25 (2020). https://doi.org/10.1186/s13643-020-1285-0
- Substance use
- Systematic review