- Open Access
Theoretical approaches to process evaluations of complex interventions in health care: a systematic scoping review protocol
Systematic Reviews volume 10, Article number: 268 (2021)
Complex interventions in health care are characterized by multiple interacting components as well as by numerous nonlinear interactions with the social systems within which they are being implemented. The process of developing, evaluating and implementing complex interventions is therefore challenging. Established guidance such as the MRC (Medical Research Council) framework for developing and evaluating complex interventions refers to process evaluations as an integral part of the development of complex evidence-based interventions. Even though the need for process evaluations is recognized, the realization of such approaches is challenging because methodological instruction is sparse, and the phenomenon of interest is complex. A number of theoretical approaches indicating how to conduct process evaluations of complex interventions in health care exist, but a systematic and comprehensive overview of these is missing. Thus, the objective of the systematic scoping review described herein is to provide an overview and analysis of theoretical approaches suitable for the planning and conducting of process evaluations.
The design and conduct of this review will follow the procedures of a systematic scoping review. The search strategy will be developed following the BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) template which has been conceptualized for structured reviews of theory. The systematic search of the MEDLINE (via PubMed), CINAHL (via EBSCO) and PsycInfo (via EBSCO) electronic databases will be complemented by “hand searching” techniques. Study selection, data extraction, and data analysis will be performed by tandems of two researchers independently of each other. Divergent decisions and judgements between the two researchers will be discussed by the whole review team.
The findings from this scoping review will provide an overview and comparison of theoretical approaches suitable for process evaluations of complex interventions in health care. The review results will support researchers in choosing the theoretical approach that best fits the respective focus of their process evaluation study.
Systematic review registration
This study has been registered with PROSPERO (International Prospective Register of Systematic Reviews) under registration number CRD42020211732.
Efforts to improve healthcare delivery and outcomes often require complex interventions. Complex interventions were originally defined as interventions with multiple interacting components [1, 2], indicating that complexity is particularly linked to the intervention itself. Ongoing debates about which characteristics constitute complex interventions [3,4,5] have complemented this original definition with the awareness that complexity is given “…primarily due to the social systems within which these actions occur, the contextually contingent nature of impacts, and the agency of the groups and individuals whose behaviours they aim to influence” .
Due to this changing definition, it is increasingly recommended to integrate process evaluations in the development and evaluation of complex evidence-based interventions to assess implementation aspects and to gain a better understanding of causal mechanisms and contextual factors associated with variation in outcomes [1, 6,7,8]. Process evaluations can especially help to distinguish between those interventions that are inherently faulty (failure of intervention) and those interventions that are delivered inaccurately (implementation failure) . Thus, they can prevent a so-called type III error, which refers to the question of whether the intervention actually has been delivered or if there is in fact something measured that does not exist .
The MRC (Medical Research Council) framework for developing and evaluating complex interventions provides a well-established guide for the development of complex interventions in healthcare [1, 2]. The revised version from 2008 [1, 11] for the first time took into account that process evaluations are highly recommendable to understand processes and to obtain explanations if interventions fail or have unanticipated consequences . Nevertheless, the 2008 framework provided only sparse information on how to integrate and conduct process evaluations. Process evaluation is described as being “nested within a trial” that in the first place focuses on outcome evaluation .
In recent years, methodological aspects of process evaluations have been increasingly discussed; thus, the current MRC provides comprehensive guidance for process evaluations of complex interventions [6, 12]. This guidance refers to  implementation,  mechanisms of impact, and  context as key aspects to be examined [6, 12]. Furthermore, it “provides researchers, practitioners, funders, journal editors and policy-makers with guidance in planning, designing, conducting and appraising process evaluations of complex interventions” ( , p. 9}. It comprises process evaluation theory as well as practical information .
Regardless of the scientific discourse in the context of the MRC framework, the concept of process evaluation is not new to the field of health care research; it had already emerged by the 1960s . Suchmann , as one of the first to emphasize the need to evaluate health interventions/programmes in relation to their practical setting, identified  the attributes of the programme,  the population exposed to the programme,  the situational context in which the programme takes place, and  the different kinds of effects produced by the programme as obligatory domains for “making sense” of the evaluative findings. Although these basic assumptions of Suchmann  remain relevant for the context of process evaluation, a wide range of developments have produced a diverse research field with many underlying concepts.
Commonly quoted concepts are recruitment (of cluster/of individuals), maintenance, context, resources, implementation, reach (of individuals), barriers, exposure, initial use, continued use, contamination, dose delivered, dose received, fidelity (degree to which an intervention is implemented in accordance with the original protocol), delivery (to clusters/to individuals), response (of clusters/of individuals), effectiveness, unintended consequences, theory, efficacy, and adoption [13, 15,16,17]. Many of these concepts are also closely linked to the broad field of implementation research . Thus, the underlying theoretical allocation and a clear distinction of these concepts is challenging.
The existing theoretical approaches relevant for process evaluations set varying priorities and organize their concepts in different ways (e.g. ). In addition, concepts termed identically in different theoretical approaches are not always defined in the same way, and they are not always selective between the different theoretical approaches. The MRC guidance for process evaluation of complex interventions  presents “an overview of a number of core frameworks” (, p. 18) relevant to process evaluation, but the authors explicitly state that they “make no claims to exhaustiveness” (, p. 18). While the theoretical approaches considered in the MRC guidance for process evaluation include many of the concepts mentioned above, the guidance does not provide a thorough analysis or comparison of the concepts the theoretical approaches refer to. Therefore, it remains unclear why the proposed theoretical approaches were selected and if there may be other theoretical approaches of relevance. This complicates the suggestion of the MRC guidance to combine concepts from different theoretical approaches for the development and conduct of process evaluations . However, given the variety of theoretical approaches that are of relevance for process evaluations, it remains challenging to select and combine theoretical approaches or single concepts that fit the requirements and aims of a specific process evaluation approach.
Therefore, we will perform a comprehensive and systematic scoping review and analysis of the theoretical approaches suitable for process evaluations of complex interventions in health care. For this purpose, we understand theoretical approaches as interrelated propositions that intend to explain and predict general sets of phenomena . Accordingly theoretical approaches help to structure our knowledge, understanding, explanation, and prediction of the world and they allow to generate new research hypotheses as well [20,21,22]. In the context of process evaluation, a theoretical foundation is essential to better understand and systematically investigate the black box of implementing complex interventions in complex social settings . At the same time, theoretical approaches are not uniform but might be e.g. implicit or explicit, descriptive, explanatory or predictive  or address high, middle or low abstraction levels . Moreover, a range of heterogenous terms is being used in order to consider theoretical aspects in research projects and real life, e.g. theoretical/conceptual model, (theoretical/conceptual) framework, theoretical concepts [21, 22] These terms are often not clearly distinguished and used in an interchangeable way [20, 21]. Based on the above-noted explanations, we will therefore consider any kind of theoretical approach suitable for process evaluation in order to achieve a comprehensive overview.
The review results will further support researchers in considering, selecting and combining the available theoretical approaches (or just single concepts) and thus promote tailored and theory-informed process evaluation approaches.
The following study objectives were defined:
To identify theoretical approaches suitable for process evaluations of complex interventions in health care;
To describe, analyse and compare the identified theoretical approaches and single concepts that these theoretical approaches comprise of.
We will perform a review of theoretical approaches. This kind of review differs substantially from a review of empirical data. Methodological issues related to such approaches have been rarely discussed in the literature , and they are not taken into account in overviews of review types [24, 25]. We will therefore consider the methodological issues related to this review carefully and ensure its transparency and comprehensiveness. The development and reporting of the review will follow the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for Scoping Reviews (PRISMA-ScR). The PRISMA extension, PRISMA-P  (see Additional file 1) has been used for the preparation of this review protocol.
Important protocol amendments will be reported with the review results.
We will use the BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) template only to develop our search strategy. This template has been recently conceived to inform the development of procedures for the structured review and identification of theory . It refers to the following categories:
Be – Behaviour of interest ➔ ways in which the population interacts with the health context;
H – Health context of interest;
E – Exclusions ➔ to exclude, e.g. non-theoretical/technical models/frameworks; and
MoTh – Models or Theories ➔ generic terms for models/frameworks, as well as named frameworks/models, if appropriate.
Table 1 gives an overview of the initial search terms which will be refined during the search process.
Based on the final search strategy, systematic electronic searches of the following databases relevant to health care research will be performed: MEDLINE (via PubMed), CINAHL and PsycInfo (both via EBSCO).
Furthermore, iterative, pragmatic approaches, in addition to keyword searches alone, are recommended as an effective method for qualitative reviews [28, 29] and seem to suit the needs of reviews of theory as well.
Thus, following the iterative approach, the systematic search will be complemented by “hand-searching” techniques such as expert consultations, trawls through specific journals, and backward as well as forward citation tracking, which is frequently recommended as being especially valuable for reviews of theory .
The development and application of the search strategy will be continuously discussed by the project team, which comprises all the authors of this study protocol.
The search results will be screened according to defined inclusion criteria and thereby resulting exclusion criteria. An initial set of inclusion criteria is outlined in Table 2.
For the purpose of our review, precise criteria definitions based on established frameworks, such as PICO  or its equivalent for reviews of qualitative studies, SPIDER , developed in the context of health research are not suitable, since relevant theoretical approaches possibly do not originate from the specific field of process evaluation or from health care research . Further, one of the key challenges in this review will be to define which criteria constitute a “process evaluation” since a consistent terminology and definition is missing. Thus, from the beginning of the review process, we have discussed within the review team which criteria could define “process evaluation”. We started out with a literature-based working definition (additional file 2), which we will continuously develop throughout the review process in iterative loops. The inclusion and exclusion criteria will be adapted accordingly. Following the iterative approach described above, amendments and specifications of the initial set of criteria will be discussed and decided throughout the review process. All decisions and amendments are being documented in a study manual, which will be provided in the final review paper to ensure transparency.
Data management will be organized using the EPPI-Reviewer4 software , which has been developed for managing and analysing data in all types of systematic reviews, such as meta-analysis, framework synthesis and thematic synthesis . Features of the software support reference management, study classification, data extraction, and synthesis. The software is recommended by the Cochrane Collaboration .
Duplicates will be removed before starting the selection process. This step, as well as the other steps of the selection process, will be supported by the EPPI-Reviewer4 software.
Based on the final set of inclusion and exclusion criteria, all titles and abstracts will be screened independently by each researcher paired up in teams of two (tandems) (all authors of this study protocol). Full texts will be obtained and assessed where the inclusion criteria seem to be met but where ratings for the title and abstract screenings differ between reviewers and no consensus can be achieved. Cases that remain unclear after a full text assessment will be discussed within the tandems and agreed upon between both reviewers. In case of non-agreement within one tandem, the publication will be discussed within the project team for a final decision.
Inclusion and exclusion of publications reporting studies that are not labelled as “process evaluations”, but that actually meet the criteria of a process evaluation and that apply a theoretical approach will be thoroughly discussed within the project team. This discussion will be based on a working definition of “process evaluation” that will be developed consecutively throughout the review process.
Publications describing the same framework will be consolidated before data extraction since the framework/model is the unit of interest and not the single report . Different versions and developments of frameworks/models will be distinguished.
Data extraction strategy
The EPPI-Reviewer4 software  will be used for data extraction. This software allows flexible coding schemes for classifying studies with regard to eligibility criteria and for capturing detailed information from the included references. We will extract data based on the following coding categories: author, year, name of theoretical approaches, scope, process of development, historical roots/theoretical basis, dimensions and constructs included in the theoretical approaches, and guidance on the utilization of the theoretical approaches.
The data extraction will again be performed independently by each researcher paired up in tandems. The EPPI-Reviewer4 software  allows the classification of theoretical approaches by multiple users, the comparison of results, and the documentation of the process of agreeing upon final decisions.
Data synthesis and presentation
The identified theoretical approaches will be presented in a table format. These tables will provide an overview and content-wise sorting of the identified theoretical approaches. The tables will also present the data extracted with regard to each of the aforementioned categories (see data extraction). A main focus will be the display and allocation of the single concepts based on the identified theoretical approaches. They will also be presented in a table. However, since concepts used in the context of process evaluation often overlap and are inconsistently defined, we will also consider divergent definitions and meanings.
The review results will be published in a peer-reviewed journal and presented at relevant conferences.
The relevance of process evaluations is being increasingly discussed in the scientific discourse on the development and evaluation of complex interventions and is now widely recognized [1, 6, 8, 15, 16]. Although an increasing number of process evaluations are being conducted, these approaches are often not well founded on theory since there is little guidance available on how to derive concrete process evaluation procedures from theoretical allocation . Frequently, this leads to arbitrary methodological procedures and subsequently to low-quality results. Furthermore, the phenomenon investigated is very demanding, since the focus is not only on the intervention to be developed but also on the implementation, the mechanisms of impact, and above all on the context [1, 6], which makes it necessary to consider complex social systems . A theoretical foundation is therefore essential for the planning and realization of process evaluations. The results of this review will provide a systematic and critical overview and comparison of existing theories for process evaluations and thus facilitate the targeted selection of suitable theoretical approaches for the respective process evaluation approach. The review will build upon and further contribute to the scientific discourse on process evaluations that has been inspired by the MRC and its guidance on process evaluation.
From a methodological point of view, there will be some challenges related to this review since little literature exists with regard to literature reviews aiming at identifying theory. However, we expect that we will achieve high-quality results with the methodological approach described in this protocol, which is based on the guidelines for conducting systematic scoping reviews and the demonstrated iterative, inductive and reflexive approaches of qualitative research.
Availability of data and materials
Behaviour of interest
Models or Theories
Medical Research Council
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
PRISMA for Protocols
PRISMA Extension for Scoping Reviews
Prospective Register of Systematic Reviews
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.
Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, et al. From complex social interventions to interventions in complex social systems: Future directions and unresolved questions for intervention development and evaluation. Evaluation. 2019;25(1):23–45.
Hawe P, Shiell A, Riley T. Theorising Interventions as Events in Systems. American Journal of Community Psychology. 2009;43(3-4):267–76.
Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ. 2008;336(7656):1281–3.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350.
Vernooij-Dassen M, Moniz-Cook E. Raising the standard of applied dementia care research: addressing the implementation error. Aging & Mental Health. 2014;18(7):809–14.
Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.
Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56(2):119–27.
Dobson D, Cook TJ. Avoiding type III error in program evaluation: Results from a field experiment. Eval Program Plan. 1980;3(4):269–76.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: new guidance. London: Medical Research Council; 2008.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions - UK Medical Research Council (MRC) guidance. London: MRC Population Health Sciences Research Network; 2014 05/06/2020.
Linnan L, Steckler A. Process Evaluation for Public Health Interventions and Research:An Overview. In: Steckler A, Linnan L, editors. Process Evaluation for Public Health Interventions and Research. New York: Wiley; 2002.
Suchmann E. Evaluative Research: Principles and Practice in Public Service and Social Action Programms: Russell Sage Foundation; 1967.
Baranowski T, Stables G. Process Evaluations of the 5-a-Day Projects. Health Educ Behavior. 2000;27(2):157–66.
Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15.
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.
Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Practice. 2008;14(2):117–23.
Leontjevas R, Gerritsen DL, Koopmans RT, Smalbrugge M, Vernooij-Dassen MJ. Process evaluation to explore internal and external validity of the "Act in Case of Depression" care program in nursing homes. J Am Med Dir Assoc. 2012;13(5):488 e1-8.
Carpiano RM, Daley DM. A guide and glossary on post-positivist theory building for population health. J Epidemiol Community Health. 2006;60(7):564–70.
Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science. 2015;10(1):53.
Rycroft-Malone J, Bucknall T. Theory, frameworks, and models - Laying down the groundwork. In: Rycroft-Malone J, Bucknall T, editors. Models and Frameworks for Implementing Evidence-Based Practice. 1 ed. Wiley; 2010.
Campbell M, Egan M, Lorenc T, Bond L, Popham F, Fenton C, et al. Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health. Systematic Reviews. 2014;3(1):114.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information Libraries J. 2009;26(2):91–108.
Booth A, Sutton A, Papaioannou D. Systematic Approaches to a Successful Literature Review. 2nd ed. London: SAGE Publications Ltd; 2016.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews. 2015;4:1.
Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Information Libraries J. 2015;32(3):220–35.
Cooke A, Smith D, Booth A. Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis. Qualitative Health Research. 2012;22(10):1435–43.
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):45.
Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S. Chapter 2: Determining the scope of the review and the questions it will address. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. Chichester: Wiley; 2019. p. 13–32.
Thomas J, Graziosi S, Brunton J, Ghouze Z, O'Driscoll P, Bond M. EPPI-Reviewer: advanced software for systematic reviews, maps and evidence synthesis. EPPI-Centre Software. London: UCL Social Research Institute; 2020.
EPPI-Reviewer 4 - Software for research synthesis: User Manual. London: Social Science Research Unit, UCL Institute of Education; 2018. Available from: http://eppi.ioe.ac.uk/CMS/Portals/35/ER4_8_0%20user%20manual.pdf.
Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M-I, et al. Chapter 4: Searching for and selecting studies. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. Chichester: Wiley; 2019. p. 67–108.
No third-party funding has been or will be received for this study; the study will be funded by the German Center for Neurodegenerative Diseases e.V. (DZNE) site Witten. Open Access funding enabled and organized by Projekt DEAL.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1.
PRISMA-P 2015 Checklist.
Additional file 2.
Initial working definition: Process evaluation.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Quasdorf, T., Clack, L., Laporte Uribe, F. et al. Theoretical approaches to process evaluations of complex interventions in health care: a systematic scoping review protocol. Syst Rev 10, 268 (2021). https://doi.org/10.1186/s13643-021-01825-z
- Process evaluation
- Complex intervention
- Health care
- Systematic scoping review