Protocol | Open | Open Peer Review | Published:
Processes, contexts, and rationale for disinvestment: a protocol for a critical interpretive synthesis
Systematic Reviewsvolume 3, Article number: 143 (2014)
Practical solutions are needed to support the appropriate use of available health system resources as countries are continually pressured to ‘do more with less’ in health care. Increasingly, health systems and organizations are exploring the reassessment of possibly obsolete, inefficient, or ineffective health system resources and potentially redirecting funds to those that are more effective and efficient. Such processes are often referred to as ‘disinvestment’. Our objective is to gain further understanding about: 1) whether how and under what conditions health systems decide to pursue disinvestment; 2) how health systems have chosen to undertake disinvestment; and 3) how health systems have implemented their disinvestment approach.
We will use a critical interpretive synthesis (CIS) approach, to develop a theoretical framework based on insights drawn from a range of relevant sources. We will conduct systematic searches of databases as well as purposive searches to identify literature to fill conceptual gaps that may emerge during our inductive process of synthesis and analysis. Two independent reviewers will assess search results for relevance and conceptually map included references. We will include all empirical and non-empirical articles that focus on disinvestment at a system level. We will then extract key findings from a purposive sample of articles using frameworks related to government agendas, policy development and implementation, and health system contextual factors and then synthesize and integrate the findings to develop a framework about our core areas of interest. Lastly, we will convene a stakeholder dialogue with Canadian and international policymakers and other stakeholders to solicit targeted feedback about the framework (e.g., by identifying any gaps in the literature that we may want to revisit before finalizing it) and deliberating about barriers for developing and implementing approaches to disinvestment, strategies to address these barriers and about next steps that could be taken by different constituencies.
Disinvestment is an emerging field and there is a need for evidence to inform the prioritization, development, and implementation of strategies in different contexts. Our CIS and the framework developed through it will support the actions of those involved in the prioritization, development, and implementation of disinvestment initiatives.
Systematic review registration
Practical solutions are needed to support the appropriate use of available health system resources as countries are continually pressured to ‘do more with less’ in health care. Countries are facing the challenge of how to maintain high quality care in the face of shrinking or slow-growing budgets[1, 2]. Due to limited resources, health systems and organizations are exploring the reassessment of possibly obsolete, inefficient, or ineffective health technologies (e.g., drugs, procedures, and devices) and potentially redirecting funds to more effective and efficient health technologies. Such processes are often referred to as ‘disinvestment’, which is defined as ‘…the processes of (partially or completely) withdrawing health resources from any existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain for their cost, and thus are not efficient health resource allocations’[4, 5].
Significant effort has been invested in developing well-defined criteria and processes that draw on the best available evidence to assess the safety, effectiveness, and cost-effectiveness of new and emerging health technologies. Yet similar efforts have not been directed towards disinvestment of 1) technologies that are believed to be ineffective or inefficient as compared to other technologies; 2) technologies that have never been adequately assessed and may not be beneficial but have already been incorporated into health care delivery; and 3) technologies that were initially worthy of investment but that have become obsolete. Failure to engage in disinvestment leads to inefficient allocation of limited health resources because health systems continue to fund technologies (and remunerate those who deliver them) that may provide limited or no health gain.
No widely accepted disinvestment method has been developed[7–10], but some HTA agencies have started to develop disinvestment lists focused on ineffective, harmful, or non-cost-effective health care practices[11–13] and criteria have been proposed for prioritizing health technologies for reassessment[11, 14]. Proposed methods for disinvestment include adopting a health technology reassessment approach, using Cochrane reviews as a guide[16, 17] or using frameworks such as the Guideline for Not Funding Health Technologies and the Program Budgeting and Marginal Analysis method. In Canada (where this synthesis has been funded), there have been initiatives aimed at prioritizing, developing, and implementing approaches to disinvestment both nationally (e.g., identifying obsolete technologies)[3, 6] and provincially, including efforts in Ontario to apply the Choosing Wisely Campaign from the United States and to develop a reassessment framework[19, 20]; in British Columbia where 44 disinvestment initiatives were implemented; and in Alberta where $40 million in savings was realized due to efficiency gains.
Notwithstanding these examples, disinvestment is an emerging field and there is a paucity of evidence to inform the prioritization, development, and implementation of strategies in different contexts. For example, two existing reviews[3, 9] do not focus on rationale-, context-, and process-related factors that need to be taken into consideration by policymakers and stakeholders involved in efforts to prioritize, develop, and implement disinvestment approaches. Considering the lack of theoretical development, our CIS will support the actions of those involved in the prioritization, development, and implementation of disinvestment initiatives.
The goal of this knowledge synthesis is to conduct a critical interpretive synthesis (CIS) to gain further understanding about:
whether, how, and under what conditions health systems decide to pursue disinvestment (i.e., agenda setting or prioritization);
how health systems have chosen to undertake disinvestment (i.e., policy development); and
how health systems have implemented their disinvestment approach (i.e., policy implementation).
Our overarching objective is to develop an explanatory framework about disinvestment related to each of the three principal aims to contribute to efforts for tailoring future disinvestment initiatives in different countries so that appropriate initiatives can be matched to context.
We will use the CIS approach, which (in general) has the core objective of developing a theoretical framework based on insights and interpretation drawn from a broad range of relevant sources (i.e., not just those that meet particular design or quality criteria). CIS is ideal when the question addressed is likely to draw on literature that is not well developed or focused[23, 24], which is the case with the literature related to the rationale, context, and processes for the prioritization, policy development, and implementation of disinvestment. Table 1 compares the key features of the CIS approach to those of traditional systematic reviews.
We have adopted an approach that complements the methods used by traditional systematic reviews with purposive sampling and inductive analysis, which are key features of CIS methods. In general, this will include:
conducting systematic searches of the literature (later complemented by purposive searches to fill conceptual gaps):
reviewing articles for inclusion;
purposively sampling articles based on their ability to inform key conceptual areas related to disinvestment;
extracting key findings according to frameworks related to government agendas, policy development and implementation and health system contextual factors; and
synthesizing and integrating findings to develop a framework about our core areas of interest.
As a result, we will employ both a very explicit and structured approach to search and review the indexed literature (similar to traditional systematic reviews) while also borrowing the inductive methods often associated with qualitative research designs, to ensure that our final sample of included papers and the framework derived from them is theoretically rich and relevant to our objectives.
We will search 15 databases that index research literature on a diversity of subject domains to ensure we identify articles addressing a broad spectrum of factors related to rationale, context, and processes for disinvestment prioritization, policy development, and implementation. The 15 databases include: MEDLINE (1946–present), Embase (1974–2014 April 29), Healthstar (1966 to March 2014), International Political Science Abstracts (1989–present), PsycINFO (1987 to April Week 4 2014), The Cochrane Library (Issue 4 of 12, April 2014 and Issue 1 of 4, Jan 2014), CINAHL (1982–present), Social Science Abstracts, Applied Social Sciences Index and Abstracts (1987–present), PAIS International (1914–present), ProQuest Political Science (1985–present), Sociological Abstracts (1952–present), Worldwide Political Science Abstracts (1975–present), Web of Science Core Collection (1976–present), and PubMed (for non-MEDLINE records). We have provided our detailed search strategy in Additional file1, which was developed in consultation with a library scientist at the Program for Assessment of Technology in Health (PATH) at McMaster University and then peer reviewed before being finalized. In addition, we will contact experts in the field (including our project team of researchers and knowledge users), search websites of organizations associated with HTAi and INAHTA, and search the reference lists of acquired publications to identify additional published and unpublished literature.
We will also conduct purposive searches to identify literature to fill conceptual gaps that may emerge during our inductive process of synthesis and analysis. We anticipate conducting additional purposive searches after we have conceptually mapped the relevant articles included in our sample frame in the article selection stage and again during our inductive constant comparative approach to analysis of the included papers. These searches will be developed collaboratively with the project team and will be based on our increased familiarity with the literature and collective expertise.
Selection criteria development
We will include all empirical and non-empirical articles that focus on disinvestment at a system level. This includes macro (i.e., national and sub-national) and meso (i.e., regions, health care organizations, or networks) levels but not micro level (i.e., individual clinicians or teams of clinicians).
Reference reviewing and article selection
Step 1 - Reviewing
We will review the titles and abstracts of all references captured by our search strategy. Each reference will be assessed in duplicate by two of us (MGW and MEE). The references will first be classified as ‘potentially relevant’ or ‘exclude’. We will then retrieve the full-text of all non-excluded articles and review them independently in duplicate to make a final assessment for whether they are relevant to disinvestment based on the broad areas outlined in our principal aims. The papers deemed to be relevant will be included in our sample frame of literature from which we will draw our purposive sample for the synthesis.
Step 2 - Conceptual mapping
We will conceptually map the relevant papers using a standardized form (see Additional file2) based on our data extraction categories outlined below and in more detail in Additional file3. Specifically, we will categorize the papers using a 3 × 3 matrix that cross links rationale, political, and health system contexts and process against our principle areas of interest (prioritization, policy development, and implementation). This mapping exercise will be similar to those often used in scoping reviews whereby the literature is categorized into domains and topics of interest to facilitate assessments of the state the literature (for an example, see Wilson et al.).
Each paper will be assessed using the standardized form by one reviewer, which will be independently checked by one of us (MGW or MEE) who will, in addition to the categories included in the conceptual mapping, review each paper and identify those that are the most relevant and likely to offer important conceptual insights that will help answer our research questions. We will use the following broad question to guide this assessment: does the paper provide clear insights into rationale, contexts, and/or processes related to why health systems pursue disinvestment and how they engage in and implement approaches to disinvestment?
Step 3 - Purposive sampling
We will use this mapping exercise to identify areas that are conceptually rich and areas where there appear to be conceptual gaps, which we will then use to guide our selection of a purposive sample of relevant papers. Specifically, project leads (MGW, MEE, JS, and TS) will review the results of the conceptual mapping and the assessments of which papers are likely to offer important conceptual insights and select a proposed set of papers to be included in the analysis, which we will then seek feedback about from the full project team. In addition, the project leads will develop a draft plan for further purposive searches (as outlined below) to help fill areas where there are conceptual gaps. We will then share the sample selection and the draft plan for further purposive searches with the full project team and convene a meeting to collectively review and make adjustments to the sample and search approach. Lastly, additional stages of purposive sampling may be conducted during the synthesis and integration of findings phase as conceptual gaps in the analysis are identified.
We will extract information from each article included in our synthesis by writing a 1–2 paragraph summary of key messages about each and then extracting specific findings according to frameworks of government agenda setting (as a set of variables that can help explain why health systems pursue disinvestment), policy development and implementation (as a set of variables that can help explain how health systems develop and implement approaches to disinvestment), and health systems (as a set of cross-cutting variables about rationale, context, and processes). We have drawn the government agenda setting factors from Kingdon’s widely used and empirically tested model that identifies three ‘streams’ of factors: 1) factors contributing to framing the problem as something deserving government attention; 2) availability of viable policies or solutions to address the problem; and 3) whether the prevailing ‘politics’ garner government attention. For policy development and implementation factors, we will use the ‘3I’ framework to extract and categorize relevant information. The 3I framework is derived from the political science literature which broadly relates to how political institutions (e.g., government decision-making structures and processes), interests (i.e., groups with a vested interest), and ideas (i.e., values and research-based knowledge) affect the actions of those making political decisions. For health system factors, we will use a framework of governance, financial, and delivery arrangements within health systems that we have used to categorize systematic reviews in Health Systems Evidence (http://www.healthsystemsevidence.org).
We will also extract the characteristics of each article, including the publication date, study time period (if applicable), type of paper (e.g., primary research versus non-primary research such as editorials), methods used (if applicable), country focus (if applicable), and the academic discipline (e.g., health services, systems and policy, economics, health technology assessment). We have outlined the detailed questions for the study characteristics and for each of the frameworks that we will use during data extraction in Additional file3.
Synthesizing and integrating findings
Based on qualitative research methods, we will use a constant comparative method throughout our analysis to develop an explanatory framework of disinvestment, which will allow us to ensure our framework is grounded in the data from the included papers but still drawing on the collective interdisciplinary expertise and experience of our study team. This will involve:
identifying common themes and concepts based on our summaries of and data extracted from each paper;
developing theoretical constructs based on the emerging themes and concepts;
critiquing the emerging theoretical constructs as a whole and with our full sample of literature to identify conceptual gaps in the available evidence in relation to our principal aims;
conducting additional purposive sampling of included papers and/or conducting additional purposive searches to fill conceptual gaps (if needed) until theoretical saturation is reached ; and
integrating the theoretical constructs into a ‘synthesizing argument’ about disinvestment (i.e., an explanatory framework).
Each of these stages is iterative in nature and the process will involve ongoing consultation with members of our team.
We anticipate that our biggest challenge will be to organize and synthesize findings across a diverse set of papers. Our mitigation strategy for this challenge has been to adopt a methodology that is designed to synthesize findings in complex topic areas and to assemble an interdisciplinary team that is keen to collaborate and can provide a broad range of methodological, analytical, and policy expertise. In addition, our plan for a full-day interdisciplinary workshop (described in our KT section) will engage policymakers and other stakeholders involved with supporting processes for and decisions about disinvestment. We anticipate that the workshop will be important for addressing any remaining challenges related to the analysis, framework development, and interpretation.
Our knowledge translation (KT) plan emphasizes our commitment to stakeholder engagement (i.e., integrated KT) as well as using a multi-faceted approach to mobilizing the findings to ensure knowledge users can efficiently find and use relevant and high-quality research evidence when needed.
As part of our commitment to integrated KT, we have assembled an interdisciplinary team of researchers and knowledge users who will collaboratively work together to ensure the project is timely and relevant, as well as progressing as intended and collectively address any methodological, analytical, and practical issues that arise. The project’s research leads (MGW and ME) will engage the full team through teleconferences and in-person at meetings where most of us will already be present (e.g., the Canadian Association of Health Services and Policy Research Annual Conference) to provide input on the inclusion criteria, results of the conceptual mapping, purposive sampling and searching, synthesis and analysis, framework, final paper, and format and content for the stakeholder dialogue (outlined in more detail below).
By drawing on the diverse expertise of our team members, we will be able to engage in interdisciplinary collective problem-solving that will ensure the review provides relevant and user-friendly results that can be used to inform policy and practice. Through engagement and collaboration with our interdisciplinary team of knowledge users and researchers in the development of this proposal, we have already begun integrated KT by ensuring the types of information sought, analyzed, and disseminated will be relevant to informing disinvestment processes.
The centerpiece of our end-of-grant KT activities will be to convene a 1-day stakeholder dialogue at the McMaster Health Forum, which will be facilitated by one of us (Lavis). We will engage approximately 18–22 participants in the stakeholder dialogue which, based on our experience with running more than 30 stakeholder dialogues, is optimal for ensuring the deliberations include a breadth of perspective but also allow for meaningful contributions from all participants. We will aim to engage Canadian and international policymakers and other stakeholders involved with supporting processes for and decisions about disinvestment. In addition, we will engage researchers from a range of disciplines (e.g., public policy, health economics, health technology assessment, knowledge translation) involved with disinvestment. Prior to the dialogue, we will produce a draft report of the synthesis, which we will distribute to participants approximately 2 weeks before convening. During the dialogue we will:
solicit feedback and deliberate about the key elements of the framework derived from the synthesis;
identify and deliberate about barriers for disinvestment processes and strategies to realistically address these barriers; and
identify and deliberate about next steps that could be taken by different constituencies to move forward with disinvestment.
Following the dialogue, we will review the suggestions from participants and make changes to the framework only after revisiting the literature to ensure its elements remain grounded in the available evidence. We will disseminate the final report through the networks of each of our team members and produce and submit at least one manuscript to a peer-reviewed journal (open access).
Health technology assessment
Critical interpretive synthesis
Health Technology Assessment International
International Network of Agencies for Health Technology Assessment
Rich G: Shrinking budgets, improving care: simplify processes and engage with patients. BMJ. 2010, 340: 1251. 10.1136/bmj.c1251.
Zalmanovitch Y, Vashdi DR: Shrinking budgets, improving care: trade-offs are unavoidable. BMJ. 2010, 340: 1259. 10.1136/bmj.c1259.
Leggett L, Noseworthy TW, Zarrabi M, Lorenzetti D, Sutherland LR, Clement FM: Health technology reassessment of non-drug technologies: current practices. Int J Technol Assess Health Care. 2012, 28: 220-227. 10.1017/S0266462312000438.
Elshaug AG, Hiller JE, Tunis SR, Moss JR: Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust N Z Health Policy. 2007, 4: 23. 10.1186/1743-8462-4-23.
Elshaug AG, Hiller JE, Moss JR: Exploring policy-makers' perspectives on disinvestment from ineffective healthcare practices. Int J Technol Assess Health Care. 2008, 24: 1-9.
Elshaug AG, Watt AM, Moss JR, Hiller JE: Policy Perspectives on the Obsolescence of Health Technologies in Canada: A Discussion Paper. 2009, Ottawa, Canada: Canadian Agency for Drugs and Technologies in Health
Mitton C, Patten S, Donaldson C: Listening to the decision makers: sustainability of PBMA in Alberta. Appl Health Econ Health Policy. 2004, 3: 143-151. 10.2165/00148365-200403030-00005.
Ibargoyen-Roteta N, Gutierrez-Ibarluzea I, Asua J: Guiding the process of health technology disinvestment. Health Policy. 2010, 98: 218-226. 10.1016/j.healthpol.2010.06.018.
Polisena J, Clifford T, Elshaug AG, Mitton C, Russell E, Skidmore B: Case studies that illustrate disinvestment and resource allocation decision-making processes in health care: a systematic review. Int J Technol Assess Health Care. 2013, 29: 174-184. 10.1017/S0266462313000068.
Donaldson C, Bate A, Mitton C, Dionne F, Ruta D: Rational disinvestment. QJM. 2010, 103: 801-807. 10.1093/qjmed/hcq086.
Elshaug AG, Moss JR, Littlejohns P, Karnon J, Merlin TL, Hiller JE: Identifying existing health care services that do not provide value for money. Med J Aust. 2009, 109: 269-273.
UK Audit Commission: Reducing Spending on Low Clinical Value Treatments. 2011, London, United Kingdom: Audit Commission
Cook S: Experts' guide to saving money in health. BMJ. 2010, 340: c1281. 10.1136/bmj.c1281.
Pearson S, Littlejohns P: Reallocation resources: how should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the Nation Health Service?. J Health Serv Res Policy. 2007, 12: 160-165. 10.1258/135581907781542987.
Henshall C, Schuller T, Mardhani-Bayne L: Using health technology assessment to support optimal use of technologies in current practice: the challenge of "disinvestment". Int J Technol Assess Health Care. 2012, 28: 203-210. 10.1017/S0266462312000372.
Garner S, Docherty M, Somner J, Sharma T, Choudhury M, Clarke M, Littlejohns P: Reducing ineffective practice: challenges in identifying low-value health care using Cochrane systematic reviews. J Health Serv Res Policy. 2013, 18: 6-12. 10.1258/jhsrp.2012.012044.
Hughes DA, Ferner RE: New drugs for old: disinvestment and NICE. BMJ. 2010, 340: c572. 10.1136/bmj.c572.
Rich P: Physicians taking lead on appropriateness of care. Can Med Assoc. 2013, 21: 8-
Paprica A: Evidence-Informed Changes to Funded Health Services and Products. 2013, Vancouver, Canada: Canadian Association of Health Services and Policy Research Annual Conference
Planning Unit (Health System Planning Research & Analysis Branch of the Health System Strategy & Policy Division - Ontario Ministry of Health and Long-Term Care): A Preliminary Literature Review on Models for Disinvestment in Health Care Services. 2011, Toronto, Canada: Ontario Ministry of Health and Long-Term Care
Mitton C, Dionne F, Damji R, Campbell D, Bryan S: Difficult decisions in times of constraint: criteria based resource allocation in the Vancouver Coastal Health Authority. BMC Health Serv Res. 2011, 11: 169. 10.1186/1472-6963-11-169.
Mitton C, Patten S, Waldner H, Donaldson C: Priority setting in health authorities: a novel approach to a historical activity. Soc Sci Med. 2003, 57: 1653-1663. 10.1016/S0277-9536(02)00549-X.
Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton A: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006, 6: 35. 10.1186/1471-2288-6-35.
Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N: Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc Sci Med. 2012, 75: 1938-1945. 10.1016/j.socscimed.2012.06.016.
Wilson MG, Lavis JN, Guta A: Community-based organizations in the health sector: a scoping review. Health Res Policy Syst. 2012, 10: 36. 10.1186/1478-4505-10-36.
Kingdon JW: Agendas, Alternatives and Public Policies (Longman Classics Edition) Second Edition. 2010, New York, NY: Addison-Wesley Educational Publishers Inc.
Lazar H, Forest P-G, Lavis JN, Church J: Paradigm Freeze: Why It Is So Hard to Reform Health Care in Canada?. 2013, Kingston and Montreal, Canada: McGill-Queen's University Press
Creswell JW: Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 2007, Thousand Oaks, CA: Sage
This project has been funded by the Canadian Institutes of Health Research through a Knowledge Synthesis Grant (grant number 315602).
The authors declare that they have no competing interests.
MGW and ME led the design of the protocol and drafted the manuscript. All authors (MGW, MEE, JNL, JMG, KAM, JS, TS, SG, RG, RG, JP, and KS) provided input into the original grant proposal to the Canadian Institutes of Health Research and have read and approved this manuscript.
Electronic supplementary material
About this article
- Critical interpretive synthesis
- Health technologies
- Health system context
- Policy development
- Policy implementation