- Open Access
Quality and performance measures of strain on intensive care capacity: a protocol for a systematic review
Systematic Reviews volume 4, Article number: 158 (2015)
The matching of critical care service supply with demand is fundamental for the efficient delivery of advanced life support to patients in urgent need. Mismatch in this supply/demand relationship contributes to “intensive care unit (ICU) capacity strain,” defined as a time-varying disruption in the ability of an ICU to provide well-timed and high-quality intensive care support to any and all patients who are or may become critically ill. ICU capacity strain leads to suboptimal quality of care and may directly contribute to heightened risk of adverse events, premature discharges, unplanned readmissions, and avoidable death. Unrelenting strain on ICU capacity contributes to inefficient health resource utilization and may negatively impact the satisfaction of patients, their families, and frontline providers. It is unknown how to optimally quantify the instantaneous and temporal “stress” an ICU experiences due to capacity strain.
We will perform a systematic review to identify, appraise, and evaluate quality and performance measures of strain on ICU capacity and their association with relevant patient-centered, ICU-level, and health system-level outcomes. Electronic databases (i.e., MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Web of Science, and the Agency of Healthcare Research and Quality (AHRQ)—National Quality Measures Clearinghouse (NQMC)) will be searched for original studies of measures of ICU capacity strain. Selected gray literature sources will be searched. Search themes will focus on intensive care, quality, operations management, and capacity. Analysis will be primarily narrative. Each identified measure will be defined, characterized, and evaluated using the criteria proposed by the US Strategic Framework Board for a National Quality Measurement and Reporting System (i.e., importance, scientific acceptability, usability, feasibility).
Our systematic review will comprehensively identify, define, and evaluate quality and performance measures of ICU capacity strain. This is a necessary step towards understanding the impact of capacity strain on quality and performance in intensive care and to develop innovative interventions aimed to improve efficiency, avoid waste, and better anticipate impending capacity shortfalls.
Systematic review registration
The growing and changing demographic profile of the population is projected to contribute to sustained increases in the demand for health services for the foreseeable future. These trends in demographics are accompanied by rising prevalence of persons who have chronic illnesses such as obesity, diabetes mellitus, hypertension, coronary artery disease, and chronic kidney disease . An older population that has greater burdens of chronic disease is more at risk for critical illness . Indeed, patients now receive more complex therapies (e.g., cancer chemotherapy), interventions, or procedures (e.g., complex surgical procedures) for chronic disease. Our ability to effectively treat previously fatal illnesses has advanced and with this so has the demand for critical care services .
The matching of critical care service supply with demand is a fundamental requisite for the timely delivery of advanced life support technologies to patients in urgent need. Mismatch in this supply/demand relationship creates intensive care unit (ICU) capacity strain. ICU capacity strain is defined as “a temporally varying influence on a given ICU’s ability to provide high-quality care for every patient who is or could become a patient in that ICU on a given day.”. While this definition has face validity, there is uncertainty on how to optimally capture strain on ICU capacity as a quality or performance measure. Indeed, every ICU knows what strain on capacity is when they are experiencing it; however, it has proven challenging to quantify using valid and reliable measures. In preparation for this proposal, we found no robust or validated measures that enable quantification of the immediate or temporal “stress” an ICU experiences due to capacity strain.
What is clear is that ICU capacity strain leads to suboptimal quality of care and directly contributes to higher risk of adverse events, premature discharges, unplanned readmissions, and death [5–11]. When ICU beds are unavailable, care processes may be negatively impacted. The rate of ICU admission refusal increases, and there is greater likelihood of changes to patients’ goals of care towards “do not resuscitate” or “palliation” [12, 13]. This “rationing” is non-transparent, is non-random, and negatively influences quality of care. ICU capacity strain can impede the timely access to critical care services. Patients who have delayed transfer to ICU have significant increases in potentially avoidable morbidity and mortality [12, 14]. A delay in transfer to ICU of ≥6 h from the emergency department has been shown to increase mortality and prolong hospitalization . ICU capacity strain also negatively impacts ICU throughput by increasing the likelihood of premature discharge from ICU [6, 15], increasing the risk of readmission (e.g. “bounce backs”) , unnecessary “after-hours” discharges [5, 10, 15], and adverse events . Sustained strain on ICU capacity contributes to the inefficient utilization of health resources and may negatively impact the satisfaction of patients, their families, and frontline providers.
We believe these observations strongly reinforce the importance of identifying valid and robust measures of ICU capacity strain. To address this issue, we propose to perform a systematic review to identify and characterize all available measures of ICU capacity strain. This is the first step in a broader program of research dedicated to the evidence-informed integration of measures of ICU capacity strain into routine practice with the aim to alleviate sustained strain, improve efficiency, and thereby ensure sustainable and consistent high-quality critical care service delivery.
The aim of our review is to systematically evaluate the literature in order to identify proposed measures of ICU capacity strain. The specific objectives are:
Create an inventory of quality and performance measures associated with strain on ICU capacity
Categorize the measures of ICU capacity strain at the patient level, ICU level, health system level, and human capital level (i.e., human resource related) and across the dimensions of health care quality (i.e., efficient, effective, timely, safe, patient-centered, equity) and the Donabedian framework (i.e., structure, process, outcome).(Tables 1 and 2)
Describe the relevant measurable outcomes associated with ICU capacity strain at the patient-level (i.e., adverse events, mortality, family satisfaction), ICU level (i.e., unplanned readmission, after-hours discharge, avoidable bed days), and health system level (i.e., ICU lengths of stay, hospital length of stay, rehospitalization)
Describe the economic and human resource implications associated with ICU capacity strain (i.e., staffing models including surge capacity, absenteeism, burnout, satisfaction)
A systematic review will be performed to identify and evaluate quality and performance measures of ICU capacity strain using the guidelines from The Cochrane Collaboration and Center for Reviews and Dissemination and described according to the PRISMA-P guideline (available at: http://www.systematicreviewsjournal.com/content/4/1/1)  (see Additional File 1).
This systematic review has been registered with PROSPERO (registration # CRD42015017931).
Criteria for considering studies for this review
Studies will be included if they mention all of the following broad themes:
Intensive or critical care: this theme refers to patients (adults, children and neonates) who are critically ill or at risk for an acute clinical deterioration that may necessitate support in an ICU setting.
Quality or performance measure: this theme refers to any measurable variable intended to evaluate the structure, process, or outcome of care provided to patients at risk of being admitted or admitted to ICU.
Capacity strain: this theme refers to the evaluation of any measurable variable intended to evaluate the untoward impact at the patient-level, ICU level, health system level, or human capacity level of the stress on ICU capacity.
Study design or article source: this theme is intended to include all studies reporting original primary or secondary data encompassing the above mentioned themes, as well as selected gray literature, including administrative reports to governments or health care agencies.
Studies will be excluded that do not fulfill all of the above criteria, published in a language other than English, and published prior to the year 1990.
Search methods for identification of studies
The search strategy will be developed in consultation with an expert librarian/information specialist at the Alberta Research Centre for Health Evidence (ARCHE) at the University of Alberta and will undergo subsequent peer review by a second specialized librarian using the Peer Review of Electronic Search Strategies checklist . The information specialist will search electronic databases: Ovid MEDLINE, Ovid EMBASE, CINAHL via EBSCO host, the Cochrane Library including the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials (CENTRAL), and the Web of Science (to also capture operations management literature). Database search results will be from inception to current and restricted to the English language (see Additional File 2).
A combination of the following search themes with the Boolean term AND will be used:
intensive care, critical care, critical illness, multi-organ dysfunction, multi-organ failure
quality indicator, quality measure, performance measure, quality improvement, quality assurance, quality control, performance improvement, best practice, processes of care, complications, adverse event, medication error, safety, effectiveness, efficiency, appropriateness, outcomes assessment, outcome, audit
strain, capacity, occupancy, census, resource, operations management, acuity, rationing, queuing, avoidable, unplanned, readmission, nighttime discharge, absenteeism, burnout, workload, discrete event simulation. Appropriate truncation and wildcards will be used in the search to account for plurals and/or variations in the spelling of search terms. Bibliographic records will be exported to an EndNote X7 (Thomson Reuters, Philadelphia, Pennsylvania) database for screening. Additional sources will be included in the search strategy. The cited and citing references of selected key studies will be searched for relevant articles. Gray literature sources will be searched for clinical practice guidelines, operations management reports, and selected conference proceedings that describe measures of ICU capacity. We will identify and search the websites of relevant organizations (i.e., American Thoracic Society, Society of Critical Care Medicine, Canadian Critical Care Society, European Society of Intensive Care Medicine, Australia and New Zealand Intensive Care Society). We will search the Agency of Healthcare Research and Quality National Quality Measures Clearinghouse (www.qualitymeasures.ahrq.gov) for “ICU capacity”- and “strain”-related quality and performance measures
As part of this process, we will also engage an inter-disciplinary committee of 10–12 stakeholders including knowledge users, decision-makers, and researchers (i.e., policy makers, managers, physicians, nurses, operations management, and quality researchers). This committee will be surveyed for additional quality or performance measures that may be relevant to ICU capacity strain. In addition, this committee will perform a quality assessment of the evidence-base of each identified measure (detailed below).
Potentially eligible articles will be initially identified by having two authors independently review the titles and abstracts of all articles identified by the search. The full text of all articles deemed potentially relevant will be retrieved, and two authors will independently review the full text for inclusion using pre-defined eligibility criteria. Any disagreements that arise will be resolved through discussion or referral to a third party.
Data will be abstracted from relevant studies using a standardized electronic data collection form. This abstraction will be performed in duplicate by the same two authors. Any disagreements that arise will be resolved through discussion or referral to a third party. The authors of the retrieved studies and/or documents will be contacted for further information as necessary. Methodological quality will be rated using the Newcastle-Ottawa Quality Assessment Scale (NOS) for observational studies and a modified version of BOAS for before-and-after studies . Qualitative studies will be evaluated using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist with a four-point scale .
Our review proposes three steps to characterize measures of ICU capacity strain: identification, definition, and evaluation.
Measures of capacity strain will be identified through triangulation of all identified articles and related documents or reports and through the survey of inter-disciplinary stakeholder committee.
We will create a summary operational definition and assess its measurement properties (reliability, test-retest), validity (face, construct, criterion)). We will further characterize how each measure is calculated (i.e., simple or complex computation), whether the measure is normalized (i.e., for benchmarking purposes, case-mix differences) and whether the measure is averaged (i.e., capable of continuous assessment of performance over a pre-specified period of observation).
Members of the inter-disciplinary stakeholder committee will independently evaluate each measure using the four criteria proposed by the US Strategic Framework Board for a National Quality Measurement and Reporting System according to (i) importance, (ii) scientific soundness, (iii) usability, and (iv) feasibility . Importance will be characterized by whether a measure has evidence of an association with outcomes at the patient, ICU, health system, and/or human capital levels. Scientific soundness will assess how plausible each identified measure’s association with respective outcomes. Usability and feasibility will characterize the logistics and process of implementation of each measure into clinical practice. Each measure will also be evaluated for its potential operational characteristics, for its potential to be integrated into electronic medical records and on its affordability, if applicable.
The primary analysis will be largely narrative. Each measure will be categorized first according to the structure, process, and outcome framework and then by agreed upon domains of evaluation [20, 21]. A comprehensive inventory of measures will be developed and summarized as counts and proportions. These summary counts and proportions will be further stratified based on relevant features such as study design, domains of health care quality, and rank and domains of evidence. When possible, articles and measures will be pooled and further analysis will be performed; however, due to the heterogeneity as well as broad scope of material, it is expected that it will not be possible to pool all measures for analysis. All analyses will be performed using STATA statistical software, version 13 (Stata Corp, College Station, Texas).
Population growth, advances in medical science, and improved capability to support critically ill patients have all translated into a sustained and rising demand for critical care services . This increased demand, however, has not and likely cannot be universally accompanied by an increased supply of critical care resources, which are costly . The costs of expanding critical care extend beyond the “ICU bed” per se and necessitate considerable investment in human resources, specialized equipment (i.e., mechanical ventilators), and supplies to sustainably operate. Moreover, the supply of critical care services is not standardized and is highly inconsistent across jurisdictions for reasons that are not based on evidence-informed scientific assessment of need [23–25]. These observations would imply some degree of redundant inefficiency for the provision of critical care.
Regardless of the reasons, mismatches between demand and supply for critical care services are increasingly encountered . This mismatch on any given day in any given ICU will create a strain on that ICU’s capacity to accommodate the next sickest patient. Accumulating evidence suggests that strained ICUs are at risk of providing suboptimal quality of care . Moreover, such strain drives higher risk of adverse events, premature discharges, unplanned readmissions, and death [7, 8, 10, 12, 15, 26]. Arguably, sustained strain on ICU capacity directly contributes to inefficient utilization of a valuable finite resource. While opening additional ICU beds may seem the simplest response, this is not necessarily sustainable and likely only a short-sighted solution.
In terms of ICU capacity strain, the challenge for health care providers is to clearly understand when and to what extent strain is negatively impacting the quality of care provided and the performance of a given ICU and to readily identify and respond to factors most responsible. There is uncertainty on how to measure ICU capacity strain, and it is unlikely a single measure will satisfy all the potential domains from which strain may originate. Moreover, there may be a limited appreciation for the lateral impact of ICU capacity strain, such as a negative impact on ambulance offloads, emergency department crowding, and postponements of elective surgery. In our preparation for this systematic review, we found little agreement upon measures of capacity strain that would enable quantification of the immediate or temporal “stress” an ICU may be experiencing due to strain on capacity.
Simple, translatable, and easily reportable quality and/or performance measures are needed to diagnose and quantify ICU capacity strain. This is a fundamental initial step towards developing innovative quality improvement interventions aimed to improve efficiency (e.g., access, throughput), avoid waste (e.g., reducing wait time for ICU discharge), and better anticipate impending capacity shortfalls. The next steps in our program focused on ICU capacity strain following completion of this systematic review will involve an evaluation of each identified measure by key stakeholders and experts, followed by prospective appraisal in a clinical context.
We anticipate our systematic review will yield a spectrum of heterogeneous measures across a number of potential clinical domains impacted by ICU capacity strain. Similarly, we expect identified measures will be of variable reliability/validity and will have undergone variable implementation and degrees of rigorous assessment. We anticipate there will be limited opportunity for pooled analysis; however, this is not the primary objective of our systematic review. Finally, as aforementioned, we anticipate that a concise “bundle” of measures will likely be necessary to optimally monitor for and characterize strain on ICU capacity, the composition of which will require further prospective evaluation.
Alberta Research Centre for Health Evidence
Cochrane Central Register of Controlled Trials
Consensus-based Standards for the selection of health Measurement Instruments
intensive care unit
Newcastle-Ottawa Quality Assessment Scale
Canada S. Diseases and health conditions. Government of Canada; 2013 [cited 2014]. Available from: http://www5.statcan.gc.ca/subject-sujet/result-resultat.action?pid=2966&id=1887&lang=eng&type=CST&pageNum=1&more=0
Bagshaw SM, Webb SA, Delaney A, George C, Pilcher D, Hart GK, et al. Very old patients admitted to intensive care in Australia and New Zealand: a multi-centre cohort analysis. Crit Care. 2009;13(2):R45.
Finfer S, Vincent JL. Critical care—an all-encompassing specialty. N Engl J Med. 2013;369(7):669–70.
Halpern SD. ICU capacity strain and the quality and allocation of critical care. Curr Opin Crit Care. 2011;17(6):648–57.
Azevedo LC, de Souza IA, Zygun DA, Stelfox HT, Bagshaw SM. Association between nighttime ICU discharge from the intensive care unit and hospital mortality: a multi-centre retrospective cohort study. BMC Health Serv Res. 2015;15(1):378.
Baker DR, Pronovost PJ, Morlock LL, Geocadin RG, Holzmueller CG. Patient flow variability and unplanned readmissions to an intensive care unit. Crit Care Med. 2009;37(11):2882–7.
Chalfin DB, Trzeciak S, Likourezos A, Baumann BM, Dellinger RP, group D-Es. Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Crit Care Med. 2007;35(6):1477–83.
Daud-Gallotti RM, Costa SF, Guimaraes T, Padilha KG, Inoue EN, Vasconcelos TN, et al. Nursing workload as a risk factor for healthcare associated infections in ICU: a prospective study. PLoS One. 2012;7(12), e52342.
Gabler NB, Ratcliffe SJ, Wagner J, Asch DA, Rubenfeld GD, Angus DC, et al. Mortality among patients admitted to strained intensive care units. Am J Respir Crit Care Med. 2013;188(7):800–6.
Laupland KB, Misset B, Souweine B, Tabah A, Azoulay E, Goldgran-Toledano D, et al. Mortality associated with timing of admission to and discharge from ICU: a retrospective cohort study. BMC Health Serv Res. 2011;11:321.
Wagner J, Gabler NB, Ratcliffe SJ, Brown SE, Strom BL, Halpern SD. Outcomes among patients discharged from busy intensive care units. Ann Intern Med. 2013;159(7):447–55.
Cardoso LT, Grion CM, Matsuo T, Anami EH, Kauss IA, Seko L, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care. 2011;15(1):R28.
Stelfox HT, Hemmelgarn BR, Bagshaw SM, Gao S, Doig CJ, Nijssen-Jordan C, et al. Intensive care unit bed availability and outcomes for hospitalized patients with sudden clinical deterioration. Arch Intern Med. 2012;172(6):467–74.
Robert R, Reignier J, Tournoux-Facon C, Boulain T, Lesieur O, Gissot V, et al. Refusal of intensive care unit admission due to a full unit: impact on mortality. Am J Respir Crit Care Med. 2012;185(10):1081–7.
Goldfrad C, Rowan K. Consequences of discharges from intensive care at night. Lancet. 2000;355(9210):1138–42.
Dissemination CfRa. Systematic reviews: CRD’s guidance for undertaking reviews in health care: University of York; 2009
Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.
Wells GA, Shea B, O’connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa, Canada: Department of Epidemiology and Community Medicine, University of Ottawa; 2011 [cited 2015 July 10, 2015]. Available from: http://www.ohri.ca/programs/.
McGlynn EA. Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Med Care. 2003;41(1 Suppl):I1–7.
Stelfox HT, Straus SE. Measuring quality of care: considering measurement frameworks and needs assessment to guide quality indicator development. J Clin Epidemiol. 2013;66(12):1320–7.
Stelfox HT, Straus SE. Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation. J Clin Epidemiol. 2013;66(12):1328–37.
Halpern NA, Pastores SM, Greenstein RJ. Critical care medicine in the United States 1985–2000: an analysis of bed numbers, use, and costs. Crit Care Med. 2004;32(6):1254–9.
Gershengorn HB, Iwashyna TJ, Cooke CR, Scales DC, Kahn JM, Wunsch H. Variation in use of intensive care for adults with diabetic ketoacidosis*. Crit Care Med. 2012;40(7):2009–15.
Rhodes A, Ferdinande P, Flaatten H, Guidet B, Metnitz PG, Moreno RP. The variability of critical care bed numbers in Europe. Intensive Care Med. 2012;38(10):1647–53.
Wunsch H, Angus DC, Harrison DA, Collange O, Fowler R, Hoste EA, et al. Variation in critical care services across North America and Western Europe. Crit Care Med. 2008;36(10):2787–93. e1-9.
Garrouste-Orgeas M, Montuclard L, Timsit JF, Reignier J, Desmettre T, Karoubi P, et al. Predictors of intensive care unit refusal in French intensive care units: a multiple-center study. Crit Care Med. 2005;33(4):750–5.
SMB is support by a Canada Research Chair (CRC) in Critical Care Nephrology. HTS is supported by an Independent Investigator Award from Alberta Innovates—Health Solutions (AIHS). LH is supported by a CIHR New Investigator Salary Award.
This work was performed at the University of Alberta.
No author has any competing interests to declare.
MS and SMB were responsible for the preparation of the protocol and manuscript preparation. MS, AI, DZ, HTS, DO, SMB were responsible for finalizing the protocol, statistical methods and completion of the final manuscript. LH informed the methods for conducting the review. RF developed the search strategy in consultation with MS and SMB. SMB conceived the project, developed the protocol, wrote and revised the manuscript. All authors provided critical revision of the protocol and final manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Soltani, S.A., Ingolfsson, A., Zygun, D.A. et al. Quality and performance measures of strain on intensive care capacity: a protocol for a systematic review. Syst Rev 4, 158 (2015). https://doi.org/10.1186/s13643-015-0145-9
- Quality indicator
- Performance measure
- Intensive care
- Health services