Skip to main content

Telemedicine with clinical decision support for critical care: a systematic review



Telemedicine applications aim to address variance in clinical outcomes and increase access to specialist expertise. Despite widespread implementation, there is little robust evidence about cost-effectiveness, clinical benefits, and impact on quality and safety of critical care telemedicine. The primary objective was to determine the impact of critical care telemedicine (with clinical decision support available 24/7) on intensive care unit (ICU) and hospital mortality and length of stay in adults and children. The secondary objectives included staff and patient experience, costs, protocol adherence, and adverse events.


Data sources included MEDLINE, EMBASE, CINAHL, Cochrane Library databases, Health Technology Assessment Database, Web of Science, OpenGrey, OpenDOAR, and the HMIC through to December 2015. Randomised controlled trials and quasi-experimental studies were eligible for inclusion. Eligible studies reported on differences between groups using the telemedicine intervention and standard care. Two review authors screened abstracts and assessed potentially eligible studies using Cochrane guidance.


Two controlled before-after studies met the inclusion criteria. Both were assessed as high risk of bias. Meta-analysis was not possible as we were unable to disaggregate data between the two studies. One study used a non-randomised stepped-wedge design in seven ICUs. Hospital mortality was the primary outcome which showed a reduction from 13.6 % (CI, 11.9–15.4 %) to 11.8 % (CI, 10.9–12.8 %) during the intervention period with an adjusted odds ratio (OR) of 0.40 (95 % CI, 0.31–0.52; p = .005). The second study used a non-randomised, unblinded, pre-/post-assessment of telemedicine interventions in 56 adult ICUs. Hospital mortality (primary outcome) reduced from 11 to 10 % (adjusted hazard ratio (HR) = 0.84; CI, 0.78–0.89; p = <.001).


This review highlights the poor methodological quality of most studies investigating critical care telemedicine. The results of the two included studies showed a reduction in hospital mortality in patients receiving the intervention. Further multi-site randomised controlled trials or quasi-experimental studies with accompanying process evaluations are urgently needed to determine effectiveness, implementation, and associated costs.

Trial registration

PROSPERO CRD42014007406

Peer Review reports


Population-based studies in the developed world suggest that the burden of critical illness is higher than appreciated and will increase as the population ages [1]. Existing critical care structures and organisational processes are perceived as inadequate to efficiently support these demands [2, 3].

Access to critical care specialists is also not provided for a number of patients in rural areas, and some hospitals provide only daytime intensivistFootnote 1 cover [4]. Implementation of evidence-based guidelines remain problematic [5, 6], and outcomes for critically ill patients demonstrate variation at the hospital, regional, and national levels [7, 8]. Safety culture varies widely across ICUs [9], and medical errors are common in critical care settings due to the fast-paced, complex nature of the work and the vulnerability of the patients [10, 11].

Telemedicine is proposed as a potential solution to address these structural inadequacies in critical care resource and variability in clinical outcomes and access to specialist expertise across units. Telemedicine is the use of telecommunications technology for medical diagnosis and patient care [12]. It offers a solution to structural problems affecting access to care [13, 14] and an additional safety net to support existing services. Critical care telemedicine uses a remotely located support centre housing a critical care team who are networked with a number of bedside critical care unit teams and patients via audio-visual communication and computer systems over the 24-h period. It offers the potential for multiple opportunities for safety and quality improvement on account of off-site support provided by intensivists and critical care nurses, continuous monitoring with early warning capabilities, rounding tools to monitor at-risk patients, inbuilt clinical decision support, and prompts regarding adherence to best practice.

Two recent Cochrane reviews have concluded there is little robust evidence about telemedicine’s cost-effectiveness, clinical benefits, and impact on quality and safety [12, 15]. These reviews focused on telemedicine applications that involve direct patient care, in which the patient is remote from the clinician. The telemedicine delivers clinical information and permits consultation and discussion between healthcare professionals and patients regardless of where the patient is located, for example, the remote monitoring of patients with chronic conditions at home, the provision of specialist consultations for patients via video-conferencing, and the provision of clinical information for patient self-management [12, 15].

Missing from these reviews are studies of telemedicine applications involving provider-provider interaction either in addition to or substituting for usual care. Critical care telemedicine is one such model as it provides remote specialist provider support to bedside staff while also involving the patient at the point of care. Existing reviews focusing on critical care telemedicine have so far been limited methodologically [1619] and have merged together diverse models of application, thus limiting their ability to reach meaningful conclusions about clinical and service utility. Currell et al. [12] and Flodgren et al. note that in order to answer questions about the efficacy of telemedicine, reviews need to focus on particular study populations and intervention functions, as well as staffing models and healthcare systems involved in delivering the intervention.

The primary objective of this review is to compare the effect of 24-h telemedicine models of critical care with standard models of care for acutely ill adults and children.


The review adhered to recommendations in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement [20] and used Cochrane methods guidance [21]. The review protocol is registered on the PROSPERO prospective register of systematic reviews (CRD42014007406).

Criteria for considering studies for this review

Types of studies

All randomised controlled trials, quasi-randomised controlled trials, controlled before-after studies, and interrupted time series studies which evaluated critical care telemedicine were included in the review [22]. We took guidance from the EPOC resources to determine the criteria for including studies employing these designs [23].

Types of participants

The population included any adults or children of either gender or any age, or ethnic group admitted to a critical care setting (this included coronary care, high dependency, and intensive care). All conditions and grades of acuity/severity of illness were included within the study population. Acutely ill adults and children cared for outside critical care settings were excluded.

Types of interventions

Studies were considered eligible for inclusion if the telemedicine intervention included (1) continuous electronic recording of patients’ vital signs at the bedside which was linked to a computer system enabling display of real-time data and (2) use of clinical decision-making algorithms and electronic alerts by (3) a remotely located team of critical care specialists including doctors, available 24/7. The review excluded telemedicine applications that were periodic (e.g. intermittent rounding or video consultations) or excluded medical decision-making (e.g. nurse-led remote screening of best practice).

Types of outcome measures

Primary outcomes

The primary outcomes were ICU and hospital mortality and length of stay.

Secondary outcomes

Additional outcomes were adverse events, staff and patient experience, costs, and protocol adherence.

Search methods for identification of studies

Electronic searches

The database search strategy is detailed in Table 1 and was conducted through to December 16, 2015; no date or language restrictions were imposed. Databases searched were MEDLINE, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), the Cochrane Library, Health Technology Assessment Database, and Web of Science (science and social science citation index). The search strategy was applied consistently across the databases. To avoid publication bias, we also searched for grey literature using OpenGrey, OpenDOAR (the Directory of Open-Access Repositories), and the HMIC (Healthcare Management Information Consortium).

Table 1 Search strategy

Searching other resources

In addition, we hand searched journals, conference proceedings, and journal supplements for papers related to telemedicine in critical care. Forward citation searching of relevant papers was also utilised.

Data collection and analysis

Study selection

The titles and abstracts of each study were independently reviewed and assessed for eligibility by a team of reviewers (NM with JS/AM/JS/EL). We used Covidence (an online systematic review production platform) to systematise the screening process through independent assessment from two reviewers. Any disagreement or lack of consensus was resolved through consultation with two other authors (MT and RM).

Data extraction and management

Data extraction was carried out independently by KF, JK, SN, AM, JS, and EL using a standardised, pre-piloted form. Extracted information included country; study setting including size and type of hospital, university affiliation, and teaching status; critical care setting including type of unit and specialisation, study population, and participant demographics; details of baseline conditions including unit staffing ratios of medical and nursing staff and allied health professionals and utilisation of information technology; details of the intervention including the nature of telemonitoring and clinical decision support, location of the telemedicine support centre, and staffing ratios and roles; commercial sponsorship and study methodology; recruitment and study completion dates; outcomes and times of measurement; and suggested mechanisms of intervention action.

Assessment of risk of bias in included studies

Study quality was assessed by the same team using the risk of bias criteria for EPOC reviews [24]. We collected information about allocation, baseline measurements, blinding, reliability of primary outcomes, protection against contamination, and other risks of bias. Any discrepancies identified were resolved through discussion with NM.

Data synthesis

We used Review Manager 5 (The Cochrane Collaboration, Oxford, England) for data synthesis. In the results, we report the following data: pre-intervention and telemedicine data and statistical significance across groups, absolute and percentage improvement, and adjusted odds and hazard ratios. Meta-analysis was not possible as one of the studies was also included within the dataset of the second study. Following correspondence with the authors, it was not possible to disaggregate the data from the two studies with sufficient confidence to use meta-analysis to pool results. Instead, we present the results of the studies and make a qualitative assessment of the effects of both studies, based on quality.


Results of the search

The initial database search in February 2014 identified 4390 studies with 39 additional records identified through other sources. An updated 2014–2015 search identified a further 1481 studies (Fig. 1 PRISMA flow diagram and Additional file 1 PRISMA Checklist). One hundred one full-text reports were reviewed. Four authors were contacted for details about the telemedicine model. The studies evaluated a wide variety of interventions, usually with very brief descriptions of the intervention. Once we excluded commentaries and studies that restricted their telemedicine models to remote screening of best practice adherence, intermittent rounding models, or limited intensivist coverage to night-time coverage, we were left with 11 potential studies that included continuous critical care telemedicine models with 24-h coverage.

Fig. 1
figure 1

PRISMA flow diagram

Excluded studies

Nine studies were excluded as the study designs did not meet the EPOC criteria for study design. A brief summary of the studies can be found in the Appendix. The majority [6] of these were excluded because they reported uncontrolled before-after study designs. Two used an interrupted time series design without enough data points pre- and post-intervention. One used a controlled before-after study design with only one control and intervention site.

Included studies

Two controlled before-after US-based studies met our inclusion criteria. Tables 2 and 3 provide a summary of the study designs and the telemedicine applications. Lilly et al. [25] used a prospective, unblinded, stepped-wedge study over 30 months, between 2006 and 2007. The research involved one academic centre with two campuses, seven adult ICUs (medical, cardiovascular, and surgical), and 834 beds. Six thousand two hundred ninety patients were included in the study: 1529 control subjects and 4761 intervention subjects.

Table 2 Characteristics of included studies
Table 3 TIDieR (Template for Intervention Description and Replication) checklist [39]

Prior to implementation of the intervention, all the units operated a closed model of care with 24/7 coverage with an intensivist, plus prescribing providers.Footnote 2 The governance structure, ICU policies and procedures, call schedules, interdisciplinary rounding structure, and the numbers of provider team members were reported as constant during the study period. The critical care telemedicine system included audio and video connectivity between the bedside and the remote team, access to medical records, decision support and review of responses to alerts, screening and rounding tools, and monitoring of adherence to best practice guidelines. The team within the remotely located support centre consisted of a hospital staff intensivist, an ICU affiliate practitioner, a systems analyst, and one or more data clerks. Clinical staff from the support centre rotated through the ICUs. The clinical staff either communicated with bedside clinicians or directly managed patients by recording clinician orders for tests, treatments, consultations, and management of life-support devices. They had jurisdiction to intervene when the bedside clinicians’ response was delayed and patients were deemed physiologically unstable. They also managed the out-of-hours cases.

Lilly et al. [26] used a non-randomised, unblinded, pre-/post-assessment of telemedicine interventions over a 5-year period (2003–2008) in 56 adult ICUs. Twenty-one healthcare systems known to be implementing a telemedicine programme were invited to collect patient-level data using standardised instruments. Nineteen participating health systems enrolled patients. Dates regarding the end of baseline and start of the intervention period were not reported. Participating ICUs were geographically dispersed across 15 US states and included 38 hospitals that ranged in size from 88 to 834 beds. The ICUs served rural, suburban, and urban populations and included medical-surgical, medical, surgical, coronary care, neuroscience, and cardiothoracic units.

Data describing the characteristics of each ICU and process of care, as well as structural and organisational characteristics before and after the implementation of the telemedicine programme, were measured for each ICU using the American College of Chest Physicians ICU Telemedicine Survey instrument and reported in an adjunct paper [27]. The authors reported aggregate data from 170 ICUs in the adjunct study rather than unit-level data, so we were unable to link characteristics to the 56 units in the included study. The authors reported that each ICU implemented similar technical components, including audio and video connections, an ICU-focused medical record, and software for detecting evolving physiologic instability. Implementation changes in the process of care delivery, ICU admission procedures, rounding and governance structure, communication among caregivers, and how performance information was used, care was documented, and technical support was provided were not reported at the unit or healthcare system level.

Hospital mortality was the primary outcome for both studies with ICU mortality and ICU and hospital length of stay included as secondary outcomes. In Lilly et al. [25], case-mix and severity-adjusted hospital mortality was pre-specified as the main study outcome. They used the Simplified Acute Physiology Score (SAPS) and the Acute Physiology and Chronic Health Evaluation (APACHE III score) for severity adjustment. The authors reported unadjusted and adjusted outcomes at the unit and study levels. Their findings were analysed and compared using multivariate logistic regression. They additionally reported best practice adherence and complication rates.

Lilly et al. [26] reported pre-specified hazard ratio for dying in the hospital as the primary study outcome. They reported at the healthcare system level, adjusted and unadjusted outcomes for ICU and hospital mortality, and adjusted only data for ICU and hospital length of stay. They used the SAPS and the Acute Physiology and Chronic Health Evaluation (APACHE IV score) for acuity adjustment. Both crude and adjusted Cox proportional hazards regression models were constructed to evaluate the effects of the telemedicine interventions on hospital and ICU mortality.

Risk of bias in included studies

Neither of the included study designs considered allocation concealment. In Lilly et al. [25], a representative sample of pre-intervention cases was obtained by identifying consecutive hospital discharge cases from an administrative database for cases managed in each of the seven ICUs. At baseline, differences in mortality and length of stay were noted within and across the ICUs. Lilly et al. [26] included patient characteristics on admission as baseline data but did not report ICU-level data, so we were unable to assess similarities in baseline outcome measurements.

In Lilly et al. [25], the intervention group had a larger percentage of medical rather than surgical patients, with slightly more abnormal laboratory and physiological values, and higher acuity scores than cases in the pre-intervention group. Similarly, the authors in the Lilly et al. study [26] noted that the telemedicine group patients had significantly higher acuity scores and predicted mortality, had a larger proportion of medical primary admission diagnoses, were less likely to have been admitted from an operating room, and had a significantly different distribution of primary admission diagnoses.

There were no reports of missing outcome measures. Missing severity scores were reported in both studies. In Lilly et al. [25], the small numbers were unlikely to bias results as the proportion of missing data was similar in the intervention and control groups. However, in Lilly et al. [26], the sizeable number of missing APACHE scores could mislead about the adjusted mortality. Blinding of primary outcomes was not undertaken in either study. Reliable outcome measures were obtained from hospital databases. All relevant outcomes listed in the ‘Methods’ section were reported. Both studies reported that electronic and manual methods of collection by abstractors yielded similar datasets and severity scores.

Effects of intervention

Table 4 summarises the key findings from the two included studies.

Table 4 Effects of intervention

Hospital and ICU mortality

The findings of Lilly et al. [25] show that after adjusting for acuity, locus of care, physiological parameters, laboratory values, and time, there was a 60 % reduction in the odds of dying in the hospital in the telemedicine group (odds ratio (OR), 0.40; 95 % CI, 0.31–0.52; p = .005). The ICU mortality yielded an OR of 0.37 (95 % CI, 0.28–0.49; p = .003) after adjustment. Subgroup analysis showed the telemedicine intervention had greater effect on hospital mortality for patients admitted after 8 pm (OR, 0.33; CI, 0.18–0.59; p < .001) than those admitted after 8 am (OR, 0.79; CI, 0.39–1.58).

In the Lilly et al. study [26], adjusting for relevant covariates revealed significantly lower hospital (hazard ratio (HR), 0.84; CI, 0.78–0.89; p < .001) and ICU (HR, 0.74; CI, 0.68–0.79; p < .001) HRs for patients in the critical care telemedicine group compared with the control group.

Hospital and ICU length of stay

Lilly et al. [25] reported that after adjustment for acuity, time trends, physiological parameters, laboratory values, and locus of care, hospital length of stay (LOS) was significantly shorter in the telemedicine group (HR for discharge, 1.44 (95 % CI, 1.33–1.56); p < .001). ICU LOS, after adjustment for all of the previously listed variables, yielded a HR of 1.26 (95 % CI, 1.17–1.36; p < .001). The reduction in LOS attributed to the telemedicine intervention was most clinically meaningful among patients who stayed in the hospital or ICU for at least 1 week. Subgroup analysis showed that cases admitted after 8 pm had longer hospital LOS (14.3 days CI 12.99–15.57 days) in the pre-intervention group than those in the telemedicine intervention group (12.4 days CI 11.22–13.58; p = .04) and longer ICU LOS (pre-intervention 7.7 days CI 6.77–8.63 versus post-intervention 5.5 days CI 4.86–6.08; p < .001).

In the Lilly et al. study [26], after adjustment, ICU LOS for the telemedicine intervention group patients was 20 % shorter (CI, 19–22 %; p = <.001) and hospital LOS was 15 % shorter (CI, 14–17 %; p = <.001) compared with control subjects. The effectiveness of the interventions for reducing LOS was clinically meaningful only among patients who remained in the hospital for at least 1 week.

Adherence to best practice and complication rates

In Lilly et al. [25], the telemedicine intervention period compared with the pre-intervention period was associated with higher rates of adherence to best practice guidelines for deep vein thrombosis (DVT) prophylaxis [28], stress ulcer (SU) prophylaxis [29], cardiovascular protection [30, 31], and prevention of ventilator-associated pneumonia (VAP) [32, 33]. Preventable complication rates were also lower in the telemedicine intervention period compared with the pre-intervention period for VAP and catheter-related bloodstream infection (CRBSI) but not for acute kidney injury (AKI).

Associations with ICU process and setting of care domains

Lilly et al. [26] reported that adjusted analyses revealed that changes in the ICU characteristics domain (OR, 0.70; CI, 0.56–0.87; p < .01), physician leadership domain (OR, 0.80; CI, 0.70–0.92; p < .01), and best practices and performance review domain (OR, 0.82; CI, 0.71–0.95; p < .01) were associated with significant reductions in hospital mortality. Individual components of the interventions that were reported to be associated with lower mortality, reduced LOS, or both included (1) intensivist case review within 1 h of admission, (2) timely use of performance data, (3) adherence to ICU best practices, and (4) quicker alert response times


Despite the increase in interest in critical care telemedicine as a potential means to improve quality and safety, the evidence base is still lacking. Our review adds to others that have found that the evidence regarding the effectiveness of telemedicine is incomplete and the quality of much of the research is poor [12, 34]. Uncontrolled before-after studies form the largest group. Most of the interrupted time series studies contain too few data collection points to account for seasonal and secular trends which might affect the data and the auto-correlation among measurements repeatedly taken over time [35]. Description of the telemedicine interventions is extremely limited making it impossible to disentangle the impact of the different elements of the telemedicine models. Very few studies consider economic outcomes, patient-reported outcomes (e.g. quality of life), or impact on work organisation and staff. There is a paucity of long-term follow-up of the effects of the intervention to allow for organisational and cultural change. We know little about patient satisfaction with critical care telemedicine and need more research on the mechanisms that contribute to changes in perceptions about safety and quality of care, particularly as telemedicine may alter relationships between patients and health professionals.

While the two included studies met the criteria for inclusion for the review, it is important to acknowledge that due to difficulties disaggregating the data between Lilly et al. [25] and Lilly et al. [26], these two studies were not independent of each other. On account of being controlled before-after studies, they were assessed as being at high risk of bias [21]. Comparison across studies and assessment of heterogeneity in terms of interventions and settings was limited by the lack of reporting detail in Lilly et al. [26].

In terms of Lilly et al.’s [25] stepped-wedge design, the variability in the length of time for the step units, which ranged from 1 day to 5 months, raises concerns about controlling for underlying temporal trends [36]. The sample size in Lilly et al. [25] was small, and the study had limited power for detecting a significant difference. Lower mortality was however associated with the telemedicine model both within ICUs over time and across ICUs during the same periods, which does suggest that the results may not be due purely to time trends [37]. Lilly et al. [25] also note that the higher acuity scores for the pre-intervention group were due in part to the staggered implementation of the telemedicine intervention and to higher rates of transfer of higher acuity medical cases from outside hospitals for the telemedicine group.

It was difficult in both studies to untangle the mechanisms involved in contributing to improved outcomes. Lilly et al. [25] attributed lower mortality to best practice adherence and decreased complication rates targeted by the intervention, although they did not report on out-of-hours versus in-hours compliance data. Telemonitoring system enables near real-time auditing and reconciliation of best practice in contrast to routine audit and feedback in critical care which is often untimely, incomplete, and not actionable [38]. Although Lilly et al. [25] suggested mortality benefits may have resulted from more rapid response to alerts for physiological instability, no baseline data was collected on triggering and response behaviour in relation to escalation. While data detailing the number of alerts and ratings of severity were generated post-intervention, this gives little indication about the timeliness or efficacy of the resulting clinical action that occurred (or did not). Neither study provided understanding about the technology’s diagnostic accuracy and the utility of the decision aids.

Lilly et al. [25] provides useful insight into bedside and off-site staffing and skill-mix and suggests team collaboration is an important variable. Lilly et al. [26] attributed earlier intensivist management, coordinated timely usage of performance information, achievement of higher rates of adherence to best practices, shorter alarm response times, more frequent interdisciplinary rounds, and a more effective ICU committee to their improved outcomes, based on the process data collected in their adjunct study [27]. They also suggest these effects are additive. However, as we did not have access to the linked data, we were unable to assess its quality.

There was a paucity of data regarding local tailoring of the programme to the local site and efforts to ensure fidelity [39]. While Lilly et al. [25] provided a detailed description of context and the programme, Lilly et al. [26] only referred to aggregate findings reported in an adjunct paper [27], so we were unable to extrapolate the ICU characteristics, structures, and processes related to the sites included in their study. The lack of standardisation of the intervention across different sites limits the ability to pool data, where sample populations, spread patterns, and definition of measures for data collection may vary. This additionally hampers the ability to compare sites for research purposes and limits the generalisability of any findings beyond their original context [40].

Organisations vary considerably in their capacity for clinical system improvement [41]. In the Lilly et al. study [25], best practices for the prevention of venous thrombosis, cardiovascular complications, ventilator-associated pneumonia, and stress ulcers were standardised and ICU daily goal setting was introduced a year before the start of the study, implying that there was opportunity for quality improvement at this stage. In the Lilly et al. study [26], all the ICUs that took part were self-selected, based on a willingness to invest in care improvement. However, the starting points of care delivery systems in terms of information technology, service improvement practices, systems, and structures [40] were not reported. The intervention period for Lilly et al. [25] was 16 months which is relatively short given the nature of organisational and cultural change associated with implementation of a complex intervention such as telemedicine.


Implications for practice

Critical care telemedicine serves a recognised need within North America, given the number of underserviced rural hospitals, lack of intensivist cover at night, and shortage of availability of intensivists. Outside the USA, providers and commissioners are showing increasing interest in telemedicine as a means of using technology effectively to re-think the way care is delivered across health economies [42]. Critical care telemedicine has important safety and quality implications as it can enable critically ill patients to stay in remote settings rather than travel to specialist centres. A networked hub and spoke structure may offer potential to level out variance in outcomes and ensure 24/7 access to specialist expertise which has implications for future workforce planning. In addition to potentially providing an additional safety net to ward-based teams, critical care telemedicine could provide a mechanism for near real-time feedback to improve situational awareness and accountability for individual actions. However, the lack of robust studies can hinder effective commissioning and can make justification of costs difficult.

Implications for research

Our review highlights the need for further methodologically robust evaluations of critical care telemedicine. The studies that were included were not independent of each other, which highlights the importance of opening up the field to include independent evaluations of different vendors, teams, health systems, and health economies. The impact of implementation of system transformation may take some years to achieve as it necessitates cultural change [43]. Adequately powered cluster randomised trials or randomised stepped-wedge or well-designed interrupted time series studies with 12–24-month follow-up would offer a pragmatic trade-off between study design and feasibility. Researchers should consider potential modifying variables such as healthcare skill-mix ratios (both unit and off-site cover) and jurisdictional, organisational, and regulatory influences. A logic model, a diagrammatic representation of the theory of the intervention, can help document causal assumptions [44]. In order to enable understanding of telemedicine as a complex intervention, precise definitions of interventions and consideration of contextual factors and interaction effects need to be provided [39]. Detailed descriptions need to include not only the technical set-up but also how it interacts with care organisation, clinical decision-making, and professional practice to open up the black box of implementation. Future studies require accompanying process evaluations to generate theory and guide replicability and enable measurement of different end points across the causal chain and fidelity of uptake of the components of the intervention [45]. Future research also needs to incorporate an economic analysis [18]. Data needs to be collected about escalation of concerns and the timeliness or efficacy of clinical action that occurs (or does not) as a result of critical care telemedicine. Electronic surveillance systems can generate a number of false alarms with adverse consequences for patient safety [46]. Future research needs to increase understanding about the technology’s diagnostic accuracy and the utility of the decision aids. Outcome measures should be expanded to include patient- or staff-derived outcomes (e.g. quality of life, patient satisfaction, staff burnout, acceptability of telemedicine applications to providers and patients) [13]. Longer term follow-up of the impact of telemedicine on patients may help us understand its impact on patients who survive serious illness experience but experience ‘chronic critical illness’ that continues well beyond ICU discharge and often culminates in long-term morbidity and mortality [47].


This review concludes that policy-makers should remain cautious about recommending increased use of, and investment in, critical care telemedicine while there is little robust evidence of clinical and economic benefit [1].


  1. Refers to a physician with intensive care certification.

  2. Prescribing providers are physicians, or critical care trained nurse practitioners, or physician assistants.



Acute kidney injury


Acute Physiology and Chronic Health Evaluation


Confidence interval


Catheter-related bloodstream infection


Hazard ratio


Intensive care unit


Odds ratio


Simplified Acute Physiology Score


Ventilator-associated pneumonia


  1. Adhikari NK, Fowler RA, Bhagwanjee S, Rubenfeld GD. Critical care and the global burden of critical illness in adults. The Lancet. 2010;376(9749):1339–46.

    Article  Google Scholar 

  2. Garland A. Improving the ICU: part 2. CHEST Journal. 2005;127(6):2165–79.

    Article  Google Scholar 

  3. Needham D, Bronskill S, Calinawan J, Sibbald W, Pronovost P, Laupacis A. Projected incidence of mechanical ventilation in Ontario to 2026: preparing for the aging baby boomers. Crit Care Med. 2005;33(3):574–9.

    Article  PubMed  Google Scholar 

  4. Angus DC, Shorr AF, White A, Dremsizov TT, Schmitz RJ, Kelley MA. Critical care delivery in the United States: distribution of services and compliance with Leapfrog recommendations. Crit Care Med. 2006;34(4):1016–24.

    Article  PubMed  Google Scholar 

  5. Dainty KN, Scales DC, Sinuff T, Zwarenstein M. Competition in collaborative clothing: a qualitative case study of influences on collaborative quality improvement in the ICU. BMJ Quality & Safety. 2013;22(4):317–23.

    Article  Google Scholar 

  6. Pronovost PJ, Rinke ML, Emery K, Dennison C, Blackledge C, Berenholtz SM. Interventions to reduce mortality among patients treated in intensive care units. J Crit Care. 2004;19(3):158–64.

    Article  PubMed  Google Scholar 

  7. Wunsch H, Angus DC, Harrison DA, Collange O, Fowler R, Hoste EAJ, et al. Variation in critical care services across North America and Western Europe. Crit Care Med. 2008;36(10):2787–93.

    Article  PubMed  Google Scholar 

  8. Kahn JM, Goss CH, Heagerty PJ, Kramer AA, O'Brien CR, Rubenfeld GD. Hospital volume and the outcomes of mechanical ventilation. N Engl J Med. 2006;355(1):41–50.

    Article  CAS  PubMed  Google Scholar 

  9. Huang DT, Clermont G, Sexton JB, Karlo CA, Miller RG, Weissfeld LA, et al. Perceptions of safety culture vary across the intensive care units of a single institution. Crit Care Med. 2007;35(1):165–76.

    Article  PubMed  Google Scholar 

  10. Nguyen Y-L, Wunsch H, Angus DC. Critical care: the impact of organization and management on outcomes. Curr Opin Crit Care. 2010;16(5):487–92.

    Article  PubMed  Google Scholar 

  11. Rothschild JM, Landrigan CP, Cronin JW, Kaushal R, Lockley SW, Burdick E, et al. The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med. 2005;33(8):1694–700.

    Article  PubMed  Google Scholar 

  12. Currell R, Urquhart C, Wainwright P, Lewis R. Telemedicine versus face to face patient care: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;2(2).

  13. May C, Gask L, Atkinson T, Ellis N, Mair F, Esmail A. Resisting and promoting new technologies in clinical practice: the case of telepsychiatry. Soc Sci Med. 2001;52(12):1889–901.

    Article  CAS  PubMed  Google Scholar 

  14. Catwell L, Sheikh A. Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Med. 2009;6(8):e1000126.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2015;9.

  16. Wilcox ME, Adhikari N. The effect of telemedicine in critically ill patients: systematic review and meta-analysis. Crit Care. 2012;16(4):R127.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Young LB, Chan PS, Lu X, Nallamothu BK, Sasson C, Cram PM. Impact of telemedicine intensive care unit coverage on patient outcomes: a systematic review and meta-analysis. Arch Intern Med. 2011;171(6):498–506.

    Article  PubMed  Google Scholar 

  18. Kumar G, Falk DM, Bonello RS, Kahn JM, Perencevich E, Cram P. The costs of critical care telemedicine programs: a systematic review and analysis. CHEST Journal. 2013;143(1):19–29.

    Article  Google Scholar 

  19. Ramnath VR, Ho L, Maggio LA, Khazeni N. Centralized monitoring and virtual consultant models of tele-ICU care: a systematic review. Telemedicine and e-Health. 2014;20(10):936–61.

    Article  PubMed  Google Scholar 

  20. Hutton B, Moher D, Cameron C. The PRISMA Extension Statement. Ann Intern Med. 2015;163(7):566.

    Article  PubMed  Google Scholar 

  21. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from

  22. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 2. Study design. Qual Saf Health Care. 2008;17(3):163–9.

    Article  CAS  PubMed  Google Scholar 

  23. Effective Practice and Organisation of Care (EPOC). What study designs should be included in an EPOC review? EPOC resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2013. Available at:

    Google Scholar 

  24. EPOC. Effective Practice and Organisation of Care (EPOC). [Suggested risk of bias criteria for EPOC reviews]. Available at: Accessed 1 Sept 2016.

  25. Lilly CM, Cody S, Zhao H, Landry K, Baker SP, McIlwaine J, et al. Hospital mortality, length of stay, and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. The Journal of the American Medical Association. 2011;305(21):2175–83.

    Article  CAS  PubMed  Google Scholar 

  26. Lilly CM, McLaughlin JM, Zhao H, Baker SP, Cody S, Irwin RS. A multi-center study of ICU telemedicine reengineering of adult critical care. CHEST Journal. 2014;145(3):500–7.

    Article  Google Scholar 

  27. Lilly C, Fisher K, Ries M, Pastores S, Vender J, Pitts J, et al. A national ICU telemedicine survey. Validation and Results Chest. 2012;142(1):40–7.

    PubMed  Google Scholar 

  28. Hirsh J, Dalen JE, Guyatt G. The sixth (2000) ACCP guidelines for antithrombotic therapy for prevention and treatment of thrombosis. CHEST Journal. 2001;119(1_suppl):1S–2S.

    Article  CAS  Google Scholar 

  29. Harty RF, Ancha HB. Stress ulcer bleeding. Current Treatment Options in Gastroenterology. 2006;9(2):157–66.

    Article  PubMed  Google Scholar 

  30. Cannon C, Driesman M, Mehta S, Rapaport E, Moses J, Tcheng J, et al. Improving acute coronary syndrome care: the ACC/AHA guidelines and critical pathways. Discussion. J Invasive Cardiol. 2003;15:22B–9B.

    PubMed  Google Scholar 

  31. Moser DK, Kimble LP, Alberts MJ, Alonzo A, Croft JB, Dracup K, et al. Reducing delay in seeking treatment by patients with acute coronary syndrome and stroke: a scientific statement from the American Heart Association Council on Cardiovascular Nursing and Stroke Council. J Cardiovasc Nurs. 2007;22(4):326–43.

    Article  PubMed  Google Scholar 

  32. Collard HR, Saint S, Matthay MA. Prevention of ventilator-associated pneumonia: an evidence-based systematic review. Ann Intern Med. 2003;138(6):494–501.

    Article  PubMed  Google Scholar 

  33. Isakow W, Kollef MH, editors. Preventing ventilator-associated pneumonia: an evidence-based approach of modifiable risk factors. Semin Respir Crit Care Med. 2006.

  34. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inform. 2010;79(11):736–71.

    Article  PubMed  Google Scholar 

  35. Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int J Technol Assess Health Care. 2003;19(04):613–23.

    Article  PubMed  Google Scholar 

  36. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6(1):54.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Kahn JM. The use and misuse of ICU telemedicine. JAMA (Chicago, Ill). 2011;305(21):2227–8.

    CAS  Google Scholar 

  38. Sinuff T, Muscedere J, Rozmovits L, Dale CM, Scales DC. A qualitative study of the variable effects of audit and feedback in the ICU. BMJ Quality & Safety. 2015;24(6):393–9.

    Article  Google Scholar 

  39. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ [Br Med J]. 2014;348:g1687

  40. Benn J, Burnett S, Parand A, Pinto A, Iskander S, Vincent C. Studying large-scale programmes to improve patient safety in whole care systems: challenges for research. Soc Sci Med. 2009;69(12):1767–76.

    Article  PubMed  Google Scholar 

  41. Walley P, Rayment J, Cooke M. Clinical systems improvement in NHS hospital trusts and their PCTs—a snapshot of current practice. Warwick University NHS Institute for Innovation and Improvement; 2006. Accessed 14 Oct 2016.

  42. Government Publications. 2016. Accessed 29 Apr 2016.

  43. Santamaria JM, Tobin AM, Holmes JBN. Changing cardiac arrest and hospital mortality rates through a medical emergency team takes time and constant review. Crit Care Med. 2010;38(2):445–50.

    Article  PubMed  Google Scholar 

  44. Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions. Medical Research Council guidance. BMJ. 2015;350:h1258

  45. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all. Qual Saf Health Care. 2008;17(3):178–81.

    Article  CAS  PubMed  Google Scholar 

  46. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14(5):542–9.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Marini JJ, Vincent J, Annane D. Critical care evidence—new directions. JAMA. 2015;313(9):893–4.

    Article  CAS  PubMed  Google Scholar 

  48. Dhakal A, Pathak L, Dhakal B. eICU—is it effective? CHEST Journal. 2013;144(4_MeetingAbstracts):552A-A.

    Article  Google Scholar 

  49. Gupta S, Varma A, Dewan S, Kaushal A. 274: EICU can save lives in myocardial infarction in remote areas in the developing world. Crit Care Med. 2013;41(12):A63.

    Article  Google Scholar 

  50. Kohl BA, Fortino-Mullen M, Praestgaard A, Hanson CW, DiMartino J, Ochroch EA. The effect of ICU telemedicine on mortality and length of stay. J Telemed Telecare. 2012;18(5):282–6.

    Article  PubMed  Google Scholar 

  51. Morrison JL, Cai Q, Davis N, Yan Y, Berbaum ML, Ries M, et al. Clinical and economic outcomes of the electronic intensive care unit: results from two community hospitals. Crit Care Med. 2010;38(1):2–8.

    Article  PubMed  Google Scholar 

  52. Nassar BS, Vaughan-Sarrazin MS, Jiang L, Reisinger HS, Bonello R, Cram P. Impact of an intensive care unit telemedicine program on patient outcomes in an integrated health care system. JAMA Intern Med. 2014;174(7):1160–7.

    Article  PubMed  Google Scholar 

  53. Sadaka F, Palagiri A, Trottier S, Deibert W, Gudmestad D, Sommer SE, et al. Telemedicine intervention improves ICU outcomes. Crit Care Res Prac. 2013;2013(456389):1–5

  54. Thomas EJ, Lucke JF, Wueste L, Weavind L, Patel B. Association of telemedicine for remote monitoring of intensive care patients with mortality, complications, and length of stay. JAMA. 2009;302(24):2671–8.

    Article  CAS  PubMed  Google Scholar 

  55. Willmitch BRNBSN, Golembeski SPRNC, Kim SSMAM, Nelson LDMDFF, Gidel LMDPF. Clinical outcomes after telemedicine intensive care unit implementation. Crit Care Med. 2012;40(2):450–4.

    Article  PubMed  Google Scholar 

  56. Zawada Jr ET, Herr P, Larson D, Fromm R, Kapaska D, Erickson D. Impact of an intensive care unit telemedicine program on a rural health care system. Postgrad Med. 2009;121(3):160–70.

    Article  PubMed  Google Scholar 

Download references


Not applicable.


This research was funded by Guy’s and St Thomas’ Charity. Jane Sandall and Nicola Mackintosh were funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South London at King’s College Hospital NHS Foundation Trust.

Availability of data and materials

All data generated or analysed during this study are included in this published article (and its supplementary information files).

Authors’ contributions

NM conceived of the study; participated in its design, coordination, and analysis; and drafted the manuscript. MT, RM, and JS participated in its design and analysis and helped draft the manuscript. AX, KF, JK, EL, AM, JS, SN, and JHM participated in the coordination of the study and searching and data extraction and helped draft the manuscript. All the authors read and approved the final manuscript.

Authors’ information

The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Nicola Mackintosh.


Table 5 Characteristics of excluded studies

Additional file

Additional file 1:

PRISMA 2009 Checklist. (DOCX 63.5 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mackintosh, N., Terblanche, M., Maharaj, R. et al. Telemedicine with clinical decision support for critical care: a systematic review. Syst Rev 5, 176 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: