We prepared the systematic review protocol using guidance from the Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols (PRISMA-P) . Our protocol has been registered in the PROSPERO database (CRD42013004519; available at: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42013004519#.UcroqDuyD_M).
The study designs included in this review will be randomized controlled trials (RCT), controlled and uncontrolled time series analyses, and other quasi-experimental designs including pretest/post-test studies. Studies will be included if the intervention described is either an EMS or VMS. Studies comparing EMS with VMS, or comparing different types of EMS/VMS, will only be included if they also include a comparison to a control group, which could include a parallel area not using EMS/VMS (for example, control arm of an RCT with multiple interventions) or baseline data obtained prior to implementing EMS/VMS (for example, baseline data from a time series analysis or pretest/post-test study). To qualify as an EMS, the described system must be able to perform at least one of the following functions: (1) count the number of hand hygiene events within a given geographic hospital area (for example, patient room, ward); (2) count the number of hand hygiene events associated with room entry and/or room exit events; (3) count the number of hand hygiene events associated with specific HCWs; or (4) estimate hand hygiene compliance using data from recorded hand hygiene events linked to HCW movement. To qualify as a VMS, the system must involve the recording of video within a patient care area with evaluation of the video by an internal or external observer using a specified definition of hand hygiene compliance. For both EMS and VMS, studies will be included regardless of whether they are used to provide feedback or real-time reminders to HCWs. However, studies will be excluded if the EMS or VMS is installed covertly, HCWs are not aware that they are being monitored, or the purpose of monitoring was exclusively to evaluate the impact of another intervention not related to the monitoring system and does not assess the impact of the monitoring system itself. Studies involving any HCW population will be included, whether in acute care or long-term care.
Studies must include at least one of the outcomes relevant to the objectives of the review. Thus, studies must include as an outcome either directly observed hand hygiene compliance, incidence of AROs or HAIs, or indirect estimates of hand hygiene compliance, which may be obtained from the monitoring system itself, or through estimates of volume of sanitizer or soap consumed, or count data measuring the number of hand hygiene events.
Studies using directly observed hand hygiene compliance as their outcome must use an accepted standard definition (for example, WHO , CDC , or similar). Studies reporting ARO or HAI incidence will be included if they report at least one HAI as an outcome, defined using standard CDC definitions or an accepted modification of CDC definitions (for example, catheter-associated urinary tract infection, hospital-acquired pneumonia, surgical site infection, catheter-associated bloodstream infection) or an ARO defined using standard microbiological definitions (for example, methicillin-resistant Staphylococcus aureus (MRSA), vancomycin-resistant enterococci (VRE)) .
There is a strong rationale for selecting directly observed hand hygiene compliance as the outcome for the primary research question, despite the fact that the ultimate goal of improvement in hand hygiene compliance is to reduce HAI and despite recognized limitations in the quality of directly observed hand hygiene compliance. First, our a priori knowledge of this field suggests that there will be little or no high quality evidence or studies linking EMS/VMS systems with reduction in HAIs and AROs in general and there is already substantive evidence linking improvements in hand hygiene compliance with reductions in HAIs/AROs, making this an important metric in its own right. Second, direct observation is currently considered the gold standard for the assessment of hand hygiene compliance. It allows hand hygiene performed by HCWs to be distinguished from hand hygiene performed by visitors or patients (as opposed to count or volume data) and is measured using standardized methodology across many studies, allowing comparisons between different EMS systems in terms of their impact on compliance. Compliance estimated by EMS systems would not be expected to be comparable between different systems as they typically detect different types of data and use different analytics and algorithms to provide an estimate of hand hygiene compliance.
Despite the above comments, we feel it is important to include studies using only a metric defined by the system itself to allow identification of all available systems that have the potential for improving hand hygiene. We do not want to exclude systems from consideration based on their failure to use direct observation, given that this measure is itself flawed and has been challenged. Thus systems that demonstrate a significant change in HCW behavior (even if not proven to improve directly observed hand hygiene) should be identified and flagged for further study using a more definitive outcome such as HAI incidence or directly observed hand hygiene. These studies would be considered separately, however, and in a qualitative fashion.
Studies included in the analyses to determine the mechanisms by which monitoring technologies lead to change must meet the inclusion criteria as described above. A subset of these studies that report on outcomes over time (hand hygiene compliance and/or HAIs/AROs) will be used to determine whether there may be a Hawthorne effect leading to change. Studies of systems that use real-time reminders, delayed feedback, or no reminders/feedback will be compared to determine if improvements in compliance or ARO/HAI incidence are related primarily to monitoring or result primarily from feedback and/or reminders.
Only published, peer-reviewed studies will be included. Inclusion will not be limited by language. Studies will not be excluded based on poor methodology as long as they meet the minimum requirements for study design described above; study quality will be assessed and taken into account as described below.
The draft eligibility criteria can be found in Appendix 1.
Database searches will be performed in MEDLINE, EMBASE, CINAHL, Web of Science, and the Cochrane Central Register of Controlled Trials (CENTRAL). All databases will be searched from inception until the present. We will conduct a hand search of a set of infection control journals most likely to publish articles on this topic including: Infection Control and Hospital Epidemiology, Journal of Hospital Infection, and The American Journal of Infection Control. A search of the proceedings of the Society of Healthcare Epidemiology of America will be performed. The reference lists of included studies and relevant review articles will also be searched. Finally, Google will be searched to identify unpublished and grey literature that may suggest publication bias.
An experienced information specialist (DL) will conduct comprehensive literature searches. Search strategies will be developed using medical subject headings (MeSH) and text words related to hand hygiene, compliance/monitoring, and electronic systems. The draft search strategy for MEDLINE can be found in Appendix 2.
Study selection process
The titles and abstracts of all retrieved studies will be independently reviewed by two reviewers (JAS and MPM) based on the eligibility criteria described above. The complete article will then be obtained for any study deemed to potentially meet the inclusion criteria by either reviewer. All complete articles will again be reviewed independently by two reviewers (JAS and MPM) and their eligibility determined based on the criteria described above. If a difference of opinion occurs between the two reviewers that cannot be resolved, a third reviewer (MG) will review the article and make a final determination regarding inclusion. The rationale for exclusion will be documented for all excluded articles.
The data abstracted will include the study setting (type of facility, number of rooms or wards, type of wards), study participants (number, professions of HCWs, age, years of experience), study design (RCT or time series or quasi-experimental design; control group, if any; use of cross-over, switching replications, stepped wedge or other quasi-experiment design elements intended to reduce bias; installation period, enrolment period, data collection period), study intervention (type of EMS or VMS, type and frequency of feedback, type and frequency of real-time reminders) and co-interventions (education or training provided, other simultaneous interventions or promotions in addition to the EMS/VMS), study-specific outcomes (for example, directly or indirectly observed hand hygiene compliance, HAIs, AROs, acceptability of technology to staff, use of direct observation, training of direct observers, measurement of inter-observer agreement, definition of hand hygiene compliance, definition and type of HAI, definition and type of ARO).
Data collection process
A data collection form will be drafted, piloted, and modified as necessary. Two reviewers (JAS and MPM) will complete the form independently for each included study. If there are discrepancies that cannot be resolved by discussion, a third reviewer (MG) will decide. Data will be collected using Microsoft Excel.
Risk of bias appraisal
Study quality and risk of bias will be assessed independently by two investigators (JAS and MPM), with disputes resolved by a third reviewer (MG). All identified RCTs, time series analyses, and controlled pretest/post-test studies will be assessed using the Cochrane Effective Practice and Organization of Care Group Risk of Bias Assessment Tool . Because we anticipate that the majority of studies will be uncontrolled pretest/post-test studies, we will also use guidelines developed specifically for the evaluation of these study designs [17, 18].
There are also specific biases that can be anticipated in studies of EMS/VMS. In particular, there is the potential for co-interventions to occur when EMS/VMS are implemented. EMS implementation requires substantial initial training and education of HCWs as well as ongoing training and assistance, which could alter hand hygiene behavior independent of the EMS itself. Ideally, a similar intensity of hand hygiene education should be provided on a control unit. Implementation of EMS systems may also include elements such as portable hand sanitizer dispensers that could affect HCW behavior independent of the effect of monitoring and feedback. Again, in the ideal study these elements would also be provided to HCWs in a control group. We will carefully assess for these potential sources of bias and, where the nature and extent of education and training is uncertain, may contact the study authors for clarification.
In addition to the specific biases that may be present within individual studies, there is a significant risk for publication bias in this area, as data supporting the use of novel technologies are likely of greater interest for publication than negative studies. Additionally, as many systems are developed by industry, financial incentives likely favor the submission of positive rather than negative studies. We will attempt to minimize the risk by including a broad search, including non-English publications, and by searching the grey literature and conference proceedings for unpublished studies. If sufficient, similar studies are available to allow meta-analysis, a funnel plot will be assessed to look for publication bias. We will also assess trial registries to determine whether there are any missing studies, suggesting potential publication bias, and will compare the methods of published studies with their original protocols to assess for outcome reporting bias.
Synthesis of results
The synthesis of results will begin with a descriptive summary of included studies, focusing on study methodology, interventions, participants, and outcomes, as well as a summary table on risk of bias in the included studies.
The effectiveness of the monitoring systems in the included studies will then be summarized as the mean change in outcome (%). The comparative effectiveness results will be presented according to the research questions, with each of the outcomes reported separately (directly observed hand hygiene compliance, HAI/ARO rates, indirect measures of hand hygiene compliance, and hand hygiene compliance as monitored by the system itself).
For this systematic review, we do not anticipate the performance of a formal meta-analysis as we expect that there will be significant heterogeneity in terms of study designs, types of technology implemented, the frequency and nature of feedback, and the types and definitions of outcomes utilized, which would preclude the calculation of summary measures. However if we do identify at least two comparable studies that describe a similar type of monitoring technology and report on the same outcomes, we will calculate summary measures based on the percent change in hand hygiene compliance and/or HAI/ARO rates. The I2 statistic will be used to assess heterogeneity, and meta-analysis will not be performed if I2 is >50%. The possible reasons for heterogeneity will be explored, including study design, HCW population, clinical setting, and features of the technology. If meta-analysis can be performed, the mean difference will be used as the summary statistic since the outcome data are continuous. If there is moderate heterogeneity (I2 of 30% to 50%), a random effects model will be used for meta-analysis .
The next step will be a narrative synthesis based on the Economic and Social Research Council (ESRC) guidance report . This will include proposing a theoretical model of how hand hygiene monitoring technology may improve hand hygiene compliance and in what circumstances, describing patterns of effect size and direction in included studies, exploring factors that might explain differences across studies, and assessing the strength of the evidence. We plan a priori to explore the two possible mechanisms by which EMS/VMS may result in improved hand hygiene compliance, namely an enhanced Hawthorne effect due to the continuous monitoring and the provision of feedback and/or reminders. We will assess whether there is any apparent relationship between the degree of improvement in compliance identified and the duration of data collection to determine whether the initial benefits of EMS/VMS decline over time as HCWs become accustomed to ongoing monitoring and thus the impact of the Hawthorne effect wanes. We will also determine whether there is any relationship between the reported efficacy of each system and their use of feedback (that is, studies without feedback vs. studies with feedback vs. studies providing real-time reminders). Finally, we will also evaluate study quality in relation to the demonstrated efficacy of each system or type of system for each of the primary and secondary outcomes.
We will evaluate the overall quality of evidence using the GRADE approach  if the synthesized data allows - that is, if an effect estimate for any given specific outcome of interest is reported by two or more controlled studies per each between-intervention comparison.