Interventions to improve emergency department use for mental health reasons: protocol for a mixed-methods systematic review

Abstract

Background

Healthcare resources are limited and unnecessary, and inappropriate emergency department use is now a highly visible healthcare priority. Individuals visiting the emergency department for mental health-related reasons are often amongst the most frequent presenters. In response, researchers and clinicians have created interventions to streamline emergency department use and several primary studies describe the effects of these interventions. Yet, no consensus exists on the optimal approach, and information on the quality of development, effectiveness, acceptability, and economic considerations is hard to find. The purpose of this study is to systematically review interventions designed to improve appropriate use of the emergency department for mental health reasons.

Method

A mixed-method systematic review using Joanna Briggs Methodology. Search combining electronic databases (EMBASE, MEDLINE, PsycINFO, CINAHL, HealthSTAR, PROQUEST, Cumulative Index to Nursing and Allied Health) and secondary searches (grey literature and hand search with consultation). Two independent reviewers will screen titles and abstracts using predetermined eligibility criteria and a third reviewer will resolve conflicts. Full texts will also be screened by two independent reviews and conflicts resolved in a consensus meeting with a third reviewer. A pilot-tested data extraction form will be used to retrieve data relevant to the study objectives. We will assess the quality and of all included studies. Data describing interventions will be summarized using logic models and reported narratively. Quality of development will be assessed using the Oxford Implementation Index. For data on intervention effectiveness, we will assess statistical heterogeneity and conduct a meta-analysis using a random effects method, if appropriate. For interventions that cannot be pooled, we will report outcomes narratively and descriptively. Qualitative data on acceptability will be synthesized using meta-aggregation and an economic evaluation of interventions will be done. The reporting of this protocol follows the PRISMA-P statement.

Discussion

Using a combined systematic review methodology and integrated knowledge translation plan, the project will provide decision makers with concrete evidence to support the implementation and evaluation of interventions to improve emergency department use for mental health reasons. These interventions reflect widespread priorities in the area of mental health care.

Systematic review registration

PROSPERO CRD42018087430

Background

While emergency departments (ED) offer accessible care 24 h-a-day and act as a portal of entry for patients requiring immediate healthcare services, overcrowding and increasing yearly visit rates are a worldwide problem [1, 2]. Visits for mental health-related reasons are known to significantly contribute to overcrowding, with reports indicating that up to 20% of visits are for mental health needs [3, 4]. Although many mental health issues (such as acute psychosis or mania) require emergency care, other issues, often instigating ED visits, may be better managed in the community through primary care and auxiliary services [5]. Inappropriate and unnecessary use of the ED has negative implications economically, organizationally, and for health professionals providing care. Furthermore, inefficient use of the ED or ED care that does not meet patient needs negatively affects these persons, their health and recovery, as well as their support systems [6].

The widespread recognition of the impact of frequent ED use by people with mental health issues has led to the publication of several primary studies and one systematic review [7]. This review explored the type and effectiveness of interventions designed to reduce ED visits. The review included 11 studies (1985–2009), and the authors concluded that interventions using a case management approach reduced cost and improved outcomes, though no meta-analysis was done. Unfortunately, this review does not provide adequate information to inform ED practice for patients with mental illness, because population-specific interventions—including those designed specifically for persons with mental health needs—were excluded. Furthermore, only quantitative studies were included and thus patient and provider perspectives of the interventions were not examined. In preparation for this protocol, we attempted to replicate and update the Althaus search to ascertain how many mental health-related interventions were either missed or published after the search date. In screening the initial 100 hits, we identified 12 studies describing such interventions (e.g., [8]). Clearly, a significant body of evidence exists that contains detailed information about the needs of persons who visit the ED for mental health-related reasons.

In the current healthcare context where financial and human resources are limited, the burden of unnecessary and inappropriate emergency department (ED) use is apparent. This has implications at all levels including patients, healthcare professionals, and policymakers. In response, researchers and clinicians have attempted to develop interventions to streamline ED use. Several primary studies describe the effects of these interventions, and yet no consensus exists on the optimal approach, including quality of development, effectiveness, acceptability, and economic considerations. The aim of this study is to conduct a systematic review of interventions designed to improve appropriate use of the ED for mental health (including substance use)-related reasons and to assess the relevance and feasibility of implementing these interventions.

Specifically, the study objectives are:

• 1. To describe interventions designed to improve the appropriate use of the ED for mental health-related reasons

• 2. To report on the quality of development, effectiveness, acceptability, and economic considerations of the identified interventions

Methods

Design

Our approach to this study will be a systematic review using the Joanna Briggs Methodology for Mixed Methods Systematic Reviews [9]. The review protocol was developed according to the PRISMA-P Statement [10] and registered with PROSPERO (CRD42018087430).

Eligibility criteria

Eligibility criteria were created using PICO (Population, Intervention, Comparison, Outcome), as specified in the Joanna Briggs Institute (JBI) methodology, and include the following:

Population: Adults (persons aged 17 years or older) who visit the ED for mental health-related reasons, excluding child, adolescent, or cognitive impairment-only participants because healthcare and services for these populations are delivered differently.

Intervention: All forms of interventions about the use of the ED for mental health reasons and/or any intervention in which ED use related to a mental health reason is a measured outcome. An intervention is defined as “an act performed for, with or on behalf of a person or population whose purpose is to assess, improve, maintain, promote or modify health, functioning or health conditions” (World Health Organization, 2019). For example, intensive case management and telephone follow-up post-discharge, excluding the following: any intervention with a different purpose or articles reporting of system-level reform, such as deinstitutionalization or Medicare.

Comparison: Other treatment intervention or normal care.

Outcome: Quality of development, effectiveness, acceptability, and economic considerations for interventions designed to improve ED use for mental health-related reason.

Types of studies: To determine the effectiveness (or report on this outcome narratively if required) of the interventions, we will search for the following: randomized controlled trials, quasi-experimental studies, cohort studies, case-control studies, case series, and cross-sectional studies. To better understand how people perceive and experience the interventions (i.e., acceptability), we will search for the following: process evaluations and qualitative studies. To evaluate the economic considerations of the interventions, we will search for the following: reports on economic evaluations of all types, excluding the following: all non-research literature (e.g., expert opinion, discussion articles).

Language: English, French, and any other language able to be translated into English or French.

Country: No limits.

Search and selection of articles

Our search strategy combines systematic searches of multiple electronic databases with secondary searches:

• 1. Formal search of online databases: MEDLINE, CINAHL, PsycINFO, EMBASE, Nursing and Allied Health, HealthSTAR, and PROQUEST

• 2. Secondary searches: grey literature search, hand search, and consultation (described below)

Online database search strategy

In consultation with our library scientist, we designed and executed a pilot search in the OVID MEDLINE database. This strategy included keywords and MESH headings based on our eligibility criteria and was used to gauge the likely number of studies eligible for inclusion. Twelve hundred and thirty-six MEDLINE citations were retrieved, of which approximately 135 passed first-level screening. From this, the library scientist finalized the search strategy and translated into all databases (Table 1).

Table 1 Search strategy

Literature will be searched from 1990 onwards. During the 1990s, in the aftermath of the deinstitutionalization movement, rates of ED use for mental health-related reasons skyrocketed with reports indicating nearly a 40% increase in visits for mental health reasons compared to an 8% increase in overall ED use [11]. No other limits were placed on the searches. The completed search was executed on October 3, 2017, in all databases. A secondary search designed to identify newly published articles will occur just prior to completion of the review. Our library scientist will lead this step.

Grey literature search strategy

We will work closely with our library scientist to design a rigorous grey literature search strategy. This will use a subject-based approach [12] and consider all possible sources of information, such as organizational websites, blogs, conference proceedings, annual reports, and others. A preliminary scan of the online grey literature and discussion with our KUs revealed a substantial body of grey literature eligible for inclusion. For example, the CADTH report on Prioritization of Care in the Emergency Department using Alternate Triage Strategies: Effectiveness and Guidelines [13] is one such publication.

Hand search and consultation process

As a final step in our search strategy, we will conduct a hand search of references lists for all included studies and relevant (but not included) reviews, reference lists of studies citing the included studies, content from selected journals, and consult regularly with KUs to ensure that all eligible and pertinent information is considered.

Audit Trail

We have a detailed record of all search strategy procedures to ensure a transparent and replicable process. This includes the following: databases searched, subject headings, and keywords used for each database. We will maintain a similar audit trail for the grey literature and hand search sources. We will report the screening and selection process using the PRISMA flow chart.

Selection process

Two reviewers will independently engage in the study selection process to reduce the possibility of rejecting relevant reports [10]. We will use Covidence©, an online citation screening tool, to facilitate and monitor study selection. This software program allows for complete screening of citations, clearly displays differences in reviewers’ ratings and will help mitigate any issues pertaining to inter-rater reliability.

After removing duplicate citations, we will select studies for inclusion based on a two-level screening process. First, two reviewers will independently screen titles and abstracts for congruence with eligibility criteria (first-level screening). At this stage, we will compare results using Covidence© data and calculate a kappa statistic to assess inter-rater reliability. A meeting will be held to discuss all instances of discrepancy, and all potentially relevant citations and those with insufficient information to determine eligibility will be retained. Second, two reviewers will independently screen the full texts of the retained articles for congruence with eligibility criteria (second-level screening). Using Covidence©, the two reviewers will document the reasons for excluding citations. A final consensus meeting will be held to re-assess inter-rater reliability and to agree upon the final set of eligible citations.

Quality appraisal of included studies

.Two reviewers will independently assess the quality of each included study using the Joanna Briggs Critical Appraisal Tools [14]. For quantitative studies, these will include the Checklist for Randomized Controlled Trials, Checklist for Quasi-Experimental Studies, Checklist for Cohort Studies, Checklist for Case Control Studies, Checklist for Case Series, and the Checklist for Analytical Cross Sectional Studies. For qualitative studies, we will use the Checklist for Qualitative Research, and for reports of economic evaluations, we will use the Checklist for Economic Evaluations. We will report on the complete critical appraisal results to provide recommendations to advance research in this area and conduct a sensitivity analysis to evaluate the effect of including and excluding low-quality studies on the results, if appropriate. The project lead will oversee this process and consult, when necessary, to settle discrepancies in ratings.

Data extraction process

We will develop a fillable form in Microsoft Word for data extraction, and pilot test our template for inter-rater reliability using the first three included articles. Once finalized, two reviewers will independently extract data from each included article using this template. The data extracted will include information on the study characteristics, participant characteristics, intervention characteristics using the Template for Intervention Description and Replication (TIDieR) Checklist [15], quality of development indicators, information on cost and other economic considerations, and outcomes (effect on ED use and other author-identified outcomes). When sufficient information is not clearly reported in the published studies, we will search for secondary publications and/or contact the authors for details.

Data synthesis

To address objective 1 (describe interventions designed to improve appropriate use of the ED for mental health reasons), we will aggregate multiple reports on individual interventions so that each intervention (rather than each study) is a unit of interest in the review.

Using synthesis tables and narrative summaries, we will report on the intervention characteristics, benefits, challenges, and implications for use. We will also draft a logic model [16] for each intervention based on this information. A logic model is a graphic representation (a map) that facilitates the understanding of how an intervention works, by illustrating the relationships between components and the mechanisms of action (i.e., resources/inputs, activities, outputs, outcomes, impact). Logic models facilitate the transparency of systematic review findings and enhance their interpretability for decision makers [17].

To address objective 2, we will report on the quality of development, effectiveness, acceptability, and economic considerations of each intervention. We will again aggregate multiple reports on individual interventions so that each intervention (rather than each study) is a unit of interest in the review.

Quality of development

Currently, there is no accepted process to assess the quality of complex interventions. For this study, we propose the use of the Oxford Implementation Index (OII) [18]. The OII is a tool designed to capture information in four domains: (1) intervention design, (2) actual delivery by trial practitioners, (3) uptake of the intervention by participants, and (4) contextual factors. We have obtained permission for its use in this study.

Effectiveness

If we obtain a series of interventions and outcomes amenable to meta-analysis [19], we will assess statistical heterogeneity and pool the results using a random effects method. We will report on the relative risk (with 95% confidence interval), absolute risk reduction, and the number needed to treat for all types of interventions amenable to meta-analysis. For interventions that cannot be pooled, we will report quantitative outcomes narratively and descriptively.

Acceptability

We will use data from qualitative studies to assess the acceptability of the interventions (i.e., the perceptions and experiences of patients and practitioners regarding the intervention). This information will be analyzed according to the JBI meta-aggregation approach and be reported narratively and descriptively.

Economic evaluation

We will conduct an economic evaluation for all interventions when information is available in published articles or through contacting primary authors. Based on our knowledge of this area, it is likely that we will obtain incomplete economic data for most interventions. As such, we will use a narrative and descriptive approach to synthesize information, led by Dr. Kronick (Economics). Further, we will provide recommendations for improving the reporting of this information and discuss the potential costs of the identified interventions with our knowledge users.

Discussion

Decreasing inappropriate and unnecessary ED use is now a highly visible healthcare priority, with nationwide attention in the area of mental health care. This review will provide decision makers with concrete evidence to support the implementation and evaluation of interventions to improve emergency department use for mental health reasons. Outputs will include a catalog of effective interventions, information on the quality of development, economic considerations, and their relevance and feasibility of implementation.

Potential limitations

Two possible limitations, which we are likely to encounter, include (1) heterogeneity of interventions eligible for inclusion and incomplete information reported about the interventions in the literature. Recognizing these possibilities, which will limit our ability to statistically compare the effectiveness of interventions, we designed our data analysis strategy to allow for narrative summary of available information and plan to request additional information, if available, from primary authors. This approach was determined in collaboration with our knowledge users. Should we encounter a situation where we need to deviate from our protocol, we will document and report the change in the completed review [20].