Patient prioritization tools and their effectiveness in non-emergency healthcare services: a systematic review protocol
Systematic Reviews volume 8, Article number: 78 (2019)
Waiting lists should be managed as fairly as possible to ensure that patients with greater or more urgent needs receive services first. Patient prioritization refers to the process of ranking referrals in a certain order based on various criteria with the aim of improving fairness and equity in the delivery of care. Despite the widespread use of patient prioritization tools (PPTs) in healthcare services, the existing literature on this subject has mainly focused on emergency settings. Evidence has not been synthesized with respect to all the non-emergency services.
This review aims to perform a systematic synthesis of published evidence concerning (1) prioritization tools’ characteristics, (2) their metrological properties, and (3) their effect measures across non-emergency services. Five electronic databases will be searched (Cochrane Library, Ovid/MEDLINE, Embase, Web of Science, and CINAHL). Eligibility criteria guiding data selection will be (1) qualitative, quantitative, or mixed methods empirical studies; (2) patient prioritization in any non-emergency setting; and (3) discussing characteristic, metrological properties, or effect measures. Data will be sought to report tool’s format, description, population, setting, purpose, criteria, developer, metrological properties, and outcome measures. Two reviewers will independently screen, select, and extract data. Data will be synthesized with sequential exploratory design method. We will use the Mixed Methods Appraisal Tool (MMAT) to assess the quality of articles included in the review.
This systematic review will provide much-needed knowledge regarding patient prioritization tools. The results will benefit clinicians, decision-makers, and researchers by giving them a better understanding of the methods used to prioritize patients in clinical settings.
Systematic review registration
In many healthcare settings, waiting lists help streamline the uncertain flow of requests and providers’ workload. Their presence in healthcare services is of paramount importance in order to rationalize capacity and hence costs . However, as the demand for services exceeds supply, these lists can grow quickly and result in unacceptable wait times. Patients from countries with publicly funded healthcare systems frequently experience excessive wait times  that have significant consequences, such as a negative impact on pain and function and deterioration in the quality of life [3, 4]. Reducing wait times in order to improve access to healthcare services has become one of the main concerns of modern healthcare systems. Many strategies can be used to manage waiting lists with the aim of improving patient flow and reducing wait times. Examples of such approaches include increasing capacity (e.g., hiring more staff or purchasing more equipment), setting maximum wait time targets, and reorganizing patterns of services (e.g., introducing methods to reduce missed appointments) [1, 5]. However, wait times for non-urgent health services such as elective surgery or the prescription of wheelchairs is often unavoidable . Therefore, waiting lists should be managed as fairly as possible to ensure that patients with greater or more urgent needs receive services ahead of those with less need and that patients with approximately the same degree of need wait about the same length of time . Here, the concept of need goes beyond the medical perspective and incorporates wider social and environmental determinants associated with a sustainable vision of health . Need in healthcare is commonly defined as the capacity to benefit. So, if health needs are present, then an effective intervention should be available to meet these needs and improve health .
One strategy to achieve fairness in managing waiting lists is to use patient prioritization methods. Patient prioritization refers to the process of ranking referrals in a certain order based on various criteria . However, there are conflicting evidence regarding their effect on wait times . Since the aim is for patients with the greatest needs to be treated first, patients’ wait times should be determined objectively on the basis of explicit and equitable criteria .
One challenge associated with patient prioritization is to compare patients’ priority dynamically based on their conditions and the amount of benefit expected from the services they are waiting for . Thus, assessing patients’ priority tends to be inconsistent in practice and assessors do not agree on criteria [11, 12]. The lack of systematic and standardized practices for patient prioritization can have an impact on healthcare accessibility. For example, it has been found that “low priority” patients, such as those with chronic conditions, do not receive publicly accessible healthcare services in a timely manner because their condition is considered less urgent than others [13,14,15].
Several studies have investigated tools that could provide an explicit, transparent, and fair prioritization of patients on waiting lists. These tools focus on priority decisions in elective (i.e., medically necessary but non-emergency) settings where there is less likelihood of dealing with critical or life-threatening situations, as opposed to emergency services . Instead, decisions may need to consider a range of factors with various degrees of importance for the main stakeholders (healthcare organizations, professionals, and patients and their families) [17,18,19]. These factors include pain, loss of function, risk of deterioration, impact of a disability on work or social roles, patient’s socioeconomic status, dependence on others, and economic impacts. Since many of these factors can be difficult to evaluate, patient prioritization is a complex task .
The purpose of patient prioritization tools (PPTs) is to help clinicians and decision-makers manage their programs or their institution’s waiting lists. These tools mainly comprise sets of general criteria that encompass personal factors (e.g., age), social factors (e.g., ability to work) , and any other factor deemed relevant (e.g., patient’s quality of life) [7, 21]. In addition, some tools are using specific criteria, which are more objective and measurable because they relate to disease-specific outcomes (e.g., visual acuity for cataract surgery) . Most tools are designed for elective interventions, including cataract surgery , hip and knee arthroplasty [23, 24], and general surgery . Similar tools have also been developed for prioritization in other non-emergency services, such as magnetic resonance imaging , children’s mental health services , and rehabilitation programs [5, 28].
PPTs can take different forms, including less formalized methods using a two- to four-level classification system (“high priority” and “low priority”), or more formal tools, which are most often priority scoring systems that assign a score to the needs of each patient . There are some systems that prioritize informally based on clinical judgment, without using an explicit written tool . In a recent study conducted in outpatient physiotherapy departments, stakeholders noted that the criteria used to prioritize patients were often vague and expressed a need for more explicit and objective criteria . In-house prioritization tools were mostly developed through consultations and discussions with stakeholders in the program . In contrast, several studies [22, 23, 25,26,27] present an iterative approach applied in different contexts to develop priority criteria based on a literature review and consultations with experts.
Many studies have tried to demonstrate the metrological properties of PPTs in healthcare services, such as reliability and validity [9, 11, 12, 22, 23, 25,26,27, 30]. In patient priority assessments, reliability refers to the interrater agreement (between two assessors) and intrarater stability (test-retest). In different contexts, studies have shown only poor to good interrater and test-retest reliability [22, 23, 25,26,27]. This means that independent assessors are likely to make different decisions regarding the rank of the same patient on the waiting list, and the same assessor may even make different decisions over time.
Validity refers to the extent to which an assessment measures what it is intended to measure and whether useful inferences can be made based on that measurement . In the case of PPTs, a good demonstration of validity refers to patients who are rated as “high priority” and actually have the most needs [11, 31]. Validating PPTs is complicated by the lack of a gold standard with which to compare them [9, 11, 31]. On the other hand, construct validity can be tested by comparing them to other measurements which should measure similar dimensions in a consistent way . For example, Escobar et al. correlated scores between a prioritization tool and Western Ontario McMaster Osteoarthritis Index (WOMAC) questionnaire dimensions (quality of life measurement) to determine its validity . They found that those with poorer WOMAC scores have a tendency to score higher on this prioritization tool, which supports the validity of this tool to be used with patients waiting for hip or knee replacement .
PPTs have already been the subject of two systematic reviews, but authors limited their search to allied health services  or elective surgeries . MacCormick et al. reviewed evidence published before 2001 about prioritization for elective surgery by focusing on different ethical-based prioritization, what different criteria are in use, how they were derived, and what different tools of summation are in use . Harding et al., meanwhile, have sought to determine the properties of triage systems for allied health services, including emergency setting, and the health services’ outcomes . They tried to synthesize evidence of outcome measures related to patient flow (e.g., waiting times, effect on waiting lists, and length of stay) but found that PPTs’ effectiveness is challenging to prove . In addition to limited evidence, studies in this review were mostly conducted in emergency settings. Measuring outcomes of a PPT needs to be documented in a variety of settings and to take different perspectives into account, including those of the professionals, the system, and the service users .
Despite the widespread use of PPTs in non-emergency healthcare services, the existing literature on this subject has not been synthesized. Indeed, there is little evidence regarding the current status of PPTs in healthcare systems. It is crucial to build on a synthesis of the properties of PPTs across healthcare services in order to have a better understanding of this innovation that aims to optimize service efficiency and access to patient care. This review aims to systematically synthesize the published evidence concerning PPTs across non-emergency settings. Our objectives are to (1) describe PPTs’ characteristics, such as format, description, population, setting, purpose, criteria, and developer; (2) identify their metrological properties, such as reliability and validity; and (3) point out their effect or outcome measures (e.g., decrease of wait times). The research question of this systematic review is: “What is the status of PPTs in healthcare systems in terms of characteristics, metrological properties and effect measures?”
A mixed studies review will be carried out to have a better understanding of this complex intervention . This review follows stages proposed by Pluye and Hong , guiding systematic reviews and types of mixed studies synthesis designs. The present protocol has been registered within the PROSPERO database (registration number: CRD42018107205) and is being reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) statement [33, 34] (see PRISMA-P checklist in Additional file 1).
The research question was formulated using the ECLIPSE model (Expectation, Client Group, Location, Impact, Professionals, Service) to guide the systematic review, which is generally used in systematic reviews in the field of health and social services management . It offers an alternative to the PICO framework that is mostly used in systematic reviews of clinical interventions but is less applicable in the domain of management. Two of the authors (JD and MEL) will determine and validate the keywords and inclusion/exclusion criteria guiding the systematic review in accordance with its objectives. They will evaluate the quality of the search strategy using the Peer Review of Electronic Search Strategies Evidence-Based Checklist (PRESS EBC) criteria [36, 37].
A professional librarian will initiate the database research using free-text terms, Boolean operators, and truncation. An example of the search strategy developed for MEDLINE/Ovid is provided in the Appendix (Table 1). This search strategy will be adapted for each database.
We will search in five electronic databases (Cochrane Library, Ovid Medline, Embase, Web of Science, and CINAHL) using the search strategy developed by a team of researchers and a professional librarian. Then, to identify additional sources, a secondary four-step search will be conducted: (1) screening of the reference lists of the identified articles; (2) citation searches performed using Google Scholar for records that meet the inclusion criteria; (3) screening of 25 similar references suggested by the databases, where available; and (4) contact authors who have published two articles or more that are included in our review. Data will not be limited by timeframe, and we will consider all studies published before September 2018.
The inclusion criteria for the selection of articles in the literature review will be (1) peer-reviewed quantitative/qualitative/mixed methods empirical studies, which includes all qualitative research methods (i.e., phenomenological, ethnographic, grounded theory, case study, and narrative) and quantitative research designs (i.e., randomized controlled studies, cohort studies, case-control, cross-sectional, descriptive, and longitudinal); (2) published in English or French before September 2018; (3) reporting the use of a PPT in a non-emergency healthcare setting; and (4) discussing characteristics, metrological properties, or effect measures of a PPT.
The following types of references will be excluded from the literature review: (1) studies focusing on strategies/methods for managing waiting lists without using a prioritization tool or system, (2) studies conducted in an emergency setting, (3) articles about healthcare systems dealing with critical or life-threatening situations (e.g., organ transplants), and (4) literature reviews.
Two reviewers (JD and SD) will import all records from the database in a reference management software (Endnote). Duplicates will be removed using the software command “Find duplicates” based on the title of the references. The remaining duplicates will be deleted during the abstract/title screening process.
Screening and selection process
The same two reviewers will independently screen all articles identified from the search. First, titles and abstracts of articles returned from initial searches will be screened based on the eligibility criteria outlined above. At this point, we estimate to select 50 to 75 potentially relevant records. Second, full texts will be examined in detail and screened for eligibility. We anticipate to find 10 to 20 relevant studies meeting all eligibility criteria. Third, references of all considered articles will be hand-searched to identify any relevant report missed in the search strategy by two reviewers independently. Any disagreement between reviewers will be resolved by discussion to meet a consensus. In case of disagreements concerning the selection of articles, a third reviewer (MEL) will evaluate the eligibility of the studies concerned. We will measure the agreement between reviewers with the Kappa coefficient.
Independently, two reviewers (JD and SD) will review all articles in the final list using an extraction grid. The extraction grid will include items such as information relating to the authors, title of publication, year of publication, level of evidence, and information about the PPT. Data will be sought to itemize format, description, population, setting, purpose, criteria, developer, metrological properties, and outcome measures. All these variables have been used to report data in former systematic reviews and will help to meet our review objectives as well. Additional information including authors’ key conclusions and other relevant findings will also be extracted.
A sequential exploratory synthesis design method will be used to describe the results of the systematic mixed studies review . This synthesis method is performed in two steps, where in phase one, qualitative data from qualitative, quantitative, and mixed studies are transformed into qualitative findings (i.e., a taxonomy of study results) using qualitative thematic analysis, for instance. At this step, we will follow an inductive thematic analysis of data and a computer-assisted qualitative data analysis software could be used (i.e., NVivo). In phase two, quantitative data from quantitative studies and mixed studies are tabulated and compared when there is a common variable across studies (i.e., a common type of study results) . This sequential exploratory synthesis method is appropriate and complete in itself because we can use qualitative studies to interpret, explain, or provide more insight to some quantitative findings .
Two independent assessors (JD and SD) will assess the methodological quality of the studies. Pluye et al.’s MMAT for appraising the methodological quality of studies with diverse designs  will be used to evaluate the quality of the articles. This tool is appropriate for our review because it contains specific criteria to assess the methodological quality of mixed methods studies. It allows the use of only one tool for concomitantly appraising all types of empirical studies, whether quantitative or qualitative . As proposed, we will discuss the strength of the body of evidence by a detailed presentation of the ratings of each criterion to better inform the quality of the included studies .
In a world where the demand for efficient and equitable health services is growing substantially, it is of utmost importance to optimize healthcare service resources and waiting list management . This systematic review will provide much-needed knowledge regarding PPTs. The results will benefit clinicians, decision-makers, and researchers by giving them a better understanding of the methods used to prioritize patients in clinical settings.
To our knowledge, there are no guidelines about tools that help clinicians and healthcare managers to make decisions about patient prioritization in non-emergency settings. This study will lead to a synthesis of these practices and a better understanding of the various prioritization methods used in healthcare organizations. A common vocabulary and clear guidelines are needed to show characteristics of PPTs, how they are developed, and how to measure their effects. Ultimately, this knowledge synthesis is expected to benefit a large population because this knowledge could be disseminated across different organizations. Knowledge translation activities could inform stakeholders about PPTs and how they could be adapted for their programs or services. We could, for example, organize workshops with decision-makers and clinicians aiming to formalize their prioritization practice according to relevant findings from this literature review.
We anticipate potential limitations in this study. First, the concept of “tool” is not well defined in the literature, which could make it difficult to identify all relevant studies using a PPT. Second, prioritization is a complex intervention mixing a wide variety of domains, such as engineering, public health, and management, so it would be challenging to synthesize data from all these fields. Third, triage and prioritization are often used as interchangeable terms, which could compromise the relevance of studies included in the review, because we want to focus precisely on PPTs. All important protocol amendments will be documented in a logbook and they will be clearly mentioned in the method section of the systematic review publication.
Cumulative Index to Nursing and Allied Health Literature
Patient prioritization tool
- PRESS EBC:
Peer Review of Electronic Search Strategies Evidence-Based Checklist
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Greater Toronto Area Rehabilitation Network. Measuring and managing supply and demand: a waiting list information management proposal for musculoskeletal rehabilitation in the Greater Toronto Area. Toronto: The Network; 2003.
Hurst J, Siciliani L. Tackling excessive waiting times for elective surgery: a comparative analysis of policies in 12 OECD countries. Health Policy. 2003;72(2):201–15.
Ackerman IN, Bennell KL, Osborne RH. Decline in health-related quality of life reported by more than half of those waiting for joint replacement surgery: a prospective cohort study. BMC Musculoskelet Disord. 2011;12(1):108.
Desmeules F, Dionne CE, Belzile E, Bourbonnais R, Frémont P. The burden of wait for knee replacement surgery: effects on pain, function and health-related quality of life at the time of surgery. Rheumatol. 2010;49(5):945–54.
Passalent LA, Landry MD, Cott CA. Exploring wait list prioritization and management strategies for publicly funded ambulatory rehabilitation services in Ontario, Canada: further evidence of barriers to access for people with chronic disease. Healthc Policy. 2010;5(4):e139–56.
Cerdá E, de Pablos L, Rodriguez MV. Waiting lists for surgery. In: Hall R, editor. Patient flow: reducing delay in healthcare delivery. Boston: Springer; 2013. p. 151–87.
Hadorn DC. Steering Committee of the Western Canada Waiting List Project. Setting priorities for waiting lists: defining our terms. CMAJ. 2000;163(7):857–60.
Wright J, Williams R, Wilkinson JR. Health needs assessment: development and importance of health needs assessment. BMJ. 1998;316(7140):1310–3.
Noseworthy T, McGurran J, Hadorn D. Steering Committee of the Western Canada Waiting List Project. Waiting for scheduled services in Canada: development of priority-setting scoring systems. J Eval Clin Pract. 2003;9(1):23–31.
Harding KE, Taylor NF, Leggat SG. Do triage systems in healthcare improve patient flow? A systematic review of the literature. Aust Health Rev. 2011;35(3):371–83.
Harding KE, Taylor NF, Shaw-Stuart L. Triaging patients for allied health services: a systematic review of the literature. BJOT. 2009;72(4):153–62.
Harding KE, Taylor NF, Leggat SG, Wise WL. Prioritizing patients for community rehabilitation services: do clinicians agree on triage decisions? Clin Rehabil. 2010;24(10):928–34.
Delaurier A, Bernatsky S, Raymond MH, Feldman DE. Wait times for physical and occupational therapy in the public system for people with arthritis in Quebec. Physiother Can. 2013 Summer;65(3):238–43.
Passalent LA, Landry MD, Cott CA. Wait times for publicly funded outpatient and community physiotherapy and occupational therapy services: implications for the increasing number of persons with chronic conditions in Ontario, Canada. Physiother Can. 2009 Winter;61(1):5–14.
Raymond MH, Demers L, Feldman DE. Waiting list management practices for home-care occupational therapy in the province of Quebec, Canada. Health Soc Care Community. 2016;24(2):154–64.
Sobolev B, Levy A, Kuramoto L. Access to surgery and medical consequences of delays. In: Hall R, editor. Patient flow: reducing delay in healthcare delivery. Boston: Springer; 2013. p. 79–100.
Sampietro-Colom L, Espallargues M, Rodríguez E, Comas M, Alonso J, Castells X, et al. Wide social participation in prioritizing patients on waiting lists for joint replacement: a conjoint analysis. Med Dec Making. 2008;28(4):554–66.
McKie J, Shrimpton B, Hurworth R, Bell C, Richardson J. Who should be involved in health care decision making? A qualitative study. Heal Care Anal. 2008;16(2):114–26.
Skirbekk H, Hem MH, Nortvedt P. Prioritising patient care: the different views of clinicians and managers. Nurs Ethics. 2018;25(6):746–59.
Harding KE, Taylor NF. Triage in nonemergency services. In: Hall R, editor. Patient flow: reducing delay in healthcare delivery. Boston: Springer; 2013. p. 229–50.
MacCormick AD, Collecutt WG, Parry BR. Prioritizing patients for elective surgery: a systematic review. ANZ J Surg. 2003;73(8):633–42.
Romanchuk KG, Sanmugasunderam S, Hadorn DC. Steering Committee of the Western Canada Waiting List Project. Developing cataract surgery priority criteria: results from the Western Canada Waiting List Project. Can J Ophthalmol. 2002;37(3):145–54.
Arnett G, Hadorn DC. Steering Committee of the Western Canada Waiting List Project. Developing priority criteria for hip and knee replacement: results from the Western Canada Waiting List Project. Can J Surg. 2003;46(4):290–6.
Escobar A, Quintana JM, Gonzalez M, Bilbao IB. Waiting list management: priority criteria or first-in first-out? A case for total joint replacement. J Eval Clin Pract. 2009;15(4):595–601.
Taylor MC, Hadorn DC. Steering Committee of the Western Canada Waiting List Project. Developing priority criteria for general surgery: results from the Western Canada Waiting List Project. Can J Surg. 2002;45(5):351–7.
Hadorn DC. Steering Committee of the Western Canada Waiting List Project. Developing priority criteria for magnetic resonance imaging: results from the Western Canada Waiting List Project. Can Assoc Radiol J. 2002;53(4):210–8.
Smith DH, Hadorn DC. Steering Committee of the Western Canada Waiting List Project. Lining up for children's mental health services: a tool for prioritizing waiting lists. J Am Acad Child Adolesc Psychiatry. 2002;41(4):367–76.
Raymond M-H, Feldman D, Prud'homme M-P, Demers L. Who’s next? Referral prioritisation criteria for home care occupational therapy. Int J Ther Rehabil. 2013;20(12):580–9.
Laliberté M, Feldman DE, Williams-Jones B, Hunt M. Operationalizing wait lists: strategies and experiences in three hospital outpatient physiotherapy departments in Montreal. Physiother Theory Pract. 2018;34(11):872–81.
Coleman B, McChesney D, Twaddle B. Does the priority scoring system for joint replacement really identify those in most need? NZMJ. 2005;118(1215).
Escobar A, Quintana JM, Bilbao A, Ibanez B. Validation of a prioritization tool for patients on the waiting list for total hip and knee replacements. J Eval Clin Pract. 2009;15(1):97–102.
Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annual rev pub health. 2014;35:29–45.
Gedda M. Traduction française des lignes directrices PRISMA pour l’écriture et la lecture des revues systématiques et des méta-analyses / French translation of the PRISMA Reporting Guidelines for writing and reading systematic reviews and meta-analyses. Kinésithérapie, la Revue. 2015;15(157):39–44.
Moher DS, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1).
Wildridge V, Bell L. How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Health Inf Libr J. 2002;19(2):113–5.
McGowan J, Sampson M, Lefebvre C. An evidence based checklist for the peer review of electronic search strategies (PRESS EBC). EBLIP. 2010;5(1).
Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.
Hong QN, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst rev. 2017;6(1):61.
Pluye P, Robert E, Cargo M, Bartlett G, O’cathain A, Griffiths F, et al. Proposal: a mixed methods appraisal tool for systematic mixed studies reviews, vol. 2. Montréal: McGill University; 2011. p. 1–8.
Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, Gagnon M-P, Griffiths F, Nicolau B, O’Cathain A, Rousseau M-C, Vedel I. Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552). In: Canadian Intellectual Property Office, Industry Canada.
The authors would like to thank Marie Denise Lavoie for helping to design the search strategy for this review. They also want to thank Bernadette Wilson and Valerie Boucher for language corrections of the text.
This study was funded by Fonds de recherche du Québec Nature et Technologie. JD received a study grant from the Centre interdisciplinaire de recherche en réadaptation et intégration sociale (CIRRIS) to conduct this review. FR and MEL are both Fonds de recherche du Québec Santé Research Scholar.
Availability of data and materials
All data generated or analyzed during this study will be included in the published systematic review article.
Ethics approval and consent to participate
Ethical approval is not required for this study as it is a systematic review.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Déry, J., Ruiz, A., Routhier, F. et al. Patient prioritization tools and their effectiveness in non-emergency healthcare services: a systematic review protocol. Syst Rev 8, 78 (2019). https://doi.org/10.1186/s13643-019-0992-x
- Patient prioritization
- Systematic review
- Healthcare services
- Waiting lists