Effectiveness of trauma centers verification: Protocol for a systematic review
Systematic Reviews volume 8, Article number: 292 (2019)
The implementation of trauma systems in many high-income countries over the last 50 years has led to important reductions in injury mortality and disability in many healthcare jurisdictions. Injury organizations including the American College of Surgeons and the Trauma Association of Canada as well as the World Health Organization provide consensus-based recommendations on resources and processes for optimal injury care. Many hospitals treating trauma patients seek verification to demonstrate that they meet these recommendations. This process may be labeled differently across jurisdictions. In Canada for example, it is called accreditation, but it has the same objective and very similar modalities. The objective of the study described in this protocol is to systematically review evidence on the effectiveness of trauma center verification for improving clinical processes and patient outcomes in injury care.
We will perform a systematic review of studies evaluating the association between trauma center verification and hospital mortality (primary outcome), as well as morbidity, resource utilization, and processes of care (secondary outcomes). We will search CINAHL, EMBASE, HealthStar, MEDLINE, and ProQuest databases, as well as key injury organization websites for gray literature. We will assess the methodological quality of studies using the Risk Of Bias In Non-randomized Studies – of Interventions (ROBINS-I) assessment tool. We are planning to conduct a meta-analysis if feasible based on the number of included studies and their heterogeneity. We will evaluate the quality of cumulative evidence and strength of recommendations using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group methodology.
This review will provide a synthesis of the body of evidence on trauma center verification effectiveness. Results could reinforce current verification modalities and may suggest ways to optimize them. Results will be published in a peer-reviewed journal and presented at an international clinical conference.
Systematic review registration
Injuries represent an estimated 15% of the global burden of disease . They are the leading cause of death under 40 years of age in North America [2,3,4]. The implementation of trauma systems in many high-income countries over the last 50 years has led to important reductions in injury mortality, disability, and overall costs in many healthcare jurisdictions [5,6,7]. Many injury organizations, most notably the American College of Surgeons (ACS) , provide consensus-based recommendations on the structure of trauma systems, and there is a growing trend towards verification of hospitals within trauma systems to determine if they meet criteria for optimal care. Trauma center accreditation is a similar process [9, 10]. We will hereafter use the term “verification” to refer both to accreditation and verification.
Generally, hospitals are designated as trauma centers before applying for verification. Trauma center designation is conducted by a regional or provincial health authority at a local or state level. Designation criteria and procedures may vary from state to state and are typically outlined through the legislative or regulatory authority . Designated hospitals may then seek verification with organizations such as the ACS, which assess adherence to recommended care related to resources, commitment, readiness, policies, patient care, and performance improvement .
ACS verification is generally accomplished in two steps: (1) hospitals submit a prereview questionnaire, which allows site reviewers to have a preliminary understanding of the trauma care capabilities, and (2) a peer review team nominated by the College conducts an on-site review of the hospital [10, 12, 13]. During the process, a center is evaluated according to its designation level. Perceived advantages of verification include strengthening partnerships with stakeholders, engagement and commitment, team work, and the identification of improvement opportunities and priorities [8, 14]. However, verification is an expensive and resource-consuming process [15, 16]. A recent study in the Georgia trauma system estimated that the average costs of American College of Surgeon verification readiness (including administrative resources, clinical medical staff, in-house operating room, and education/outreach) for level I and level II trauma centers were $6.8 and $2.3 million respectively . It is essential to know if verification is a good investment of money and resources in terms of improving patient outcomes [16, 18]. We lack a systematic evidence synthesis on the effectiveness of trauma center verification.
This review aims to synthesize evidence on the effectiveness of trauma center verification for improving hospital mortality, morbidity, resource utilization, and processes of care.
The protocol is developed and presented using the structure suggested by Preferred Reporting Items for Systematic review and Meta-Analysis protocols (PRISMA-P)  (Additional file 1). It has been registered in the International Prospective Register of Systematic Reviews (PROSPERO) database, CRD42018107083 . Any important protocol amendments will be reported and justified in the subsequent systematic review manuscript.
Participants and study designs
Our study population consists of injured patients treated at trauma centers. We will include randomized and non-randomized controlled trials, quasi-experimental studies, controlled before-after studies, cross-sectional studies, and prospective or retrospective observational studies. Case reports, case series, and narrative studies which do not provide an estimate of the association between verification and the investigated outcomes will be excluded. No geographical area, language, or date of publication restrictions will be applied.
Interventions and comparators
The intervention under evaluation is trauma center verification (Table 1). Comparison groups will be non-verified centers or the same center before it was verified.
The primary outcome is hospital mortality. Secondary endpoints include morbidity (e.g., complication), resource utilization (e.g., length of stay, costs), and adherence to evidence-based processes of care (e.g., venous thromboembolism prophylaxis).
Search strategy and data sources
The search strategy will be developed by information specialists using appropriate Boolean operators to combine keywords and controlled vocabulary. Keywords will be identified by a group of experts and will be based on a combination of the terms “accreditation/verification” and “injury/trauma.” Cumulative Index to Nursing and Allied Health Literature (CINAHL), Excerpta Medica dataBASE (EMBASE), HealthStar, MEDLINE, and ProQuest Dissertation & Theses databases will be searched. Moreover, the websites of key injury organizations will be screenedFootnote 1. A preliminary version of the search strategy is presented in Additional file 2.
References will be managed using EndNote software (version X9, New York City: Thomson Reuters, 2018). Duplicates will be identified and eliminated using a peer-reviewed published approach which consists of electronic and manual screening . Multiple publications based on the same data will be identified by crosschecking authors, dates, and settings. Only one publication will be kept for analyses using criteria based on study dates (most recent), sample size (largest), and risk of bias (lowest risk).
To ensure consistency in study selection, reviewers will evaluate two to three sets of 100 randomly identified records. Once satisfactory inter-rater agreement has been reached (Kappa score > 0.7) between the two reviewers (BB, MC), they will independently review titles and abstracts of all identified records by applying the inclusion and exclusion criteria to select potentially eligible studies for full-text review. Full texts of the latter will then be retrieved and examined to determine eligibility. Discrepancies will be resolved by consultation with a third reviewer (LM). Study selection process as well as reasons for exclusions of potentially eligible studies will be described using a PRISMA flow chart.
Using a piloted standardized extraction form, two reviewers (BB, MC) will independently extract the following information: study reference, setting, design, years of the study, sample size (hospitals and patients), characteristics of the study sample (age, injury type), organizations responsible for verification, outcomes (including details of units of measurement), measures of associations for each outcome as reported, and covariates adjusted for. If information is available solely in figures, a computer-assisted program will be used to extract graphical data [22, 23]. Corresponding authors of included studies will be contacted by email (up to three attempts) if the aforementioned data are not reported.
Risk of bias
Quality appraisal will be conducted by two independent trained reviewers (BB, MC) with content and methodological expertise in epidemiology, statistics, and trauma. Discrepancies will be resolved by discussion with a third reviewer (LM). The Risk Of Bias In Non-randomized Studies – of Interventions (ROBINS-I) assessment tool will be used to assess the methodological quality of studies and risk of bias . If any RCT is included, the revised Cochrane risk of bias tool for randomized trials (RoB 2) will be used . Both tools will be piloted on a random sample of 5% of the included studies to ensure consistency among reviewers.
Characteristics of primary studies will be presented using a table and described narratively. If more than two studies have evaluated the same category of outcome (hospital mortality for example), a meta-analysis will be performed . Pooled effect estimates and 95% confidence intervals will be calculated using random effects models weighted by the inverse of the variance of estimates. Publication bias will be explored using funnel plots . The quality of evidence and strength of recommendations will be evaluated using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group methodology . We will present results using a narrative synthesis if heterogeneity across studies in terms of populations, design, or methods is too great.
Subgroup and sensitivity analyses
If data are available, analyses will be conducted by trauma center designation level. Previous studies have indicated that patients taken to level I centers have improved survival and better functional outcomes compared with those taken to level II centers [29, 30]. We will also present the results stratified by study years (in decades) to account for the evolution of the verification process since its introduction, by geographical regions (North America, Europe, Asia, Africa, and Australia), and by risk of bias (low, medium, and high), in line with the PRISMA statement .
Timeframe for conducting the review
We are planning to complete the review and submit it for publication within 1 year of protocol registration on PROSPERO. An update will be undertaken if more than 6 months separate the date of the last search and the date of submission for publication.
This review is being undertaken as part of a research project aiming to advance knowledge on characteristics of the verification process that could optimize its effectiveness, including the optimal frequency of verification visits.
Verification of trauma centers is currently used in the USA and in many other high-income countries on the basis that it strengthens partnerships with stakeholders, team work, and identification of improvement opportunities and priorities [8, 14]. Therefore, synthesizing available evidence on its effectiveness represents an important step towards improving the management of trauma systems and consequently injury outcomes. The results of this review will fill an important knowledge gap and will provide information which may reinforce current verification modalities and may suggest ways to optimize them.
Our proposed systematic review is based on state-of-the-art methodological and reporting standards [19, 24, 28]. However, despite our intention to employ widely accepted statistical models for meta-analysis, elevated statistical heterogeneity in most of our planned analyses is expected. We anticipate this variability due to several factors, the most notable being that we expect that most studies will be based on observational data. Finally, verification is a complex health intervention and we expect substantial heterogeneity in intervention characteristics among eligible studies, due to the evolution of the verification process since its introduction and geographical variations.
We plan to disseminate our results via presentation at international clinical conferences and by publishing the results in a peer-reviewed journal.
Availability of data and materials
The datasets that will be generated and analyzed during the review will be available from the corresponding author on reasonable request.
The American College of Surgeons, Trauma Association of Canada, International Association for Trauma Surgery and Intensive Care, Australasian Trauma Society, Trauma Audit Research Network, American Association for the Surgery of Trauma, Eastern Association for the Surgery of Trauma, American Trauma Society, British Trauma Society, Orthopaedic Trauma Association, Western Trauma Association, Trauma.org, The Society of Trauma Nurses, International Trauma Anaesthesia and Critical Care Society, Brain Trauma Foundation
American College of Surgeons
Cumulative Index to Nursing and Allied Health Literature
Excerpta Medica dataBASE
Grading of Recommendations Assessment, Development and Evaluation
Preferred Reporting Items for Systematic review and Meta-Analysis protocols
International Prospective Register of Systematic Reviews
The Risk Of Bias In Non-randomized Studies – of Interventions
World Health Organization
Abubakar I, Tillmann T, Banerjee A. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;385(9963):117–71.
Haegerich TM, Dahlberg LL, Simon TR, Baldwin GT, Sleet DA, Greenspan AI, et al. Prevention of injury and violence in the USA. Lancet. 2014;384(9937):64–74.
Centers for Disease Control Prevention. Leading causes of death by age group, United States–2010. Atlanta, Ga: Centers for Disease Control, undated As of December. 2011;12:2011.
Parachute. The cost of injury in Canada. Toronto, ON; 2015. https://parachute.ca/en/professional-resource/cost-of-injury-report/.
MacKenzie EJ, Rivara FP, Jurkovich GJ, Nathens AB, Frey KP, Egleston BL, et al. The national study on costs and outcomes of trauma. J Trauma Acute Care Surg. 2007;63(6):S54–67.
Moore L, Turgeon AF, Lauzier F, Emond M, Berthelot S, Clement J, et al. Evolution of patient outcomes over 14 years in a mature, inclusive Canadian trauma system. World Journal of Surgery. 2015;39(6):1397–405.
Gabbe BJ, Lyons RA, Fitzgerald MC, Judson R, Richardson J, Cameron PA. Reduced population burden of road transport-related major trauma after introduction of an inclusive trauma system. Annals of Surgery. 2015;261(3):565–72.
American College of Surgeons Committee on Trauma. Resources for optimal care of the injured patient 2014 Accessed; 2014 [cited 2019 2019-03-12]. Available from: https://bit.ly/2RWVyFs.
Accreditation Canada. Trauma Distinction information package 2014 [cited 2019 2019-03-12]. Available from: https://accreditation.ca/files/trauma-info-package-en.pdf.
Trauma Association of Canada. Trauma System Accreditation Guidelines. 2011. p. 88.
America Trauma Society. Trauma center levels explained 2018 [cited 2019 2019-03-12]. Available from: http://www.amtrauma.org/?page=TraumaLevels.
American College of Surgeons. Verification, review, and consultation program for hospitals - Hospital Prereview Questionnaire (PRQ) 2013 [cited 2018 2018-03-12]. Available from: http://ow.ly/yoQm30mkGob.
American College of Surgeons. Resources for optimal care of the injured patient 2014: Resources Repository 2018 [cited 2019 2019-03-12]. Available from: https://bit.ly/1nXDI1p.
Grossman MD, Yelon JA, Szydiak L. Effect of American College of Surgeons trauma center designation on outcomes: measurable benefit at the extremes of age and injury. J Am Coll Surg. 2017;225(2):194–9.
Brown JB, Watson GA, Forsythe RM, Alarcon LH, Bauza G, Murdock AD, et al. American College of Surgeons trauma center verification versus state designation: are level II centers slipping through the cracks? The Journal of Trauma and Acute Care Surgery. 2013;75(1):44–9.
Rotondo MF, Bard MR, Sagraves SG, Toschlog EA, Schenarts PJ, Goettler CE, et al. What price commitment: what benefit? The cost of a saved life in a developing level I trauma center. J Trauma Acute Care Surg. 2009;67(5):915–23.
Ashley DW, Mullins RF, Dente CJ, Garlow L, Medeiros RS, Atkins EV, et al. What are the costs of trauma center readiness? Defining and standardizing readiness costs for trauma centers statewide. Am Surg. 2017;83(9):979–90.
Fairbrother G, Gleeson M. EQuIP accreditation: feedback from a Sydney teaching hospital. Austr Health Rev. 2000;23(1):153–62.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.
Batomen B, Moore L, Carabali M, Strumpf E, Nandi A. Trauma centers verification or accreditation: protocol for a systematic review PROSPERO2018 [cited 2019 2019-03-12]. Available from: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=107083.
Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association: JMLA. 2016;104(3):240.
de Oliveira IR, Santos-Jesus R, Po ALW, Poolsup NJF, pharmacology c. Extracting numerical data from published reports of pharmacokinetics investigations: method description and validation. 2003;17(4):471-472.
Robson RC, Hwee J, Thomas SM, Rios P, Page MJ, ACJJoce T. Few studies exist examining methods for selecting studies, abstracting data, and appraising quality in a systematic review; 2018.
Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.
Eldridge S, Campbell M, Campbell M, Dahota A, Giraudeau B, Higgins J, et al. Revised Cochrane risk of bias tool for randomized trials (RoB 2.0): additional considerations for cluster-randomized trials. 2016.
Valentine JC, Pigott TD. Rothstein HRJJoE, Statistics B. How many studies do you need? A primer on statistical power for meta-analysis. 2010;35(2):215–47.
Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Publication bias. Introduction to meta-analysis: John Wiley & Sons; 2011. p. 277–92.
Berkman ND, Lohr KN, Ansari M, McDonagh M, Balk E, Whitlock E, et al. Grading the strength of a body of evidence when assessing health care interventions for the effective health care program of the Agency for Healthcare Research and Quality: an update. Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet]: Agency for Healthcare Research and Quality (US); 2013.
Cudnik MT, Newgard CD, Sayre MR, Steinberg SM. Level I versus level II trauma centers: an outcomes-based assessment. J Trauma Acute Care Surg. 2009;66(5):1321–6.
Herrera-Escobar JP, Rios-Diaz AJ, Zogg CK, Wolf LL, Harlow A, Schneider EB, et al. The “mortality ascent”: hourly risk of death for hemodynamically unstable trauma patients at level II versus level I trauma centers. 2018;84(1):139–45.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine. 2009;6(7):e1000097.
To Andrea Quaiattini Ms., who was the librarian helping to refine the research question, keywords, and MeSH terms for the preliminary search strategy
Funds for this project are covered by the Fonds de recherche du Québec – Santé (FRQS) PhD scholarship (BB) and a CIHR Foundation grant (LM).
Ethics approval and consent to participate
We have received ethics approval for this project from McGill University Faculty of Medicine Research Ethics committee.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Batomen, B., Moore, L., Carabali, M. et al. Effectiveness of trauma centers verification: Protocol for a systematic review. Syst Rev 8, 292 (2019). https://doi.org/10.1186/s13643-019-1239-6