A protocol for ongoing systematic scoping reviews of World Trade Center Health research

Abstract

Background

The World Trade Center (WTC) Health Program (“Program”) seeks to assess the inventory, quality, and impact of its funded research in the context of all clinical and translational research involving WTC populations. This paper presents a protocol for ongoing scoping reviews of WTC-related health research.

Methods

Using terms relevant to the September 11 attacks, we will search OVID MEDLINE, PsycINFO, Scopus, Web of Science, CINAHL, and Embase for records of peer-reviewed publications. Title, abstract, and full text screening will be used to exclude records according to a priori criteria. Data abstraction will be performed on all articles that meet inclusion criteria using a standardized query form that was developed in collaboration with NIOSH. A team of reviewers will be trained to abstract data from included articles. Articles will be double-reviewed, and disagreements will be adjudicated.

Results

We will summarize existing research involving WTC populations. The summary will assess the extent, nature, and signals of impact of WTC-related health research.

Conclusions

Our review will lay the groundwork for additional study of research impact by identifying population, clinical, and translational topics that can be assessed through future focused reviews. It will also support planning activities by Program policy makers and stakeholders as they work to achieve the Program’s research goals.

Systematic review registration

This publication serves as documentation of the protocol.

Background

In 2017, the World Trade Center Health Program (hereafter, “Program”) sought to better understand the state of health research involving 9/11 responders and survivors. To respond to this need, we published a comprehensive scoping review of clinical and translational research on 9/11 populations [1]. The scoping review involved the collection of 160 data points on each of nearly 1000 peer reviewed research articles published from September 11, 2001, through October 30, 2020. We found that WTC-related health research addressed a range of health conditions and was balanced in terms of survivor and responder populations. It was concentrated on establishing links between 9/11 exposures and health conditions while concentrating less so on developing health interventions and services for treatment of WTC-related conditions.

Evidence from that review has since been used in scientific planning by the Program in several ways. The review was used to support new funding for interventional and health services research to improve health care delivery and outcomes; further inquiry into special topics such as a focused review and scientific planning meeting on youth-focused research¹; and direct public access to peer-reviewed research findings [2]. For instance, a publications dashboard scheduled for release in 2023 [3] will offer the public more extensive information on the extent and nature of WTC-related populations, health conditions, clinical interventions, health care quality, and outcomes.

In this article, we present a protocol for future, ongoing scoping review updates. As the Program is authorized to fund medical monitoring, care, and research through 2090, continued growth in published research on WTC-related health can be anticipated, and ongoing scoping review updates will be needed. Ongoing reviews will support scientific research policy, planning, and evaluation over time. They will also support direct public access to research evidence and lay the groundwork for additional focused study of WTC-related research impacts.

Research questions guiding ongoing scoping review updates include the following: (1) What has been studied with respect to the health effects of 9/11 since the initial review was completed?² (2) What signals of research impact can be documented through “mapping” WTC health-related research to recognized frameworks for translational research? [4,5,6] (3) How is WTC health research impact evolving over time?

Methods

The updated, ongoing review will encompass evidence from any English language peer-reviewed report of research conducted on or about the health of WTC populations, including case reports, observational studies, controlled trials, and systematic or narrative reviews. The research may be funded by the Program or by other sponsors and may be published between November 1, 2020, and December 31, 2022. Our approach to completing the record search, eligibility screening, data abstraction, analysis, and evidence map of WTC clinical and translational research is described below. In our previous work [1], we validated the search and screening approach by comparing results to lists of pertinent studies identified by topic experts and the “Summary of World Trade Center Health Program Research: NIOSH Research Compendium” [7] completed in 2019 by program staff.

Record search

We will use a systematic and validated search strategy to find records of WTC-related health research from databases of peer reviewed publications. An information specialist (Larkin) was involved in developing and validating the strategy. We will group search terms into four hierarchical statements, shown below. A full list of search terms is presented in Additional file 1.

• 1. Terms related to the terrorist attacks on September 11, 2001

• 2. Terms related to the numerical date (i.e., 9/11) and the type of disaster

• 3. Terms related to the New York City and New Jersey locations and the type of disaster

• 4. Terms related to the Shanksville, Pennsylvania, and Pentagon locations and the type of disaster

To find records of peer-reviewed publications, we will search six databases using terms relevant to the September 11 attacks. Using the four statements described above and in Additional file 1, we will search titles, abstracts, and keywords in OVID MEDLINE, PsycINFO, Scopus, Web of Science, CINAHL, and Embase for records of peer-reviewed articles, books, book chapters, conference abstracts, and dissertations.

Validation results, such as recall rates, will be reported in the manuscript. Automated exclusions for non-health research (such as construction engineering studies) will be applied. Included records will be limited to articles published over a 2-year period, from November 1, 2020, through December 31, 2022.

Eligibility screening

We (Concannon, Faherty, Chari) will apply five exclusion filters of increasing specificity. The filters are as follows: (1) not in English; (2) not peer reviewed research; (3) not about 9/11 attacks; (4) not about 9/11 populations; (5) not about health conditions, care, or outcomes. Titles and abstracts collected during the record search will be screened first. Records that pass sequentially through all five screens will be included in the data abstraction process described below. Those that do not will be excluded and reported as such in a Preferred Items for Reporting on Systematic Reviews and Meta Analyses (PRISMA) flowchart [8]. The first reason for exclusion will be recorded in Supplementary material. Records that are deemed uncertain will proceed to full text screen. In this stage, full text articles will be obtained, and the screening process will be repeated until every record is determined to be eligible or ineligible for inclusion. Final results of the screening process will be reported in the PRISMA flowchart.

Data abstraction

A standardized query form, shown in Additional file 2, was developed in collaboration with NIOSH to guide data abstraction. Included query term items will be coded into DistillerSR [9] as a survey tool with drop down, numeric text, and free text fields response options. Experienced reviewers (Concannon, Faherty, Chari) will abstract data from included articles. The first 10% of included articles will be double reviewed and adjudicated by the whole team. Single reviews will commence on subsequent rounds of 10% of included articles if 85% agreement is achieved on priority items in the survey tool on at least one of the rounds.

Inventory of research (RQ1)

An inventory of research (RQ1) will be captured by categorizing research publications according to elements of the PICOTSS and PECOTSS frameworks (population, interventions or exposures, comparators, outcomes, timing, setting, and study design). PECOTSS is the environmental equivalent of the Agency for Healthcare Research and Quality’s PICOTS expansion of the PICO framework, which was developed for evaluations that focus on clinical interventions rather than exposures [10].

Evidence of research impacts (RQs 2 and 3)

An assessment of translational impacts (RQs 2 and 3) will be captured by mapping research to recognized translational research frameworks (the National Institute of Environmental Health Sciences Translational Research Framework for Environmental Health Sciences [4], the Tufts Clinical and Translational Science Institute Six Types of Evidence Framework [5], and the NIOSH Research to Care (RTC) model [6], a logic diagram designed to help in program planning. The National Institute of Environmental Health Sciences (NIEHS) framework is especially well suited to evaluating the translation of research in an environmental disaster. This framework includes a tool for visualizing how ideas and knowledge move from the earliest stages of fundamental questioning to the later stages of impact (Fig. 1). The visualization depicts a series of concentric rings moving from research that addresses Fundamental Questions (purple ring, rectangles) to research that addresses Application and Synthesis (light blue ring, ovals). Other research includes Implementation and Adjustment (green ring, hexagons), Practice (dark blue ring, circles), and Impact (black ring, triangles). On each ring are nodes that identify approaches, methods, and activities that are typically used within each type of research.

Fig. 1

The National Institute of Environmental Health Sciences translational research framework (Reproduced from Environmental Health Perspectives with permission from the authors)

Reporting and visualization of results

The inventory of research (RQ1) will be reported in tables and accompanying narrative. We will present information in sequence for each item in the PICOTSS and PECOTTS frameworks. Items pertaining to the translational aspects of research (RQs 2 and 3) will be presented in tables and, where possible, we will use figures to depict concentrations of research publications along specific parts of the translational frameworks (see Fig. 1 for an example). All data collected and all data prepared for visualization and reporting will be made available with the publication of a peer reviewed and publicly available RAND Report. All RAND Reports are made avialable online and are indexed and searchable through major scientific public databases.

As this is a scoping review designed to provide data on existing research across a vast range of populations, conditions, and outcomes, we do not plan to conduct risk of bias or strength of evidence analyses at the effect level or study level. However, we will collect information (see Additional file 2) that can be used to assess risks of bias and strength of evidence in future systematic reviews on specific populations, conditions and outcomes.

Discussion

The Program carries out an active translational research program on behalf of its more than 120,000 members. This protocol describes an approach for documenting the extent and nature of WTC translational research and its impacts. It is designed to assist the program in keeping an ongoing inventory of this research by identifying the populations, interventions, and exposures that have been studied, in what settings, and over what time frames. Beyond keeping an ongoing inventory, the Program also seeks to assess the translational impacts of WTC-related health research. This protocol is designed to address this need by mapping existing research to recognized translational frameworks and by documenting changes in translational impacts over time.

In conclusion, this work will establish a systematic process for conducting ongoing scoping reviews on the growing body of WTC-related health research. The evidence we produce from ongoing scoping reviews can be used to support Program planning and public access to research evidence.