Formulating a research question
Our research objective was pre-defined and prioritised by Cochrane following a request from the World Health Organisation. This objective encompassed all types of healthcare workers, IPC guidelines and respiratory infectious diseases [3]. These concepts are broad in their meaning and added complexity to the search and the review itself. We used the SPICE (Setting, Population, Phenomenon of Interest, Comparison, Evaluation) [14] criteria to clarify these concepts. Involving a topic expert (XHC) on the team was crucial in efficiently defining our inclusion and exclusion criteria to enable us to conduct a review that was responsive to the COVID-19 pandemic, yet also relevant to other respiratory infectious diseases requiring IPC measures.
Search
Interim rapid review guidance from Cochrane recommends that an information specialist should be involved [15]. In our review, the information specialist was integral to the team [3], developing a search strategy that was then peer-reviewed by the team, including our methodological expert (AB), who drew on his combined information specialist/systematic review background to optimise the specificity of the search. It was subsequently formally peer-reviewed by a Cochrane information specialist.
The aforementioned scoping review [2] reports considerable variation across the number of electronic databases searched for in rapid QES (ranging from 3 to 7 databases). Due to our restricted timeframe, and guided by Cochrane’s interim rapid review guidance [15], we targeted a single MEDLINE database, informed by estimates of high yields of included studies [16]. We did not include grey literature and record this as a limitation of our work [3]. Whilst we acknowledge that an exhaustive search may not be necessary for QES, we undertook an additional scoping exercise of more than 1500 title and abstracts. We searched the reference lists of key papers [16, 17]. This supplementary searching was helpful as it yielded 30 additional studies for screening [3].
Selection of studies
Cochrane’s interim rapid review guidance suggests a title and abstract screening approach that involves two screeners in double independent screening for at least 20% of abstracts with conflict resolution and one screener thereafter (using a second reviewer to screen all excluded abstracts to validate the process) [15]. A similar approach is suggested for full-text screening. Most reviews included in the scoping review of rapid QES used single reviewer screening [2]. In our review, the core team double-screened the title and abstracts independently within Covidence (Covidence Systematic Review Software. Available at www.covidence.org). A similar process was applied to full-text screening. Disagreements were resolved by discussion or by involving a third reviewer [3].
On reflection, this approach had two specific benefits. First, constant communication between the core team on how we understood our eligibility criteria helped us to minimise bias at the screening stage. Second, the review authors sensitised themselves to the studies and the data from the screening stage. Potentially, this early immersion facilitated our rapid transition across the QES methods of screening, data extraction and synthesis.
On completion of full-text screening, we identified 36 studies that met our inclusion criteria [3]. Whilst it is not appropriate to prescribe how many studies should be included in a QES [17], we were aware that this number could make it challenging to become sufficiently familiar with the data [18] and so could impair the quality of our analysis. We decided to purposively sample 20 of the studies for inclusion in the synthesis (based on relevance, geographical spread, and depth of insight [19]) [3]. Sampling strategies are increasingly accepted within QES as they seek variation in concepts rather than an exhaustive sample [19]. However, sampling strategies can have important impacts on review findings and benefit from lengthy consideration. The timeframe we had to make this decision was, therefore, not ideal. We have described the sampling strategies we used as a point of reflection, not as prescriptive guidance.
Data extraction
Cochrane’s interim rapid review guidance recommends that one reviewer extracts the data using a piloted form whilst a second reviewer checks the extracted data for accuracy and completeness [15]. The scoping review of 15 rapid QES noted single reviewer data extraction as the most commonly reported method [2]. For our review, the core members used a data extraction form (Google Forms) designed specifically for this synthesis. The form was piloted on three studies, one reviewer extracted data, which was reviewed for accuracy and completeness by a second member of the team [3]. Following minor revisions to the form, each reviewer extracted data from one study, which was reviewed resulting in final alterations before full data extraction from all included studies. Continuous discussion and moderation across studies were conducted to ensure consistency.
The data extraction form was developed to support the “best fit” framework approach [20] used to analyse and synthesise the evidence (see the ‘Analysis and Synthesis’ section). Categorising the data against a ‘best fit’ framework during the extraction phase, we suggest, allowed a seamless progression to the ‘best fit’ a priori framework approach to analyse and synthesise findings.
Assessment of methodological limitations
Cochrane’s interim rapid review guidance suggests that quality assessment of included studies is conducted by a single reviewer, with verification of all judgements by a second reviewer [15]. Most of the rapid QES reported in the scoping review [2] conducted a quality assessment (mainly using the Critical Appraisal Skills Programme Qualitative Checklist Tool (CASP) [21]). We, similarly, used an adaptation of the CASP tool (the adapted tool is documented in the published rapid QES). Each included study was appraised independently by two core team members with disagreements resolved through discussion [3]. On reflection, we could have considered a single reviewer assessment with a process of verification. However, we persisted with double, independent assessments in seeking to balance speed with rigour.
Analysis and synthesis
Cochrane’s interim rapid review guidance on analysis was not considered relevant to our review, given its focus on effectiveness reviews [15]. The scoping review of rapid QES reports use of the same approaches used by regular QES [2]. We used a ‘best fit’ framework approach [20, 22]. This deductive approach is considered by its creators, and others [23], as particularly suitable for rapid synthesis [22], and this guided our decision. This pragmatic method builds on an a priori framework, enriching existing theory, rather than employing a grounded or inductive form of synthesis [24, 25]. We identified a pre-existing framework [26] that we used to analyse the studies under the domains of organisational, environmental and individual factors that can impact IPC guideline adherence.
In keeping with ‘our process’, the core team conducted analysis and synthesis simultaneously. Using the ‘best fit’ framework approach to inform data extraction meant that individual reviewers could analyse the data to populate different domains of the framework. We engaged in several online discussions to ensure consistency across analysis and synthesis, reduce overlap in the findings by mapping the range and nature of reviewed concepts and identify how the themes addressed the review question and review aim [3]. We believe that this contemporaneous critical peer review of synthesised findings made our process more trustworthy, contributing to the coherence and relevance of the findings.
Engaging in all stages of the QES, from screening to synthesis, helped the review authors move between the data and the developing themes more effectively. In our experience, this intense part of the review, given the immersion needed during analysis, requires uninterrupted focus facilitated by conversations conducted in real-time.
Assessing the confidence in the findings
Cochrane’s interim guidance about assessing the certainty of evidence in rapid reviews suggests a single reviewer approach, with verification from a second reviewer [15]. This topic is not documented in the scoping review of rapid QES [2]. For our review, the core team independently used the Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) approach to assess confidence in the findings [27]. The review authors initially applied GRADE-CERQual to those findings that were within the same domains they had explored previously in-depth during the synthesis stage. The final assessment was based on a consensus across the core team [3]. Whilst our assessment was rigorous, we did not have time to edit and include full evidence profiles in the published review (the evidence profiles are now complete and are documented in Appendix 2 of the published rapid QES).
Having team members who were familiar with GRADE-CERQual was important for expediting this stage. We were able to refine our approach in line with previous experiences and used a piloting process where each of the core team performed the assessment on three key findings and presented back to the others providing the rationale for the level of confidence we awarded. This pilot exercise was valuable in identifying any ambiguity in our GRADE-CERQual judgements early in the process. The review authors undertaking the GRADE-CERQual assessment had conducted the data extraction and the assessment of methodological limitations in relation to these studies informing the findings. Again, this knowledge and familiarity helped to expedite the process.
Developing implications for practice
Developing implications for practice was not a specific requirement in the Cochrane interim rapid review guidance. However, we felt that to support decision-makers, this section was critical. Our review team included an editor from the Cochrane Effective Practice and Organisation of Care Group (EPOC) with experience of communicating findings in this way [28], who could prepare this section quickly. This work involved mapping individual review findings into questions for consideration by decision-makers. We also included these questions in an evidence summary for end users [29].
To ensure that this summary was accessible and relatable to end users, we invited 18 nurses, doctors and care home staff from different continents and areas of healthcare to review the summary. Fifteen volunteers responded within 24 h, and minor edits were made accordingly. We sourced volunteers to translate the summary into Spanish, Portuguese, French and Norwegian. Volunteers also translated the plain language summary of the review into five languages. In addition, we recorded a podcast, since translated into multiple languages, developed an infographic and featured in Evidence Aid [30].
These efforts and generosity from others increased the accessibility and usefulness of the rapid QES findings. Future rapid review guidance needs to consider knowledge translation as a key element.
