Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Describing methods and interventions: a protocol for the systematic analysis of the perioperative quality improvement literature

  • Emma Jones1Email author,
  • Nicholas Lees2,
  • Graham Martin1 and
  • Mary Dixon-Woods1
Systematic Reviews20143:98

DOI: 10.1186/2046-4053-3-98

Received: 23 July 2014

Accepted: 13 August 2014

Published: 5 September 2014

Abstract

Background

Quality improvement (QI) methods are widely used in surgery in an effort to improve care, often using techniques such as Plan-Do-Study-Act cycles to implement specific interventions. Explicit definition of both the QI method and quality intervention is necessary to enable the accurate replication of effective interventions in practice, facilitate cumulative learning, reduce research waste and optimise benefits to patients. This systematic review aims to assess quality of reporting of QI methods and quality interventions in perioperative care.

Methods

Studies reporting on quality interventions implemented in perioperative care settings will be identified. Searches will be conducted in the Ovid SP version of Medline, Scopus, the Cochrane Central Register of Controlled Trials, the Cochrane Effective Practice and Organisation of Care database and the related articles function of PubMed. The journal BMJ Quality will be searched separately. Search strategy terms will relate to (i) surgery, (ii) QI and (iii) evaluation methods. Explicit exclusion and inclusion criteria will be applied. Data from studies will be extracted using a data extraction form. The Template for Intervention Description and Replication (TIDieR) checklist will be used to evaluate quality of reporting, together with additional items aimed at assessing QI methods specifically.

Systematic review registration

PROSPERO http://CRD42014012845

Keywords

Quality improvement Perioperative care Surgery Interventions Reporting Quality of care Description

Background

Quality Improvement (QI) methods are specially designed efforts and processes aimed at generating improvements in patient care [1]. Such methods include those based on Lean, Six Sigma, Plan-Do-Study-Act (PDSA) cycles, Total Quality Management and Continuous Quality Management, audit and feedback, and many others [2]. Guidance on reporting of QI studies [3] and of intervention delivery in evaluative studies [47] has been published. Surgery is an especially important area for quality improvement: an estimated 234 million surgical interventions are performed every year worldwide [8], yet it remains hazardous and prone to error and complication. An international drive to improve quality of care in surgery is now supported by initiatives such as the Centre for Global Surgery [9] and the Lancet Commission for Global Surgery [10]. Yet the quality of reporting of interventions in QI studies in surgery is unknown. This is an important problem, as it is increasingly recognised that explicit descriptions of interventions are necessary to ensure that successful interventions can be replicated in practice, to avoid research waste, to facilitate cumulate learning and to ensure that patients gain the best possible benefits from any learning from QI studies [11, 12]. We seek to adopt and adapt the Template for Intervention Description and Replication (TIDieR) checklist [4] to evaluate the quality and completeness of reporting of studies of quality improvement interventions in perioperative care.

Perioperative care is a process encompassing care received before, during and after a surgical procedure [13]. The translation of successful QI strategies into surgical practice has the potential to contribute towards ensuring the delivery of safe, high-quality, accessible and affordable surgery [2, 1418]. Systematic review has evaluated data generated by QI methods across cardiothoracic, colorectal [14], vascular, hepatobiliary and upper gastrointestinal specialties [2, 15, 16] and has reported measureable improvements across the whole perioperative journey including the preoperative period (reduction in time to surgery [16]), intraoperative period (reduction of sepsis [16]) and postoperative period (reduction of surgical site infection [14, 16], central venous catheter infection [16] and venous thromboembolism [16]) Yet the QI literature in surgery has also been found to suffer from a range of problems including lack of explicit rationale, poor detail and overlapping components in the published descriptions of QI methods [2, 19] and quality interventions [20]. The extent and quality of patient and public involvement (PPI) in surgical research are also unclear, despite recommendations that the Guidance for Reporting Involvement of Patients and Public (GRIPP) checklist be used in order to provide a quality assurance on the level of PPI reporting [21].

One problem in assessing the literature on quality improvement is a degree of conceptual and terminological confusion over the term ‘intervention’. The methods of improvement are sometimes referred to as interventions, yet so too are the quality interventions that such methods seek to implement. Thus, for example, the literature may use the term ‘intervention’ interchangeably to describe both application of the PDSA method and a quality intervention such as a checklist or ‘bundle’.

For purposes of this review, QI methods will be defined as the processes (such as PDSA cycles) which are typically intended to support the implementation of a quality intervention. Quality interventions will be defined as the individual components of care delivery which are implemented in order to achieve an improvement in the delivery of patient care. Quality interventions need to be described explicitly and precisely if it is to be possible to implement them. The parameters that might be used in describing such interventions include

  •  What (which materials, and activities should be used)

  •  Who (qualification type and competency)

  •  How (face-to-face, in a group, via the internet)

  •  Where (setting, infrastructure)

  •  When and how much (dose, timing, frequency, duration)

  •  Tailoring (personalisation)

  •  Modifications (changes during the course of the study)

  •  How well (what challenges were identified, e.g. dropouts or missing data)

Aims

This systematic review aims to assess the completeness of reporting within the perioperative literature on QI methods and quality interventions and to identify which elements are most frequently missing.

Methods

Design

We will undertake a review of the published qualitative and quantitative surgical literature on QI. We will define QI methods as the processes which are usually intended to support implementation of the quality intervention such as PDSA cycles. We will define quality interventions as the individual components of care delivery which are selected in order to make an improvement (such as issuing checklists or care bundles).

Eligibility criteria

This review will include

  •  All studies published between 1 January 2000 and 28 May 2014 to capture all papers indexed since the publication of the Institute of Medicine's ‘To Err is Human: Building a Safer Health System’ report [22], which highlighted the importance of systems-based interventions to address quality and safety problems

  •  All surgical specialities

  •  Adult surgical services

  •  Elective and emergency (trauma) surgery

  •  Primary and secondary care, because hospital stay is just one aspect of the surgical patient's whole pathway [23]

  •  Studies using both validated and unvalidated measures

  •  Studies meeting the criteria within the QI taxonomy generated by Shojania et al. [24] (Table 1)

Table 1

Quality improvement taxonomy

QI strategy

Definition

Methods

Surgical examples

1. Provider education

Dissemination of information

Educational outreach visits

Component separation training and recurrence rates

Distribution of educational material

Cadaveric training and surgeon confidence

2. Provider reminder systems

Any ‘clinical encounter-specific’ information intended to prompt a clinician to recall information or consider a specific process of care

Decision aids

MEWS

Reminders

The WHO surgical safety checklist

3. Patient reminders

Any methods of encouraging patient compliance to self-management

Appointment reminders

SMS exercise reminders before bariatric surgery

4. Promotion of self-management

Access to a resource that enhances the patients' ability to manage their condition

BP devices

Follow up phone calls with recommended adjustments to care

Fit Bits/pedometers

5. Audit and feedback

Any feedback of clinical performance

PROMs

Percentage of patients achieving target LOS

LOS

Morbidity and mortality

6. Patient education

Dissemination of information

Distribution of educational material

Tri-modal pre-habilitation programme compliance and effect on LOS

Individual or group sessions

7. Organizational change

Any change in organizational structure

Multidisciplinary teams

Changes to staff rota to facilitate early patient mobilization after elective arthroplasty

Communication

Health records

8. Financial, regulatory, or legislative incentives

Any financial bonus, reimbursement or provider licensure scheme

Positive or negative incentives for providers or patients

18-week wait target for elective orthopaedic surgery

9. Facilitated relay of clinical data to providers

Transfer of clinical information from patients to the provider when data was not collected during a patient visit

Telephone call

Relay of BP measurements to the pre-assessment team

Postal contact

Collection of postoperative complication data through postal survey

Adapted from Shojania et al. [24] Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies (Vol. 2: Diabetes Mellitus Care). Technical Reviews, Rockville (MD): Agency for Healthcare Research and Quality (US). LOS length of stay, MEWS Modified Early Warning System, BP blood pressure, WHO World Health Organization, SMS Short Message Service.

  •  All epubs ahead of print which are indexed in one of the selected databases by the end date specified for the review

  •  QI papers reporting upon a deliberate effort to produce change in perioperative care. This may be in the form of a QI report, or a study of a QI method or quality intervention

This review will exclude

  •  Audits, unless they explicitly report on the implementation of a QI method which is designed to produce and evaluate a change

  •  Qualitative papers reporting exclusively on staff or patient experience of using QI methods

  •  Papers reporting on screening programmes and diagnostic interventions such as endoscopy and end-of-life care

  •  Papers reporting on secondary analyses where the main results have been published elsewhere

  •  Editorials and articles not published in the English language

  •  Abstracts and conference proceedings

Disagreements about eligibility will be resolved by discussion within the team.

Search strategy

Databases will be selected for their ability to represent surgical and improvement method literature. Searches will be performed in the Ovid SP version of Medline, Scopus, the Cochrane Central Register of Controlled Trials, the Cochrane Effective Practice and Organisation of Care (EPOC) database (which indexes interventional studies focused on improvement in healthcare delivery) and the related articles function of PubMed. The journal BMJ Quality will be searched online using the find function for perioperative and surgical terms. MeSH terms, search terms, thesaurus mapping and Boolean operators will be used.

The search strategy (Figure 1) was designed by three reviewers (ELJ, MDW and GPM). It will be conducted by an experienced research fellow (ELJ) who will apply the restrictions of publication year (2000–2014), humans NOT animals, and NOT infants. The search strategy is intended to capture terms relating to (i) surgery, (ii) quality improvement and (iii) methodology. Improvement terms were adapted from the improvement science research scan produced by the Health Foundation [1].
Figure 1

Search strategy.

A training exercise was undertaken with a sample of search results where two authors (ELJ and MDW) considered selected full-text articles and discrepancies were resolved with a third reviewer (GPM). This enabled the authors to refine inclusion and exclusion criteria, ensuring consensus and reliable article selection.

A further training exercise will be undertaken whereby two authors (ELJ and NJL) will independently rate a sample of full-text articles against the TIDieR checklist until a high agreement is reached. TIDieR [4] is recommended by the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network as an extension of the Consolidated Standards of Reporting Trials (CONSORT) [25] and SPIRIT [26] statements to improve reporting across all ‘evaluative’ study designs. Each item in the checklist will have an explanatory statement to guide the rater on how it should be interpreted. In accordance with previous work conducted by Hoffman and colleagues [12], the 12 items on the TIDieR checklist will be rated as ‘Yes’ (indicating that the description of that element of the intervention had been explicit) or ‘No’ (not reported or not clearly described).

Two reviewers (ELJ and NJL) will independently assess titles and abstracts of all abstracts to select and obtain full-text articles. The search results will be supplemented with hand searching of the reference lists of the full-text articles and of one recently published systematic review on improvement science [2] (ELJ).

Data extraction

This review defines a quality intervention as a change to process directed at securing improvement, for example, introducing joint working patterns at weekends for assistant practitioners and foundation year 1 doctors to improve the rate of peripheral cannula insertion to reduce missed antibiotics. A QI method is defined as the process by which the change is supported and facilitated, for example, PDSA cycles. Patient and public involvement will be defined as the incorporation of the knowledge, skills and experience of patients, carers and the public into a study [27, 28].

Data will be extracted from each selected paper by ELJ and NJL using standardised Excel templates. The first template will contain the 12 TIDieR checklist items [4] for recording the description of the intervention. The second template will contain key elements of the QI method. For purposes of this analysis, the salient features of quality improvement methods were determined by the authors following review and discussion of the relevant literature (see Table 2 for the list of features). When reports contain descriptions of two interventions, they will be rated separately. Checklists have also been published to facilitate reporting of PPI [21, 29]. However, the emphasis of this review is upon the completeness of the QI reporting so the papers will be judged against only one PPI criterion, scoring ‘DONE’ if patient involvement is specified, ‘NOT CLEAR’ if it is not reported and ‘NOT DONE’ if patients are explicitly not involved. Discrepancies in the allocation of checklist scores will be resolved by discussion with a third reviewer (GPM or MDW).
Table 2

Data extraction template items

Demographics

Quality intervention (TIDieR parameters)

QI method

Author, year, country, surgical speciality

1. Brief name

1. Sample size

2. Why (rationale for intervention)

2. Baseline measurement

3. What (materials used to apply the intervention)

3. Data collection schedule

4. Procedures (processes used in the intervention)

4. Data analysis (e.g. driver diagrams)

5. Who (who delivered the intervention, including level of training)

5. Data volume/duration (e.g. length of PDSA cycle)

6. How (mode of delivery: face to face, internet)

6. Explicit description of prediction of change

7. Where (location: emergency or elective, and primary or secondary care)

7. Missing data (and reasons given)

8. When and how much (duration, dose, intensity)

8. Description of generalizability

9. Tailoring (was the intervention planned to be personalised)

9. Adverse effects (on health care providers and resource utilisation)

10. Modifications (describe what, why, when and how modifications were made)

10. Presence and type of patient or stakeholder involvement (collaborative or consultative)

11. How well (strategies to improve or maintain compliance)

12. How well (outcome of compliance assessment)

Data analysis

Data will be analysed descriptively using an Excel data extraction sheet. Nominal data will be used to present the proportion of complete and incomplete TIDieR checklist items. The potential transferability of findings between contexts will be considered. This is a descriptive review, and meta-analysis will not be undertaken.

Consistent with the principle that reviews may engage an iterative process [30], the review may evolve iteratively to include additional analysis such as bibliometric measures and descriptions of the fidelity of the interventions.

Discussion

This review has a number of strengths and limitations. It will be the first review assessing how well QI methods and quality interventions are described across diverse settings (emergency and elective, and primary and secondary care) in perioperative care. This will advance understanding on what is required to improve reporting on QI methods and quality interventions. The review will extend beyond a presentation of raw outcome data, also considering the following: What rationales are provided for the application of specific QI strategies? How is QI defined? To what extent are patients involved? The findings of this review will be used to generate a research protocol to identify and resolve the challenges associated with defining and providing accounts of all of the elements of QI methods and interventions in surgery. This knowledge will generate a practical framework to facilitate the replication of effective QI strategies in practice. This framework will be pilot tested to confirm that it is a reliable method of specifying and describing the elements of QI methods and quality interventions. We anticipate that this work will be relevant to a wide multi-disciplinary community of clinicians and researchers who wish to reliably accelerate positive changes to practice to improve quality of care for patients and to improve the quality of the reporting of QI methods and interventions in perioperative literature.

Our study may have limitations. Papers not published in English will be excluded due to resource limitations, which may introduce bias. However, research has suggested that such exclusions tend to have a limited effect overall on systematic review conclusions [31]. Steps have been taken to limit potential subjectivity in data analysis by including standardised data extraction tools and checklists and by achieving consensus with a third reviewer. A team of social scientists and clinicians will ensure that key messages most appropriate to a surgical audience will be disseminated, addressing gaps in the current reporting of QI methods and interventions.

Abbreviations

ELJ: 

Emma Jones

EPOC: 

Cochrane Effective Practice and Organisation of Care

GPM: 

Graham Martin

GRIPP: 

Guidance for Reporting Involvement of Patients and Public checklist

MDW: 

Mary Dixon-Woods

NJL: 

Nicholas Lees

PDSA: 

Plan-Do-Study-Act

PPI: 

Patient and public involvement

TIDieR: 

Template for Intervention Description and Replication

QI: 

Quality improvement.

Declarations

Acknowledgements

This work was conducted as part of a Doctoral Research Fellowship funded by The Health Foundation. Mary Dixon-Woods' work is supported by a Wellcome Trust Senior Investigator award ref WT097899.

Authors’ Affiliations

(1)
Department of Health Sciences, University of Leicester
(2)
Department of Anaesthesia, Royal Brompton and Harefield NHS Foundation Trust

References

  1. The Health Foundation: Report: Improvement Science. Research Scan. 2011, London: The Health FoundationGoogle Scholar
  2. Nicolay CR, Purkayastha S, Greenhalgh A, Benn J, Chaturvedi S, Phillips N, Darzi A: Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg. 2012, 99: 324-335. 10.1002/bjs.7803.View ArticlePubMedGoogle Scholar
  3. Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J, Nelson W, Pronovost PJ, Provost L, Rubenstein LV, Speroff T, Splaine M, Thomson R, Tomolo AM, Watts B: The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008, 17 (1): i13-i32.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Hoffmann TC, Glasziou PP, Barbour V, Macdonald H: Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014, 1687: 1-13.Google Scholar
  5. Schulz R, Czaja S, McKay J, Ory M, Belle S: NIH Public Access Intervention Taxonomy (ITAX): describing essential features of interventions (HMC). Am J Heal Behav. 2010, 34: 811-821.Google Scholar
  6. Cochrane Effective Practice and Organisation of Care Group (EPOC): Cochrane Effective Practice and Organisation of Care Review Group Data Collection Checklist. 2002, Ontario: Institute of Population Health, University of OttawaGoogle Scholar
  7. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P: Methods and processes of the CONSORT Group: example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med. 2008, 148: W60-W66.PubMedGoogle Scholar
  8. Weiser TG, Regenbogen SE, Thompson KD, Haynes AB, Lipsitz SR, Berry WR, Gawande AA: An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet. 2008, 372: 139-144. 10.1016/S0140-6736(08)60878-8.View ArticlePubMedGoogle Scholar
  9. :Center for Global Surgery Annual Report 2013, University of Utah. 2013,http://medicine.utah.edu/globalsurgery/cgs-annual-report.pdf,
  10. Frazier J a, McDougle CJ: The lancet commission on global surgery. Harv Rev Psychiatry. 2014, 22: 61-64. 10.1097/HRP.0000000000000028.View ArticlePubMedGoogle Scholar
  11. Glasziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews?. BMJ. 2008, 336: 1472-1474. 10.1136/bmj.39590.732037.47.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Hoffmann TC, Erueti C, Glasziou PP: Poor description of non-pharmacological interventions: analysis of consecutive sample of randomised trials. BMJ. 2013, 3755: 1-10.Google Scholar
  13. National Institute for Health and Clinical Excellence: Keeping Patients Warm Before, During and After an Operation. 2008, London: NICE, 1-7.Google Scholar
  14. Berenguer CM, Ochsner MG, Lord SA, Senkowski CK: Improving surgical site infections: using National surgical quality improvement program data to institute surgical care improvement project protocols in improving surgical outcomes. J Am Coll Surg. 2010, 210: 737-741. 10.1016/j.jamcollsurg.2010.01.029. 741–3View ArticlePubMedGoogle Scholar
  15. Koetsier A, van der Veer SN, Jager KJ, Peek N, De Keizer NF: Control charts in healthcare quality improvement. A systematic review on adherence to methodological criteria. Methods Inf Med. 2012, 51: 189-198. 10.3414/ME11-01-0055.View ArticlePubMedGoogle Scholar
  16. Powell AE, Rushmer RK, Davies HTO: A Systematic Narrative Review of Quality Improvement Models in Health Care. 2009, NHS Quality Improvement Scotland: EdinburghGoogle Scholar
  17. Steinman M, Sumat R, Shojania K, Gonzales R: Improving antibiotic selection: a systematic review and quantitative analysis of quality improvement strategies. Med Care. 2006, 44: 617-628. 10.1097/01.mlr.0000215846.25591.22.View ArticlePubMedGoogle Scholar
  18. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat A-HS, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MCM, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AA: A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009, 360: 491-499. 10.1056/NEJMsa0810119.View ArticlePubMedGoogle Scholar
  19. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE: Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014, 23: 290-298. 10.1136/bmjqs-2013-001862.View ArticlePubMedGoogle Scholar
  20. Weaver S, Lubomski L, Wilson E, Martinez K, Dy S: Promoting a culture of safety as a patient safety strategy. Ann Intern Med. 2013, 158: 158-View ArticleGoogle Scholar
  21. Staniszewska S, Brett J, Mockford C, Barber R: The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011, 27: 391-399. 10.1017/S0266462311000481.View ArticlePubMedGoogle Scholar
  22. Kohn LT, Corrigan JM: DMS: To Err Is Human: Building a Safer Health System. 2000, Washington, DC: The National Academies Press, 1,Google Scholar
  23. Lombardi AV, Berend KR, Adams JB: A rapid recovery program: early home and pain free. Orthopedics. 2010, 33: 656-PubMedGoogle Scholar
  24. Shojania KGMKWROD: Closing the quality gap: a critical analysis of quality improvement strategies. (Vol. 2: Diabetes Mellitus Care) Technical Review 9 (Contract No. 290-02-0017 to the Stanford University–UCSF Evidence-based Practice Center). AHRQ Publ No 04-0051-2. 2004, Rockville, MD: Agency for Healthcare Research and Quality U.S. Department of Health and Human ServicesGoogle Scholar
  25. Group CS of RT (CONSORT): CONSORT 2010 checklist of information to include when reporting a randomised trial. 2010,http://www.consort-statement.org/consort-statement/checklist#/checklists/view/32-consort/66-title?&_suid=140594095207908582895774067327,Google Scholar
  26. Chan A, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleža-jeric K, Laupacis A, Moher D: Research methods and reporting SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013, 346 (e7586): 1-42.Google Scholar
  27. Vale C: Public involvement in clinical trials: supplement to the briefing notes for researchers. INVOLVE National Institute for Health Research,http://www.invo.org.uk/wp-content/uploads/2012/04/INVOLVEpublicinvolvementinclinicaltrialsBriefingnotes2012.pdf,
  28. INVOLVE National Institute for Health Research: What is public involvement in research?. 2013,http://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/,Google Scholar
  29. Wright D, Foster C, Amir Z, Elliott J, Wilson R: Critical appraisal guidelines for assessing the quality and impact of user involvement in research. Health Expect. 2010, 13: 359-368. 10.1111/j.1369-7625.2010.00607.x.View ArticlePubMedGoogle Scholar
  30. Moher D, Liberati A, Tetzlaff J, Altman D: Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Medicine. 2009, 6 (7): 1-6.View ArticleGoogle Scholar
  31. Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M: Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002, 31: 115-123. 10.1093/ije/31.1.115.View ArticlePubMedGoogle Scholar

Copyright

© Jones et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement