An evaluation of methods used to teach quality improvement to undergraduate healthcare students to inform curriculum development within preregistration nurse education: a protocol for systematic review and narrative synthesis
© Armstrong et al.; licensee BioMed Central. 2015
Received: 3 July 2014
Accepted: 9 December 2014
Published: 14 January 2015
Despite criticism, quality improvement (QI) continues to drive political and educational priorities within health care. Until recently, QI educational interventions have varied, targeting mainly postgraduates, middle management and the medical profession. However, there is now consensus within the UK, USA and beyond to integrate QI explicitly into nurse education, and faculties may require redesign of their QI curriculum to achieve this. Whilst growth in QI preregistration nurse education is emerging, little empirical evidence exists to determine such effects. Furthermore, previous healthcare studies evaluating QI educational interventions lend little in the way of support and have instead been subject to criticism. They reveal methodological weakness such as no reporting of theoretical underpinnings, insufficient intervention description, poor evaluation methods, little clinical or patient impact and lack of sustainability. This study aims therefore to identify, evaluate and synthesise teaching methods used within the undergraduate population to aid development of QI curriculum within preregistration nurse education.
A systematic review of the literature will be conducted. Electronic databases, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Psychological Information (PsychINFO), Education Resources Information Centre (ERIC), Medical Literature Analysis and Retrieval System Online (MEDLINE) and Applied Social Sciences Index and Abstracts (ASSIA), will be searched alongside reference list scanning and a grey literature search. Peer-reviewed studies from 2000–2014 will be identified using key terms quality improvement, education, curriculum, training, undergraduate, teaching methods, students and evaluation. Studies describing a QI themed educational intervention aimed at undergraduate healthcare students will be included and data extracted using a modified version of the Reporting of Primary Studies in Education (REPOSE) Guidelines. Studies will be judged for quality and relevance using the Evidence for Policy and Practice Information and Co-ordinating Centre’s (EPPI) Weight of Evidence framework and a narrative synthesis of the findings provided.
This study aims to identify, evaluate and synthesise the teaching methods used in quality improvement education for undergraduate healthcare students where currently this is lacking. This will enable nursing faculty to adopt the most effective methods when developing QI education within their curriculum.
Systematic review registration
KeywordsSystematic review protocol Quality improvement Curriculum Education Preregistration nursing Teaching methods Evaluation Narrative synthesis
Quality improvement (QI) has no single definition but can be conceptualised as an umbrella term which encompasses many different systematic ‘change methods’ to support improvement and better outcomes for patients and services . It remains high on the UK political and educational agenda, continuing to be a key priority within the healthcare system [2, 3].
The first signs of a ‘quality’ boom in health care could be seen at the start of the millennium when the Institute of Medicines’ advancing approaches to innovation and improvement was publicised through Crossing the Quality Chasm . Almost a decade later, Scotland adopted a similar approach, making clear their intentions to transform the NHS through The Healthcare Quality Strategy . Despite both influential policy documents highlighting that a) an ‘all’ healthcare professionals approach would be necessary and b) a bottom-up approach was required; disappointingly, translation of this message within the nursing profession has shown slow progress.
That is not to say however that a lack of effort or educational resources exists; quite the opposite. For one, many faculties offer postgraduate education at Diploma or Masters Level for practitioners albeit at a cost . The Institute of Healthcare Improvement has alternatively developed a range of free Patient Safety and QI e-learning modules for students and staff to access through their Open School . Building on this further, the World Health Organization (WHO) have well established ‘ready-to-teach’ programmes for faculties to utilise within their curriculum alongside a selection of learning resources;  as have the 1000 Lives Plus Campaign in NHS Wales . In Scotland, the Quality Improvement Hub provide a free national platform, home to multiple resources such as case studies, e-learning modules and even a curriculum framework. Here, staff are able to identify what skills, knowledge and behaviours are conducive to QI . It raises an important question therefore as to why, when there are scrupulous resources available, that transition of a QI culture within the nursing profession has been slow to develop.
Empirically, evidence has suggested that lack of staff engagement and time may be key factors . Likewise, no gold standard QI curriculum exists for preregistration nurses. Instead, QI content remains rather implicit, varying greatly between faculties with regard to content and quantity and is potentially subject to dissimilarities in academic staff knowledge [10, 11]. Furthermore, the choice of teaching methods employed by individual faculties remains discretionary and at best selected, understandably, on the resources available. However, faculty my take a different approach to the choice of teaching methods adopted if there was a suitable evidence base. The research literature  and an additional scoping review only magnify the lack of studies evaluating teaching methods for QI nurse education. If we are to create the capacity and capability of a future workforce both skilled and knowledgeable in improving the care of our patients and services, then surely the impact of different teaching methods should be fully understood?
The scoping review carried out to explore this further has necessitated greater need for a comprehensive systematic review of the literature. Although there is an increasing body of literature in QI nurse education, lack of empirical evidence exists to support or inform effective curriculum development [12–14]. Instead, a selection of previous systematic reviews of QI educational interventions exists and targets mainly large collaboratives (multi-organisational and multi-disciplinary teams who facilitate large-scale QI initiatives), middle management, the acute sector or the postgraduate medical profession [3, 15–22]. The focus of these studies includes attitudes of medical students to inform curriculum development , non-technical skills training in acute areas , factors that promote or hinder curricular implementation  and effects of teaching QI to clinicians and senior doctors [19, 20]. Whilst these studies do confirm some improvement in learner knowledge [20, 23], no changes have been identified in learner behaviour, organisational or patient outcomes [22, 23]. Furthermore, these studies are criticised for poor study design, utility of invalidated assessment tools, lack of intervention description, little use of longitudinal evaluation methods and no theoretical underpinnings [20, 22].
The latter is surprising in that the Medical Research Council (MRC) advocate that a strong theoretical underpinning is a key to any intervention success . For example, employing the philosophical assumption of constructivism would suggest that students who engage in active learning processes, as opposed to ones that are passive, will translate knowledge more effectively into practice . One could proclaim therefore that teaching methods composing a practical element may be more effective in teaching QI. That being said, this philosophy lays only the foundations to a theoretical framework and our review wishes to build on this further by adopting the use of two closely related impact theories. Impact theories set the expectations of how an intervention is likely to enable change, detailing potential causes and effects as well as highlighting the facilitating and hindering factors that may be involved .
Experiential learning is the first example. This educational theory most associated with David Kolb targets individuals to improve skills, knowledge and attitudes by the process of learning from direct experience and reflection . Kolb’s Experiential Learning Model presents a four-stage cyclical process that allows learners to 1) be actively involved in the experience, 2) reflect on the experience, 3) utilise analytical thinking and 4) make decisions and solve problems using new ideas . This model bears close resemblance to that of Langley et al. and their Model for Improvement . The Model for Improvement, although a two-part model consists of rapid cycle changes called Plan-Do-Study-Act. The ‘plan’ phase requires individuals to know the who? what? when? where? and what data to collect—which indicates that individuals would need to be actively involved to do this. The ‘do’ and ‘study’ phase requires both analysis of data and reflection on what was learned from each cycle, leaving the ‘act’ phase to determine (from that data) what modifications can be made . Due to the similarities of these two models, it would seem indicative that teaching QI to undergraduate students using an experiential learning teaching method could be extremely effective; but this is not necessarily correct in all cases.
The second example of Social Learning theory highlights this well. Bandura’s Social Learning Theory suggests that learning occurs in social contexts through continuous interactions with others by the process of observation. This means that learning may occur as a result of observing good behaviour demonstrated by a group or individual but equally as a result of the consequences of poor behaviour; this process is called ‘modelling’ . Understanding and combining Experiential Learning and Social Learning Theory may assist in our understanding of how QI educational interventions may work (or not) and allow us to formulate an appropriate hypothesis.
For example, considering both impact theories, a hypothesis that experiential learning would impact most positively on students’ skills knowledge and attitude could be made. However, the influence of observed behaviours in the social learning context may dictate whether a positive or negative impact on student behaviour occurs. James et al.  and The Health Foundation  support this view by asserting that QI should not be limited to the theoretical learning of technical skills involved (PDSA cycles and run charts) but should include the soft skills (social psychology of change), the learning skills (critical reflection, action learning) and indeed the interactions between them.
Study aims and objectives
The aim of this study therefore is to systematically review and evaluate the educational methods used to teach QI to undergraduate healthcare students.
Identify and describe the variety of teaching methods used in QI education for undergraduate healthcare students.
Determine what teaching methods impact most positively on two primary outcome measures: skills, knowledge and attitude and behaviour.
Determine what teaching methods impact most positively on two secondary outcome measures: student reaction and patient outcomes.
Identify factors that promote or hinder the effectiveness of teaching methods in skills, knowledge and attitude and behaviour.
This study will be guided and conducted using the Evidence for Policy and Practice Information and Co-ordinating Centre’s (EPPI) approach to systematic reviewing and Guidance on the Conduct of Narrative Synthesis in Systematic Reviews [31, 32]. This approach will enable a rigorous process to be followed ensuring that both methodological decisions taken remain appropriate to the review type and anticipated data set.
To identify studies relevant to the review question, a range of databases will be searched from Education, the Social Sciences and Health care.
Developed search strategy for MEDLINE using OVID
exp quality Improvement*/
(science of improvement or improvement science or continuous quality improvement or total quality management or quality standards or improvement models).tw.
1 or 2
4or 5 or 6 or 7 or 8 or 9
11 or 12 or 13 or 14
exp programme evaluation/
16 or 17
3 and 10 and 15 and 18
Reference list searching
Eligible studies will have a full scan of their reference list carried out to identify any potentially relevant studies not detected initially by the search strategy. Where difficulties in retrieving studies are encountered, authors will be contacted to request a copy of the article.
Websites of leading organisations and institutions, within the field of QI will be searched for published or unpublished materials relevant to the review question. Where a search engine is available, keywords will be input individually or combined if this option is available. Otherwise, websites will be searched manually.
Screening criteria for study inclusion/exclusion to review
Within geographical subset
Undergraduate healthcare student
We will exclude studies that describe both pre- and post-university educational interventions, describe only patient safety content, do not document original data or do not fall within the predetermined dates. We will exclude studies where the educational intervention does not include undergraduate students nor has an evaluative outcome. As previous studies have retrieved most results from the UK, USA, The Netherlands, Scandinavia, Canada and Australasia, any studies found out with this geographical subset will be excluded .
Data collection and management
One researcher will conduct and save each search within corresponding databases. To ensure that each search is accessible, reproducible and transparent, a manual record keeping log will be produced. To allow reviewer access, studies eligible for screening will be uploaded electronically or manually to a shared named folder within reference management software (Refworks). Duplicates at this stage will be removed.
Screening and selection of studies
Exclusion criteria and codes
No original/primary data
Targets different population
Outwith geographical subset
Data extraction sheet modified from REPOSE Guidelines
Country of origin
Study aims and rationale
Study research question(s)
Type of learner(s)
QI model used
(1) Knowledge, skills and attitude/behaviour
(2) Student reaction/patient outcomes
e.g. support structures
e.g. lack of resources
Quality and relevance
Weight of Evidence and TAPUPAS for quality and relevance of studies
Weight of Evidence
A = Trustworthiness of results of study (methodological quality)
(T) Transparency (A) Accuracy (A) Accessibility (S) Specificity
B = Appropriateness of study design to review question (methodological relevance)
C = Appropriateness of focus to answer review question (topic relevance)
(U) Utility (P) Propriety
D = Overall Weight of Evidence (based on A, B, C)
Low, medium or high
As a general framework, Guidance on the Conduct of Narrative Synthesis in Systematic Reviews has been applied . This non-linear approach comprises of four key stages: (1) developing a theory of how the intervention may work (e.g. combining our two impact theories to formulate a hypothesis), (2) developing a preliminary synthesis of the findings of included studies, (3) exploring relationships in the data and (4) assessing the robustness of the synthesis (e.g. applying the EPPI Weight of Evidence framework).
To develop a preliminary synthesis (2), we will display all characteristics and results of all eligible studies through tabulation. As a means of simplifying the process and aiding subsequent analysis, we will then group individual studies by population (e.g. nursing or medicine) to allow easier identification of patterns emerging within and between studies and/or disciplines. Tabulated data here will include study design, intervention description, teaching methods, outcome measures and effect. We will organise facilitating and hindering factors through thematic analysis, including factors as a theme where it appears more than once across studies. We will apply vote counting by assigning each factor one vote every time it is presented across studies to calculate the frequency of results. A summary of each theme will be reported through tabulation and a brief textual description.
Due to the expected variability in outcomes and heterogeneity in design, a meta-analysis would not be appropriate. Therefore, we will conduct a subgroup analysis using levels 2 and 3 of Kirkpatrick Model  as a framework and will explore relationships in the data (3) through concept mapping. This method allows a visual representation of possible cause and effect relationships between the data by using flow charts and diagrams. Findings will be synthesised through a textual narrative directed by the review objectives.
QI continues to dominate all aspects of healthcare delivery. Although it is emerging as a taught component in preregistration nurse education, little empirical research exists to inform the most effective way to teach QI to undergraduate healthcare students. Identifying, evaluating and synthesising teaching methods as well as awareness of the key facilitating and hindering factors associated have potential to aid nursing faculties in the development of QI education, where currently this is lacking.
LA is a recent Clinical Academic Fellow conducting a PhD within the field of Quality Improvement and was involved as an undergraduate nursing student in piloting the Institute of Healthcare Improvement Quality Improvement Practicum at University of Stirling.
Applied Social Sciences Index and Abstracts
Cumulative Index to Nursing and Allied Health Literature
Evidence for Policy and Practice Information and Co-ordinating Centre
Education Resources Information Centre
Institute of Healthcare Improvement
Medical Literature Analysis and Retrieval System Online
This systematic review is supported by funding from the School of Health Sciences at the University of Stirling.
- Heath Foundation: Quality Improvement Made Simple. London: Health Foundation; 2013.Google Scholar
- Scottish Government: The Healthcare Quality Strategy for NHSScotland. Edinburgh: Scottish Government; 2010.Google Scholar
- Health Foundation: Quality Improvement training for healthcare professionals. London: Health Foundation; 2012.Google Scholar
- Institute of Medicine (IOM): Crossing the Quality Chasm: A new healthcare system for the 21st century. Washington: National Academy Press; 2001.Google Scholar
- Institute of Healthcare Improvement: Open School Homepage. 2014. (accessed 04/04/2014) http://www.ihi.org/Pages/default.aspx Google Scholar
- World Health Organisation (WHO): Patient Safety Curriculum Guide: Multi-professional Edition. Switzerland: WHO; 2011.Google Scholar
- 1000 Lives Plus: The Quality Improvement Guide: The Improving Quality Together Edition Cardiff: 1000 Lives Improvement. 2014.Google Scholar
- NHS Scotland: Quality Improvement Hub Homepage. 2014. (Accessed 12–05–2014) http://www.qihub.scot.nhs.uk/default.aspx Google Scholar
- Health Foundation: Overcoming challenges to improving quality. London: Health foundation; 2012.Google Scholar
- Jeffs L, Abramovich IA, Hayes C, Smith O, Tregunno D, Chan WH, et al.: Implementing an inter-professional patient safety learning initiative: insights from participants, project leads and steering committee members. BMJ Qual Saf 2013. doi: 10.1136/bmjqs-2012–001720Google Scholar
- Creswell K, Howe A, Steven A, Smith P, Ashcroft D, Fairhurst K, et al.: Patient safety in healthcare pre-registration educational curricula: multiple case study-based investigations of eight medicine, nursing, pharmacy and physiotherapy university courses. BMJ Qual Saf 2013,22(10):843–54. 10.1136/bmjqs-2013-001905View ArticleGoogle Scholar
- Jones A, Williams A, Carson-Stevens A: Integrating quality improvement into pre-registration education. Nurs Stand 2013, 27:44–8.View ArticlePubMedGoogle Scholar
- Tella S, Liukka M, Jamookeeah D, Smith NJ, Partanen P, Turunen : What do nursing students learn about patient safety? An integrative literature review. J Nurs Educ 2014, 53:7–13.PubMedGoogle Scholar
- Carson-Stevens A, Mehta G, Edwards A, Panesar S: Evaluating the outcomes of quality improvement-focussed inter-professional education for pre-registration healthcare professionals. PROSPERO 2014, CRD42014006576.Google Scholar
- Abbas M, Quince TA, Wood D, Benson JA: Attitudes of medical students to medical leadership and management: a systematic review to inform curriculum development. BMC Med Educ 2011, 11:93. 10.1186/1472-6920-11-93View ArticlePubMed CentralPubMedGoogle Scholar
- Nadeem E, Olin S, Campbell Hill L, Hoagwood KE, Horwitz SM: Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q 2013, 91:354–94. 10.1111/milq.12016View ArticlePubMed CentralPubMedGoogle Scholar
- Schotuen LMT, Hulscher MEJL, Evan JJ, Huijsman R, Grol PTM: Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008, 336:1491–4. 10.1136/bmj.39570.749884.BEView ArticleGoogle Scholar
- Benn J, Burnett S, Parand A, Pinto A, Iskander S, Vincent C: Perceptions of the impact of a large-scale collaborative improvement programme: experience in the UK Safer Patients Initiative. J Eval Clin Pract 2009, 15:1356–1294.View ArticleGoogle Scholar
- Ahmed M, Arora S, Baker P, Hayden J, Vincent C, Sevdalis N: Building capacity and capability for patient safety education: a train-the-trainers programme for senior doctors. BMJ Qual Saf Healthc 2013, 22:618–25. 10.1136/bmjqs-2012-001626View ArticleGoogle Scholar
- Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB: Effectiveness of teaching quality improvement to clinicians, a systematic review. JAMA 2007, 298:1023–37. 10.1001/jama.298.9.1023View ArticlePubMedGoogle Scholar
- Okuyama A, Martowirono K, Binjen B: Assessing the patient safety competencies of healthcare professionals: a systematic review. BMJ Qual Saf 2011, 20:991–1000. 10.1136/bmjqs-2011-000148View ArticlePubMed CentralPubMedGoogle Scholar
- Gordon M, Daryshire D, Baker P: Non-technical skill training to enhance patient safety: a systematic review. Med Educ 2012, 46:1042–54. 10.1111/j.1365-2923.2012.04343.xView ArticlePubMedGoogle Scholar
- Wong BM, Etchells EE, Kuper A, Levinson W, Shojania KG: Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med 2010, 85:1425–39. 10.1097/ACM.0b013e3181e2d0c6View ArticlePubMedGoogle Scholar
- Medical Research Council: Developing and evaluating complex interventions: new guidance. London: MRC; 2008.Google Scholar
- Thomas G: Education and Theory: Strangers in Paradigms. Maidenhead: Open University Press; 2007.Google Scholar
- Grol RPTM, Bosch MC, Marlies EJL, Eccles MP, Wensing M: Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q 2007, 85:93–138. 10.1111/j.1468-0009.2007.00478.xView ArticlePubMed CentralPubMedGoogle Scholar
- The Experiential Learning Cycle: Homepage. 2014. (Accessed 01/09/2014) http://www.learningandteaching.info/learning/experience.htm Google Scholar
- Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP: The Improvement Guide: A Practical Approach to Enhancing Organizational Performance (2nd edition). San Francisco: Jossey-Bass Publishers; 2009.Google Scholar
- James B, Beattie M, Shepherd A: Quality and Safety in Practice: Why Theory Matters, [Poster]: Enhancing Nursing Through Educational Research (ENTER): Inaugural Scottish Conference, University of Stirling. 2012. 28th NovemberGoogle Scholar
- Health Foundation: Skilled for Improvement, Learning Communities and the Skills Needed to Improve Care and Evaluate Service Development. London: Health Foundation; 2014.Google Scholar
- Gough D, Oliver S, Thomas J: An Introduction to Systematic Review. London: Sage; 2013.Google Scholar
- Popay J, Roberts H, Sowden A, Petticrew M, Aria L, Rodgers M, et al.: Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. A product from the ESRC Methods Programme; 2006.Google Scholar
- Powell AE, Rushmer A, Davies HTO: A systematic narrative review of quality improvement models in healthcare. Quality Improvement Scotland; 2009.Google Scholar
- Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009, 62:1006–12. 10.1016/j.jclinepi.2009.06.005View ArticlePubMedGoogle Scholar
- Evidence for Policy and Practice Information: REPOSE (Guideline for the REPOrting of primary empirical research Studies in Education) EPPI. 2005.http://eppi.ioe.ac.uk/cms/ Google Scholar
- Parry GJ, Carson-Stevens A, Luff DF, McPherson ME, Goldman DA: Recommendations for evaluation of healthcare improvement initiatives. Acad Peadiatrics 2013, 13:S23–30. 10.1016/j.acap.2013.04.007View ArticleGoogle Scholar
- Carter K, Edwards J, Mohammad I, Myemba I, Rees J, Sullivan M: Evaluate work-based learning. In School of Postgraduate Medical and Dental Education. Cardiff: Cardiff University; no date
- Gough D: Weight of Evidence: a framework for the appraisal of the quality and relevance of evidence. Res Papers Educ 2007, 22:213–28. 10.1080/02671520701296189View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.