- Open Access
Teaching improvement science to paramedicine students: protocol for a systematic scoping review
Systematic Reviews volume 7, Article number: 236 (2018)
It is now more important than ever to equip paramedic students, the likely future managers and leaders of ambulance services, with the knowledge and skills of improvement science. Effective teaching requires a range of teaching methods that will engage students actively in learning. Although the array and effectiveness of methods used for teaching improvement science to clinicians and healthcare students has been systematically reviewed, the evidence regarding the specific sub-group of paramedicine students has yet to be fully explored and synthesized in the literature. The aim of this scoping review is to systematically explore and critically appraise the current state of evidence regarding strategies to teach improvement science to paramedicine students.
A number of electronic databases (i.e., PubMed, CINAHL, Embase, Scopus, and ERIC) and gray literature (i.e., ProQuest Dissertations and Theses, Open Thesis, and Networked Digital Library of Theses and Dissertations) will be searched for published and unpublished evidence regarding teaching improvement science to paramedicine students. Included studies will undergo narrative synthesis to examine similarities and differences and to identify patterns, themes, and relationships (e.g., how and why certain teaching strategies or methods have worked in achieving desired learning outcomes (or not) and factors that might have influenced this).
To the knowledge of the authors, this is the first review that will systematically explore and critically appraise the current state of research evidence regarding strategies to teach improvement science specifically to paramedicine students. It is anticipated that the findings of this review will help to inform academics, developers of paramedicine teaching curricula, and researchers who are planning projects in this area.
Systematic review registration
Scoping reviews are currently not eligible for registration on the international prospective register of systematic reviews (i.e., PROSPERO).
While a definition of paramedicine is still a subject of debate within this young healthcare discipline , it is clear that one of the core functions of paramedics is the provision of effective and safe out-of-hospital care. The relatively young age of the paramedicine profession and associated lack of research capacity, combined with the complexities of conducting data collection in a mobile health care setting, have meant that out-of-hospital practice has generally been based on relatively weak evidence [2,3,4,5]. However, research progress is being made and has real and future potential to substantially expand and raise the evidence for out-of-hospital systems and care [4, 6, 7]. Besides the production of evidence, one of the most important challenges for any healthcare service in this century is the implementation of research evidence into routine practice . Ambulance services are no exception, and thus, with the evidence base expected to grow as well as pressure to function in progressively complex and demanding environments , the success of out-of-hospital care systems is becoming increasingly dependent on effective implementation and broader quality improvement strategies.
Successful improvement efforts rely not only on the knowledge, experience and intuitions of subject matter experts, but, to be most effective, these insights must be framed scientifically and tested . Therefore, in modern healthcare services, the unceasing efforts to advance the quality of care and to find effective solutions to problems are underpinned by a science of its own domain—improvement science. It entails system thinking, understanding variation, psychology of change, and the theory of knowledge that are applied to improve the performance of processes and organizations . Improvement science is an evolving field which can be considered the scientific foundation of quality improvement, exploring how it can be best conducted . It uses robust scientific methods to understand and evaluate the quality improvement process. In principle, improvement science aims to ensure that quality improvement efforts are based as much on evidence as the best practices they seek to implement .
Quality improvement is not new to the health services though and has its roots in the work of Avedis Donabedian, a physician and researcher most famous for the development of the widely applied Donabedian Model . Further evolution of quality improvement in health care was greatly influenced by other industries, most notably manufacturing . W. Edwards Deming, significantly contributed to what became known as total quality management. In the 1990s, Toyota’s Lean and Motorola’s Six Sigma methods influenced how quality was managed in health care and contributed towards a shift from quality assurance to continuous quality improvement [14, 15]. In 1999, the Institute of Medicine published To Err is Human: Building a Safer Health System: Building a Safer Health System, a report on adverse events in health care . Similar research was conducted in countries other than the USA with equally concerning occurrence rates [17, 18]. Research into adverse events in health care gave patient safety and quality improvement further momentum and placed increased pressure on health care organizations to demonstrate that patients could trust in the safety and quality of the services they provide.
Traditionally, ambulances services had been considered predominantly as a call-handling and transportation service, encompassing some aspects of pre-hospital care . This operational focus combined with the paucity of evidence for clinical interventions has contributed to paramedicine somewhat lagging behind in applying improvement methods compared to the hospital-based healthcare services [20,21,22]. A scoping review of quality indicators for out-of-hospital care suggests that a specific definition of quality in the context of paramedicine remains elusive and many metrics used to evaluate it remain focused on traditional ideologies . A recent study investigating the adoption of quality programs in ambulance services throughout the USA found a high degree of variability in the tracking of performance measures and demonstrated the need for better access to quality improvement tools, guidance on how to adopt best practice, and staff development . Timely conveyance of patients with urgent and emergency care needs to an appropriate health care facility remains a primary function of modern ambulance services. However, the scope of out-of-hospital care and coverage that ambulance services provide has evolved significantly [25,26,27,28,29]. Although modern ambulance services can simply be seen as one part of the larger healthcare system, out-of-hospital and mobile health care remains unique in many ways , not least of which is the physical environment in which it is provided. These factors may make the application of general improvement principles challenging, highlighting the need for ambulance service leadership to have the knowledge and skills of scientific quality improvement. In other healthcare disciplines, it has been argued that improvement science is now as essential to good practice as the human life sciences that is taught during undergraduate training .
In light of the growing body of evidence for out-of-hospital systems and care, the need for this evidence to be implemented, as well as the required scientific approach to do this effectively, it would appear that it is now more important than ever to equip paramedic students, the likely future managers and leaders of ambulance services, with the knowledge and skills of improvement science. Effective teaching requires a repertoire of teaching methods that will engage students actively in learning . Although the range and effectiveness of methods used for teaching improvement science to clinicians and healthcare students has been systematically reviewed in other healthcare disciplines such as medicine and nursing [32,33,34,35], the evidence regarding the specific sub-group of paramedicine students has yet to be fully explored and synthesized in the literature. The aim of this scoping review is to systematically explore and critically appraise the current state of evidence regarding strategies to teach improvement science to paramedicine students. To achieve this overall aim, the review will:
Systematically search both published and gray literature to identify evidence concerning methods for teaching the theories and practices of improvement science to paramedicine students;
Map key concepts (e.g., publication date, geographical location, teaching methods, reported outcomes) to gain insight into the extent, range, and nature of research activity in this area;
Provide a critical narrative of the current teaching methods, identifying successes and challenges; and
Identify questions for future research regarding strategies for teaching improvement science in paramedicine education.
There are a number of reasons why a scoping review, rather than a more traditional type of systematic review, might be conducted . In the current study, the scoping review methodology was considered most suitable because it aims to explore and examine the extent, range, and nature of research activity in the area of interest and, in doing so, identify research gaps in the existing literature. Scoping reviews may be especially relevant to disciplines or areas with incipient evidence as they include a variety of study designs from both published and gray literature . Quality improvement science, especially applied in the paramedicine discipline, is an emerging concept and unpublished sources which do not meet rigorous academic standards may still provide important information and insights. Beyond underpinning future systematic review or research, scoping reviews can also inform practice . This scoping review could potentially provide a considerable evidence base to aid in the development of teaching curricula and selection of teaching methods in paramedicine programs.
This review protocol was developed using the PRISMA-Protocols (PRISMA-P) 2015 checklist (Additional file 1: Table S1) , and the final review output will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement . Considering the iterative nature of scoping review methodology, any amendments to the protocol will be reported explicitly (i.e., date, description, and rationale) in the final review output. Currently, scoping reviews are not eligible for registration on the international prospective register of systematic reviews (i.e., PROSPERO). This review was exempt from human research ethics review at Western Sydney University.
The aim of this scoping review is to systematically explore and critically appraise the current state of evidence regarding strategies to teach improvement science to paramedicine students. To meet this aim, any evidence that meets the following PICOCS (participants, interventions, comparators, outcomes, context, study designs) and study type criteria will be considered for inclusion.
Students studying towards a qualification in paramedicine and/or academic staff teaching paramedicine, both full-time and part-time students, will be included. Academic staff will be defined as teachers or scholars (e.g., tutors, lecturers, professors) in permanent or casual employment on a full-time or part-time basis.
Any teaching strategy or learning activity used to facilitate the learning of concepts, strategies, methods, or techniques of improvement science. A teaching strategy will be defined as the structure, system, methods, techniques, procedures, or processes that academic staff use to deliver relevant information. These teaching strategies, such as lectures, seminars, printed materials, or audio-visual materials, are employed or provided by academic staff to assist student learning. Learning activities refer to the academically guided instructional tasks or assignments for students. These are student activities, such as group discussions, problem-solving workshops, or quality improvement projects.
No comparator required.
Articles that report any outcome related to the effectiveness of teaching strategies (e.g., structured interviews with students, student satisfaction surveys, tests and exams, content analysis, peer observation feedback or specific outcome measures).
One of the main aims of this review is to inform paramedicine curriculum developments and potential primary research projects in this field in Australia. The education and training of the paramedicine workforce in Australia is in the final stages of its transition from vocational training programs to university-level education programs . However, considering that in many other countries paramedicine education is offered through non-university training courses, these will be included. For the purpose of this scoping review, undergraduate and postgraduate university programs as well as non-university programs will be considered.
Any study design (i.e., quantitative, qualitative, or mixed methods) as well as opinion pieces, commentaries, letters and editorials. Industry reports, position papers, or program reports will be included. Review articles will be excluded, but relevant papers will be used to crosscheck for primary papers. Personal blogs and social media posts will be excluded.
The search strategy will aim to find both published and unpublished studies. As recommended by the Joanna Briggs Institute (JBI) , a three-step search strategy will be utilized. An initial limited search of PubMed and CINAHL (Cumulative Index to Nursing and Allied Health Literature) will be undertaken followed by analysis of the text words contained in the title and abstract, and of the index terms used to describe articles. A second search using all identified keywords and index terms will then be undertaken across all included databases. Thirdly, the reference list of all identified reports and articles will be searched for additional studies. Only English language papers will be included for pragmatic reasons. Searches will not be limited by date. The databases to be searched include PubMed, CINAHL, Embase, Scopus, and ERIC (Educational Resources Information Centre). The search for unpublished studies will include ProQuest Dissertations and Theses, Open Thesis, and NDLTD (Networked Digital Library of Theses and Dissertations) as well as backtracking of references. Furthermore, experts in the field of study will be consulted to identify potential literature for inclusion. Experts will be contacted through the professional networks of the authors.
The search terms will likely be comprised of three facets : terms to describe paramedicine,  terms to describe improvement science, and  terms to describe teaching strategies. The search terms will be developed by the principle investigator (RP) and one co-investigator (LS), with intellectual input from the rest of the review team (AM, PS, DW) and in consultation with a librarian. Where available, recommended search filters, such as one for paramedicine recently developed by Olaussen et al. , will be consulted. Details of the final search terms use will be included in the review output.
All search results will be exported to Mendeley (Mendeley Ltd., 2018), an online reference management system. Mendeley will be used to delete duplicate records as well as to screen and share records between reviewers. Microsoft (MS) Office Excel (Microsoft Corporation, 2018) will be used for the data extraction process.
Study selection will be undertaken in accordance with the above inclusion/exclusion criteria. Initially, titles and abstracts will be screened against the eligibility criteria. If potentially eligible, the full text of the papers will be read to determine whether the article should be included in the review. If additional information is required to resolve queries about eligibility, the principle investigator (RP) will attempt to obtain this information via email from the respective corresponding author. Any non-responses will be reported anonymously in the final report. The principal investigator (RP) will undertake all selection stages, and 20% of all papers will be double-screened by a second reviewer (LS). Disagreements between the two reviewers will be discussed. Should reviewers not reach consensus, a third reviewer (AM) will be involved in the process to make a final decision on inclusion or exclusion. Reviewers will not be blinded to the journal title, study authors or associated institutions. A PRISMA flow diagram will be presented in the final output to detail search results and selection process.
A standardized data extraction form will be developed a priori by RP and LS. This form will be published as an appendix in the final review output. Data from included articles will be extracted by the principle investigator (RP). To assess risk of error in data extraction, 20% of extracted data will be verified by a co-investigator (LS). Any disagreements between the reviewers will be discussed. Should reviewers not reach consensus, a third reviewer (AM) will be involved. If data is unclear, the principle investigator (RP) will attempt to obtain this clarification via email from the respective corresponding author. Any non-responses will be reported anonymously in the final report. Data to be extracted will likely include the following:
Study characteristics: author(s), publication year, title, place of publication, national setting, study aims/objective(s)/research question(s), study design, sample size, inclusion/exclusion criteria, recruitment method, data collection method, data analysis method and funding source/body.
Participant characteristics: student age, study load (part-time/full-time), and year of study.
Teaching strategy characteristics: curriculum (compulsory versus elective), teaching duration and spread (e.g., year, semester, module, hours per week) knowledge domain  (health care as process/system; variation and measurement; customer/beneficiary knowledge; leading, following, and making changes in health care; collaboration; social context and accountability; developing new locally useful knowledge; professional subject matter) and teaching method (e.g., lectures, group discussion, case study, project, web-based learning).
Outcome characteristics: student feedback (structured interviews, satisfaction surveys, etc.), before-after test/exam results, content analysis, and peer observation feedback.
During data extraction, it may become apparent that additional unforeseen data can be usefully charted . Therefore, a pilot data extraction will be undertaken by RP and verified by LS. The charting process will be iterative in nature, enabling the principal investigator to update the data extraction form.
Congruent with the aim of a scoping review to provide an overview of the existing evidence regardless of quality , studies will not be excluded based on quality. However, to inform discussion around the overall “strength of the evidence” in this area, each paper will be scored for the strength of the findings in line with the system developed by the Best Evidence Medical Education (BEME) Collaboration .
The final review output will present extracted data from the included papers in diagrammatic and/or tabular form. Visual representation of appropriate form(s) will be provided in the results section to support the mapping of the extent, range, and nature of research activity in the area. Quantitative, qualitative, and mixed methods findings will be subjected to a narrative synthesis and the review output will present this in a descriptive format that aligns with the objectives and scope of the review. The narrative synthesis will seek to investigate similarities and differences between studies to explore patterns, themes, and relationships and propose explanations for findings, e.g., how and why certain teaching strategies or program have worked or have not worked. The synthesis will also assess the strength of evidence using the BEME grading system. Data synthesis will be undertaken by the principal investigator (RP) and discussed amongst the review team (LS, AM, PS, and DW) for validation.
To the knowledge of the authors, this is the first review that will systematically explore and critically appraise the current state of research evidence regarding strategies to teach improvement science specifically to paramedicine students. It is anticipated that the findings of this review will help to inform academic staff, developers of paramedicine teaching curricula, and researchers planning projects in this area.
Best Evidence Medical Education
Cumulative Index to Nursing and Allied Health Literature
Networked Digital Library of Theses and Dissertations
Population, intervention, comparison, outcome, context, study design
Preferred Reporting Items for Systematic Reviews and Meta-Analysis
Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols
International Prospective Register for Systematic Reviews.
Long DN, Lea J, Devenish S. The conundrum of defining paramedicine: more than just what paramedics “do.”. Australas J Paramed. 2018;15(1):1–3.
MacFarlane C. The advances and evidence base for prehospital care. Emerg Med J. 2003;20(2):114–5.
Myers JB, Slovis CM, Eckstein M, Goodloe JM, Isaacs SM, Loflin JR, et al. Evidence-based performance measures for emergency medical services systems: a model for expanded EMS benchmarking. Prehospital Emerg Care. 2008;12(2):141–51.
Turner J. Building the evidence base in pre-hospital urgent and emergency care: a review of research evidence and priorities for further research. Sheffield: University of Sheffield Medical Care Research Unit; 2010.
Turner S, Lang E, Brown K, Leyton C, Bulger E, Sayre M, et al. Is prehospital care supported by evidence-based guidelines? An environmental scan and quality appraisal using AGREE II. Can J Emerg Med. 2017;19(Suppl 1):S35.
Travers S, Segal N, Maurin O, Lemoine S, Ernouf C, Jost D. Research in prehospital emergency medicine: comparison by geographic origin of publications. Eur J Emerg Med. 2015;22(5):377.
O’Meara P, Maguire B, Jennings P, Simpson P. Building an Australasian paramedicine research agenda: a narrative review. Health Res Policy Syst Health Research Policy and Systems. 2015;13(1):79.
World Health Organization (WHO). Bridging the “know–do” gap: meeting on knowledge translation in Global Health. Geneva: World Health Organization (WHO); 2005.
Harrison L. IBISWorld Industry Report Q8591 - ambulance services in Australia. Melbourne; 2018. Available from: https://www.ibisworld.com.au/
Perla RJ, Provost LP, Parry GJ. Seven propositions of the science of improvement: exploring foundations. Qual Manag Health Care. 2013;22(3):170–86.
The Health Foundation. Improvement science [Internet]. Evidence scan: Improvement Science. London; 2011. Available from: http://www.health.org.uk/sites/health/files/ImprovementScience.pdf
Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff. 2005;24(1):138–50.
Donabedian A. Evaluating the quality of medical care. Millbank Meml Fund Q. 1966;44(3):166–206.
Sollecito WA, Johnson JK. McLaughlin and Kaluzny’s continuous quality improvement in health care. 4th ed. Burlington, MA: Jones & Bartlett Learning; 2013.
Warrier S. The evolution of quality improvement. R I Med J. 2011;94(7):211–2.
Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system. Vol. 21, Annales francaises d’anesthesie et de reanimation. Washington, D.C.: National Academic Press; 2000. p. 453–4.
Willis E, Reynolds L, Keleher H. Understanding the Australian Health Care System. 2nd ed. Chatswood: Elsevier; 2012. p. 56.
Department of Health. An organisation with a memory - report of an expert group on learning from adverse events in the NHS. In: London; 2000.
National Audit Office. Transforming NHS ambulance services. In: London; 2011.
Linwood R, Day G, FitzGerald G, Oldenburg B. Quality improvement and paramedic care - What does the literature reveal for pre-hospital emergency care in Australia? Int J Health Care Qual Assur. 2007;20(5):405–15.
Price R, Bendall JC, Patterson JA, Middleton PM. What causes adverse events in prehospital care? A human-factors approach. Emerg Med J. 2013;30(7):583–8.
Peters M, Barnard S, Dorrian M, Mackway-Jones KA. Strategy for managing quality in ambulance services. In: Wankhade P, Mackway-Jones K, editors. Ambulance services: leadership and management perspectives. Cham: Springer International Publishing; 2015. p. 31–42.
Pap R, Lockwood C, Stephenson M, Simpson P. Indicators to measure prehospital care quality: a scoping review. JBI Database Syst Rev Implement Reports. 2018;16(11):2192–223.
Redlener M, Olivieri P, Loo GT, Munjal K, Hilton MT, Potkin KT, et al. National assessment of quality programs in emergency medical services. Prehospital Emerg Care. 2018;22(3):370–8.
O’Meara PF, Tourle V, Stirling C, Walker J, Pedler D. Extending the paramedic role in rural Australia: a story of flexibility and innovation. Rural Remote Health. 2012;12(2):1978.
Cooper SJ, Grant J. New and emerging roles in out of hospital emergency care: a review of the international literature. Int Emerg Nurs. 2009;17(2):90–8.
O’Meara P, Grbich C. Paramedics in Australia. Frenchs Forest: Pearson education Australia; 2009.
Woollard M. The role of the paramedic practitioner in the UK. J Emerg Prim Heal Care. 2006;4(1):1–9.
Trevithick S, Flabouris A, Tall G, Webber CF. International EMS systems: New South Wales, Australia. Resuscitation. 2003;59(2):165–70.
Siriwardena AN, Gillam S. Quality improvement in primary care: the essential guide. London: CRC press, Taylor & Francis Group; 2014.
Hunt L, Chalmers D, Macdonald R. Chapter 2: effective classroom teaching. In: Hunt L, Chalmers D, editors. University teaching in focus: a learning-centred approach. Melbourne: ACER Press; 2012.
Ogrinc G, L a H, Mutha S, Coleman MT, O’Donnell J, Miles PV. A framework for teaching medical students and residents about practice-based learning and improvement, synthesized from a literature review. Acad Med. 2003;78(7):748–56.
Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians. J Am Med Assoc. 2007;298(9):1023–37.
Wong BM, Etchells EE, Kuper A, Levinson W, Shojania KG. Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med. 2010;85(9):1425–39.
Jones AC, Shipman SA, Ogrinc G. Key characteristics of successful quality improvement curricula in physician education: a realist review. BMJ Qual Saf. 2015;24(1):77–88.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.
Peters M, Godfrey C, McInerney P, Baldini Soares C, Khalil H, Parker D. Chapter 11: Scoping Reviews. In: Aromataris E, Munn Z, editors. Joanna Briggs Institute Reviewer’s Manual; 2017. Available from: https://reviewersmanual.joannabriggs.org.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1–9.
Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;6(7):339–b2535.
Olaussen A, Semple W, Oteir A, Todd P, Williams B. Paramedic literature search filters: optimised for clinicians and academics. BMC Med Inform Decis Mak. 2017;17(1):1–6.
Institute for Healthcare Improvement. Knowledge domains for health professional students seeking competency in the continual improvement and innovation of health care. Boston, MA: Institute for Healthcare Improvement; 1998. p. 2. Available from: xxg98
Hammick M, Dornan T, Steinert Y. Conducting a best evidence systematic review. Part 1: from idea to data coding. BEME Guide No. 13. Med Teach. 2010;32(1):3–15.
The review team would like to thank Dr. Liz Thyer from the School of Science and Health at the Western Sydney University who proofread this review protocol.
No funding will be received.
Availability of data and materials
Further information related to this review can be provided upon reasonable request. Interested readers should contact the corresponding author.
Ethics approval and consent to participate
This research was exempt from Western Sydney University Human Research Ethics review. Consent to participate is not applicable.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Table S1. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: recommended items to address in a systematic review protocol*. (DOCX 33 kb)
About this article
Cite this article
Pap, R., Shabella, L., Morrison, A.J. et al. Teaching improvement science to paramedicine students: protocol for a systematic scoping review. Syst Rev 7, 236 (2018). https://doi.org/10.1186/s13643-018-0910-7
- Improvement science
- Teaching methods