Which learning activities enhance physiotherapy practice? A systematic review protocol of quantitative and qualitative studies
© The Author(s). 2017
Received: 26 October 2016
Accepted: 5 April 2017
Published: 17 April 2017
Learning activities are fundamental for the development of expertise in physiotherapy practice. Continuing professional development (CPD) encompasses formal and informal learning activities undertaken by physiotherapists. Identifying the most efficient and effective learning activities is essential to enable the profession to assimilate research findings and improve clinical skills to ensure the most efficacious care for clients. To date, systematic reviews on the effectiveness of CPD provide limited guidance on the most efficacious models of professional development for physiotherapists. The aim of this systematic review is to evaluate which learning activities enhance physiotherapy practice.
A search of Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO (Psychological Abstracts), PEDro, Cochrane Library, AMED and Educational Resources and Information Center (ERIC) will be completed. Citation searching and reference list searching will be undertaken to locate additional studies. Quantitative and qualitative studies will be included if they examine the impact of learning activities on clinician’s behaviour, attitude, knowledge, beliefs, skills, self-efficacy, work satisfaction and patient outcomes. Risk of bias will be assessed by two independent researchers. Grading of Recommendations Assessment, Development, and Evaluation (GRADE) and Confidence in the Evidence from Reviews of Qualitative research (CERQual) will be used to synthesise results where a meta-analysis is possible. Where a meta-analysis is not possible, a narrative synthesis will be conducted.
Systematic review registration
KeywordsSystematic review Protocol Physiotherapy Education Knowledge translation Expertise Professional development
Formal learning activities:
Attending accredited courses, conferences, seminars, forums, distance learning, further education
Completing tertiary courses which lead to a degree or higher degree
In-service educational programsa
Online learning (interactive discussion and chat rooms)
Work-based learning contracts
Planning or running a course
Courses leading to research degree
Informal learning activities:
Learning by doing, learning from experience, in-service educationa, case studies, reflecting on experience, audit of others, discussions with colleagues, peer reviewb, journal clubs, committee membership, shadowing, secondments, clinical supervision, job rotation, journal club, project work, coaching, benchmarking, role expansion, self-assessment questionnaire completion, significant analysis of events, quality assurance activities, being promoted
Professional body involvement, membership of special interest group, mentoring, lecturinga, tutoringa, examining, branch meetings, organising journal clubs, maintaining or developing specialist skills, being an expert witness, giving presentationsa, organising coursesa, supervising research, being national assessor, committee participation, information sharing at meetings
Reading journals or articles, reviewing books or articles, updating knowledge through internet research or TV, keeping a file of progress.
Identifying the most efficient and effective learning activities is essential to enable the profession to assimilate research findings and enhance clinical expertise to maximise patient outcomes. Not only would this assist with the design of educational activities, but also managers and clinicians would find this helpful when deciding which learning activities in which to invest their limited time and resources. This is important, as a lack of time has been reported by physiotherapists to be a major barrier to the implementation of evidence-based practice [5–8]. Further, there is a large industry dedicated to formal CPD activities . Indeed, a recent Australian Physiotherapy Association publication advertised 95 short courses by 20 different providers, with some courses running up to 4 weeks and costing over AU$6000 . Understanding which learning activities provide the greatest impact on physiotherapists’ knowledge, expertise and competence as well as the outcomes in their patients is important given the significant costs associated with these courses.
To date, systematic reviews on the effectiveness of CPD provide limited guidance on the most efficacious models for physiotherapists. Reviews focusing on formal learning including courses, audit, feedback and educational outreach report small effect sizes for patient outcomes and professional performance with only moderate effect sizes for changes in knowledge. [11–14]. However, these systematic reviews did not include the full spectrum of learning activities omitting informal CPD [11–13], excluded qualitative research designs and were not specific to the physiotherapy profession [11–14]. These reviews also omitted qualitative research, so do not capture the possible experiences of the learners. As learning can be a highly individualised experience, understanding the possible participant’s interpretations of these experiences is important to fully understand learning activities.
One review included qualitative research designs and compared different models of education; however, that review only examined knowledge translation interventions for allied health practitioners, including but not limited to physiotherapists . This review found equivocal results regarding effectiveness of knowledge translation interventions for professional and patient outcomes . Again, knowledge translation activities included only formal learning activities such as workshops and outreach visits. Further, this review had a limited search strategy focusing on translation of research into practice Thus, a review with a comprehensive search strategy that includes both formal and informal learning activities is required to provide the foundation for determining which learning activities are best suited to improving clinical expertise and patient outcomes. This systematic review will address the question ‘Which learning activities enhance physiotherapists’ practice?’
This systematic review protocol is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols (PRISMA-P) checklist  and has been registered on International Prospective Register of Systematic Reviews (PROSPERO, CRD42016050157).
The aim of this systematic review is to evaluate which learning activities enhance physiotherapy practice. Enhancement of physiotherapy practice will be demonstrated by changes in knowledge, skills, behaviour, self-efficacy, work satisfaction and patient outcomes. As individuals may respond differently to learning activities, the possible experiences, attitudes and beliefs of physiotherapists with regard to the various learning activities will also be explored.
To be included in this review, studies need to be published in peer reviewed journals or higher degree dissertations. Eligible studies are required to be published in English language. However, there will be no limits on language. Instead, the number of articles excluded due to non-English language will be reported during the selection process. This will give an appreciation of the articles excluded due to language. Studies will not be excluded due to geography or publication date. As this is a new and emerging area of research, both quantitative and qualitative research designs will be included.
Studies will be eligible for inclusion if qualified physiotherapists/physical therapists are participants. Studies where physiotherapists are included with other health professionals (e.g. occupational therapists) will be included if physiotherapy data is reported as a separate subgroup and the educational intervention is considered directly relevant to physiotherapy practice. Studies with participants who are in the process of completing an entry-level physiotherapy/physical therapy degree will be excluded.
Included studies will need to have participants’ complete formal or informal learning activities that aim to enhance physiotherapy practice. Formal activities include online or face to face short CPD courses, conferences, structured performance feedback, structured electronic reminders (e.g. twitter feeds), structured workplace mentoring and outreach programs (where educator goes to the workplace) and completion of post professional courses towards a degree. Informal activities include talking to colleagues/peers, reflection, committee participation, surfing the internet and reading.
Quantitative studies with comparison groups of no educational intervention as a control or comparison groups with a differing educational intervention will be included. Observational pre-post test design studies will be also be included. Qualitative studies will not be required to have a comparison group.
Studies will be eligible if they include any outcomes that evaluate change in clinician’s knowledge, skills, behaviour, self-efficacy and work satisfaction. Skills include manual therapy, decision-making, communication and clinical reasoning. The behaviour outcome pertains to how the skills are used in clinical practice. Studies which explore the experiences, attitudes and/or beliefs of the physiotherapist learner will be eligible. Studies will also be eligible if they include patients’ outcomes using reliable and valid measures for that population. Examples of patient outcomes include pain, function and satisfaction with care. Published outcome measure studies using the Consensus-based Standards of health status Measurement Instruments (COSMIN) checklist, will be referred to when determining whether the outcomes used in the studies are reliable and valid .
The following electronic databases will be searched from inception to March 2017: Ovid MEDLINE, EMBASE, AMED, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO (Psychological Abstracts), PEDro, Cochrane library and Educational Resources and Information Center (ERIC). PubMed will not be searched, as it is the same database as MEDLINE. Citation searching will be completed for included studies using the Science Citation Index Expanded via the Web of Science. Reference lists of included studies will be hand searched for additional included studies.
Data management software that will be used in this review will be Endnote 7 and Microsoft Excel (Office 2013).
The search strategy used will be developed in consultation with a research librarian employed at the administering institute. Highly sensitive search strategies will be developed by combining index terms and text words relevant to three concepts based on the research questions. The following concepts are: (1) physiotherapists, (2) learning activities and (3) clinician or patient outcomes. Clinician outcomes refer to clinician knowledge, skills, behaviour, knowledge, self-efficacy, work satisfaction, experiences, attitudes, thoughts, feelings and beliefs. An example of full search strategies is provided in Appendix 1.
Studies will be imported from the databases into an Endnote 7 file. Duplicates will be removed, and then the data will be imported into an excel spreadsheet. Data imported will be author, title, journal and year. Two independent researchers (EL, FB) will screen the titles and abstracts for inclusion using the predetermined inclusion criteria. This will be recorded on the excel spreadsheet, which will have columns for included, excluded or unsure. Full-text publications will be sourced for those studies labelled as ‘included’ and ‘unsure’. These will be evaluated for eligibility using the inclusion criteria and recorded on the excel spreadsheet with columns ‘inclusion,’ ‘exclusion,’ and ‘reason for exclusion’. The reason for exclusion column will be broken up further into ‘review’, ‘not English’, ‘pre-qualified students’, ‘data not able to be extracted separate from other health professionals’ and ‘other’. Disagreements at any point during the selection process will be resolved by consensus or a third reviewer (LC).
Data collection process
An Excel spreadsheet will be piloted on ten studies for data extraction. Data extraction will be completed by one researcher (EL) and verified by two other researchers (FB, LC).
Data extracted will include author, year, study design, sample size, participant characteristics, funding sources, location, setting, context, timeframe, design, results from studies and educational intervention details. Educational intervention details will include whether learning styles and preferences have been considered in the educational design. For quantitative studies, outcome data extracted will include statistically significant results, means and confidence intervals. For qualitative studies, results such as themes will be extracted. Where data are not clearly reported, the original authors will be contacted to clarify missing details.
Outcomes and prioritisation
Primary outcomes to be collected will be any changes in clinician’s knowledge, skills, behaviour, self-efficacy and work satisfaction. Secondary outcomes to be collected will be valid and reliable measures that evaluate patient change. Qualitative data such as the experiences, thoughts and feelings of clinicians reported as themes and quotes will also be collected related to the primary and secondary outcomes.
Risk of bias in individual studies
Two independent reviewers (EL, FB) will assess the methodological quality and risk of bias of included studies. Any disagreements will be settled by discussion with a third reviewer (LC) making the final decision if required. If greater than 80% of the quantitative studies are randomised controlled trials, then the PEDro scale  will be used. The PEDro scale is a reliable  and valid  tool for the assessment of clinical trials. While primarily used for the evaluation of randomised controlled trials, it has also been used in a recent systematic review for non-randomised clinical trials to give an appreciation of relative quality of the different study types . This approach allows for a clearer picture of the relative quality of included studies and eliminates the need for using multiple quality scales for different quantitative study types. The PEDro scale consists of 11 items, 10 of which address internal validity components and one which addresses external validity. The scale’s internal validity items cover randomisation, allocation concealment, baseline comparability, blinding, dropouts, intention-to-treat analysis, statistical comparisons as well as reporting of point estimates and variability data. Individual studies are rated as to whether or not they satisfy each of the items. The internal validity items are added to give a final methodological quality score out of 10. PEDro rating results will be communicated in this review via a table which will indicate which items have been satisfied and which have not within each study reporting quantitative data. The final score will also be indicated on this table. If less than 80% of quantitative studies are randomised controlled trials, then a validated and reliable generic risk of bias tool will be used such as the Downs and Black Checklist .
The Quality Assessment for Qualitative Research Reports (QAQRR) scale will be used to evaluate risk of bias in qualitative studies . The QAQRR has face validity and has been used in an educationally focussed peer reviewed systematic review . The scale assesses 24 items related to risk of bias in qualitative studies. These items included study design, recruitment methods, sampling strategies, data collection, analysis, disclosure, results and discussion reporting.
Cohen’s Kappa will be calculated to determine the level of inter-rater agreement for risk of bias assessment in both individual qualitative and quantitative studies.
Studies will be grouped into qualitative and quantitative studies, which will be analysed separately. Within qualitative and quantitative groups, results for each learning activity will be described. Where appropriate, the qualitative and quantitative will be combined and triangulated in a narrative synthesis.
Quantitative data synthesis
Extracted results from quantitative studies will be combined in a meta-analysis where sufficient homogeneity is present. Review Manager 5 software will be used to complete a meta-analysis using a random or fixed effects model, depending on the results from the I 2 test for heterogeneity . A randomised effects model will be used if I 2 is greater than 50%. If I 2 is 50% or less, then a fixed effects model will be used. Heterogeneity will be assessed by considering variability in participant, patient, intervention and outcomes in each study. Where significant heterogeneity is present, study results will not be combined in a formal meta-analysis. Instead, a narrative description of the results will be provided.
Effect sizes will be calculated where possible for each subgroup and communicated in a table to allow for comparison. For continuous outcomes, standardised mean differences with 95% confidence interval (CI) will be reported and analysed. For dichotomous outcomes risk ratio with 95% CI or absolute risk difference with 95% CI will be reported and analysed.
Qualitative data synthesis
Where possible, synthesis of data from qualitative studies will be conducted using a thematic analysis approach . Two researchers will independently identify key themes with supporting evidence regarding experiences and thoughts pertaining to professional development courses. Any disagreements will be resolved by consensus or through a third reviewer.
To evaluate whether reporting bias is present, reported trials will be compared to their registered protocols, where available, to check if selective reporting of outcomes has occurred.
Confidence in cumulative evidence
For quantitative data, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach will be used to report on the overall quality of evidence for each outcome . The strength of evidence will be assessed across the domains of type of evidence, risk of bias, consistency, directness, precision and publication bias . Overall quality of the evidence will be rated as either high, moderate, low or very low . Published worksheets will be completed to assist with the GRADE rating of each outcome .
For qualitative data, the GRADE-Confidence in the Evidence from Reviews of Qualitative research (CERQual) approach will be used to report the overall confidence of the evidence . The confidence in the evidence will be assessed across the domains of methodological limitations, relevance, coherence and adequacy of data. The overall confidence in the evidence will be rated as either high, moderate, low or very low. A table will be used to summarise key assessment determinations and confidence in the evidence.
Confidence in the Evidence from Reviews of Qualitative research
Consensus-based Standards of health status Measurement Instruments
Continuing professional development
Grading of Recommendations Assessment, Development, and Evaluation
Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols
Quality Assessment for Qualitative Research Reports
Availability of data and materials
All authors contributed to the development and writing of this manuscript. EL is the primary author and guarantor of the review. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Kamper SJ, Moseley AM, Herbert RD, Maher CG, Elkins MR, Sherrington C. 15 years of tracking physiotherapy evidence on PEDro, where are we now? Br J Sports Med. 2015;49(14):907–9.View ArticlePubMedGoogle Scholar
- Physiotherapy Board of Australia. Guidelines for continuing professional development; 2010. http://www.physiotherapyboard.gov.au/Codes-Guidelines/FAQ/CPD-resources.aspx Accessed 6 Aug 2016
- French H, Dowds J. An overview of continuing professional development in physiotherapy. Physiotherapy. 2008;94(3):190–7.View ArticleGoogle Scholar
- Health and Care Professionals Council. Continuing professional development and your registration. In.; 2015.
- Iles R, Davidson M. Evidence based practice: a survey of physiotherapists' current practice. Physiother Res Int. 2006;11(2):93–103.View ArticlePubMedGoogle Scholar
- Ramirez-Velez R, Bagur-Calafat MC, Correa-Bautista JE, Girabent-Farres M. Barriers against incorporating evidence-based practice in physical therapy in Colombia: current state and factors associated. BMC Med Educ. 2015;15:220.View ArticlePubMedPubMed CentralGoogle Scholar
- Scurlock-Evans L, Upton P, Upton D. Evidence-based practice in physiotherapy: a systematic review of barriers, enablers and interventions. Physiotherapy. 2014;100(3):208–19.View ArticlePubMedGoogle Scholar
- da Silva TM, Costa Lda C, Garcia AN, Costa LO. What do physical therapists think about evidence-based practice? A systematic review. Man Ther. 2015;20(3):388–401.View ArticlePubMedGoogle Scholar
- Chipchase LS, Johnston V, Long PD. Continuing professional development: the missing link. Man Ther. 2012;17(1):89–91.View ArticlePubMedGoogle Scholar
- Australian Physiotherapy Association. In: Physiotherapy In-Motion. vol. June. Australia; 2015.
- Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;15(2):Cd003030.Google Scholar
- Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:Cd000259.Google Scholar
- Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof. 2007;27(1):6–15.View ArticlePubMedGoogle Scholar
- O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;17(4):Cd000409.Google Scholar
- Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, Jones CA, Klassen TP, Kovacs Burns K, Newton AS, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:70.View ArticlePubMedPubMed CentralGoogle Scholar
- Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.View ArticlePubMedGoogle Scholar
- Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, Bouter LM, de Vet HC. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22.View ArticlePubMedPubMed CentralGoogle Scholar
- de Morton NA. The PEDro scale is a valid measure of the methodological quality of clinical trials: a demographic study. Aust J Physiother. 2009;55(2):129–33.View ArticlePubMedGoogle Scholar
- Maher CG, Sherrington C, Herbert RD, Moseley AM, Elkins M. Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther. 2003;83(8):713–21.PubMedGoogle Scholar
- Pritchard SA, Blackstock FC, Nestel D, Keating JL. Simulated patients in physical therapy education: systematic review and meta-analysis. Phys Ther 2016;96(9):1342–53.View ArticlePubMedGoogle Scholar
- Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377–84.View ArticlePubMedPubMed CentralGoogle Scholar
- Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60.View ArticlePubMedPubMed CentralGoogle Scholar
- Bearman M, Dawson P. Qualitative synthesis and systematic review in health professions education. Med Educ. 2013;47(3):252–60.View ArticlePubMedGoogle Scholar
- Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.View ArticlePubMedGoogle Scholar
- Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6.View ArticlePubMedGoogle Scholar
- Effective Practice and Organisation of Care (EPOC). EPOC worksheets for preparing a Summary of Findings (SoF) table using GRADE. EPOC Resources for review authors..Oslo: Norwegian Knowledge Centre for the Health Services; 2013. http://epoc.cochrane.org/epoc-specific-resources-review-authors Accessed 17 May 2017
- Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gulmezoglu M, Noyes J, Booth A, Garside R, Rashidian A. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.View ArticlePubMedPubMed CentralGoogle Scholar
- Physiotherapy Board of New Zealand. How to complete your professional development report; 2016. http://www.physioboard.org.nz/sites/default/files/PDRGuide2013-2016.pdf Accessed 5 Apr 2016