A systematic review protocol to evaluate the psychometric properties of measures of function within adult neuro-rehabilitation
Systematic Reviews volume 4, Article number: 86 (2015)
Spasticity in the upper limb is common after acquired brain impairment and may have a significant impact on the ability to perform meaningful daily activities. Traditionally, outcome measurement in spasticity rehabilitation has focused on impairment, however, improvements in impairments do not necessarily translate to improvements in an individual’s ability to perform activities or engage in life roles. There is an increasing need for outcome measures that capture change in activity performance and life participation.
We will conduct a systematic review of the psychometric properties of instruments used to measure upper limb functional outcomes (activity performance and participation) in patients with spasticity. Assessments (n = 27) will be identified from a recently published systematic review of assessments that measure upper limb function in neurological rehabilitation for adults with focal spasticity, and a systematic review of each assessment will then be conducted. The databases MEDLINE, CINAHL and EMBASE will be searched from inception. Search strategies will include the name of the assessment and the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) published search strategy for identifying studies of measurement properties. The methodological rigour of the testing of the psychometric quality of instruments will be undertaken using the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist. International Classification of Functioning, Disability and Health (ICF) definitions of impairment, activity and participation will be used for content analysis of items to determine the extent to which assessments are valid measures of activity performance and life participation. We will present a narrative synthesis on the psychometric properties and utility of all instruments and make recommendations for assessment selection in practice.
This systematic review will present a narrative synthesis on the psychometric properties and utility of assessments used to evaluate function in adults with upper limb focal spasticity. Recommendations for assessment selection in practice will be made which will aid clinicians, managers and funding bodies to select an instrument fit for purpose. Importantly, appropriate assessment selection will provide a mechanism for capturing how applicable to everyday life the outcomes from individualised rehabilitation programs for the upper limb really are.
Systematic review registration
Spasticity is most commonly accepted as being a velocity-dependent increase in stretch reflex with exaggerated tendon jerks that occurs as one part of the upper motor neurone syndrome . Pandyan et al  more recently redefined spasticity as disordered sensorimotor control, resulting from an upper motor neurone lesion, presenting as intermittent or sustained involuntary activation of muscles. In recent years, there has been a recognition within neuro-rehabilitation that spasticity management programs must go well beyond treatment of impairments, in line with contemporary understandings of health emerging from the World Health Organisation’s International Classification of Functioning, Disability and Health Framework (ICF) . This framework starts with the assumption that health is not a state independent of individuals in the context of everyday life; thus spasticity, a neuro-muscular condition, cannot be considered independent of the person who has it and their daily life. This makes understanding, measuring and monitoring the impact of neuro-rehabilitation programs on function in everyday life as important as measuring and monitoring spasticity.
Contemporary neuro-rehabilitation uses the ICF as an organising framework. Apart from the environment, there are three domains in the classification—impairment, activity and participation . The ICF defines “impairment” as a problem in body (physiological and psychological) function or structure (anatomical parts of the body), “activity” is the execution of a task or action by an individual and “participation” is involvement in a life situation . In upper limb rehabilitation, function is an important and debated term—because impairments can affect function and it is through function that activity and participation goals can be achieved. The term function is used variably within the literature; it alludes to impairments, activity performance and/or participation in life situations in addition to associations with active task performance. Whilst concepts associated with “function” can vary, operationally, functional use of the spasticity-affected upper limb has been defined by Ashford and Turner-Stokes . That is: active task performance; the affected limb actively completes the task or passive task performance; the task is completed by the affected limb with assistance from the unaffected limb or the task is assisted with or completed by a carer; a key area for spasticity interventions . This three-part operational definition of upper limb function is used in the present study.
Multi-disciplinary person-centred approaches are needed to address rehabilitation needs at impairment, activity and participation levels [5–7]. Neuro-rehabilitation clinical practice guidelines recommend collaborative goal setting [7–10], so that patient preferences and priorities can inform programs. Practice guidelines also recommend the use of standardised assessments to measure impairment, activity and participation dimensions of performance relevant to everyday real life [4, 6, 10, 11]. Although most rehabilitation clinicians measure treatment outcomes , evidence suggests that many have limited awareness of the range of assessments available . Those who do use assessment use predominantly impairment-based measures—few use measures that capture activity or participation performance [6, 14]. There are measures available. Although an earlier systematic review of functional outcome measures in the hemiparetic upper limb was conducted in 2008 , this study was unable to identify a single valid and reliable outcome measure that captured “real life” function. But a more recent review by Ashford and Turner-Stokes in 2013  identified n = 27 functional assessments used in upper limb neuro-rehabilitation for people with spasticity (with and without botulinum toxin-A injection). Their inclusion criteria required the assessments to explore function in the context of everyday real life. As yet, these assessments have not been appraised in relation to the psychometric rigour or clinical utility [4, 16]. This study aims to fill that gap.
This study will use the same n = 27 assessments from the Ashford and Turner-Stokes study  to investigate the psychometric properties of each and draw conclusions regarding their relative rigour and relevance. A key focus will be the validity of these assessments in their ability to capture change in activity performance and life participation. The ICF will be used as the framework to appraise assessment content to determine the extent to which items address activity and participation domains in addition to impairment (body structures and function). Determining the content validity of items in relation to these domains is important not only to see how valid the assessment is in measuring “health” as it is defined by the ICF but also because these domains reflect common patient goals.
Common neuro-rehabilitation goals for people with upper limb spasticity include reducing pain, increasing the range of movement, preventing contractures and reducing spasticity to enable movement training, splinting or casting [6, 12, 17]. Other goals relate to increasing a person’s ability to perform activities and participate in their life situation [6, 11, 16, 18–20]. To date, no systematic review has done this.
Outcomes of this review will be helpful to clinicians and researchers alike working with people who have upper limb spasticity. Right now, some attention has been given to neuro-rehabilitation for people with upper limb spasticity on function in everyday real life [11, 16], but it is a relatively new focus of program evaluation [4, 10, 11] and a challenging one [17, 20, 21]. Determining whether or not interventions impact functional outcomes in everyday life for people with upper limb spasticity has, to date, been complicated by methodological problems, not just in relation to function but also spasticity measurement [15, 22] and the use of weak study designs [17, 21]. There is a need for more research to show that multidisciplinary upper limb spasticity management rehabilitation programs have an impact on the ability of people to perform activities in everyday real life [11, 23, 24].
The objectives of this systematic review are to locate and appraise existing evidence of the psychometric properties of the outcome measures identified by Ashford and Turner-Stokes  and classify those measures to conclude the best tool available for the purpose of measuring activity performance and participation outcomes following upper limb spasticity rehabilitation.
This systematic review builds on the systematic search conducted by Ashford and Turner-Stokes  by synthesising and appraising the research of the psychometric (measurement) properties of outcome measures reported within the published paper. From the 22 studies located in the published search , n = 33 assessment approaches were identified. On review of those assessment approaches, some were in fact developed for that particular study, for example, three functional tasks (palm hygiene, cutting the fingernails, placing an arm through the sleeve), and consequently do not have published psychometric properties and were excluded from the current study. The remaining n = 27 outcome measures had published research investigating their psychometric (measurement) properties and will therefore form the sample for the present study. The authors acknowledge the creation of a degree of assessment selection bias due to this method.
The aims of this systematic review will be:
To classify the functional outcome measures reported by Ashford and Turner-Stokes  according to whether activity and or participation outcomes following upper limb spasticity rehabilitation are being assessed; activity performance and participation will be defined according to the ICF model ; and
To locate all of the existing evidence of the properties of the outcome measures, to evaluate the strength of this evidence and to come to a conclusion about the best measure available for the particular purpose of measuring activity and/or participation outcomes following upper limb spasticity rehabilitation.
Publication/study inclusion criteria
The aim of the study should be to develop or evaluate the measurement properties of a measurement instrument identified in the review published by Ashford and Turner-Stokes ;
The instrument should aim to measure activity performance or participation, as defined by the ICF 
Activity performance is defined as “the execution of a task or action by an individual” or requires assistance from or be completed by a carer for the individual. Participation is defined as “involvement in a life situation.”
The instrument is evaluated in adult patients, over 18 years of age, with upper limb spasticity (as defined by the authors of the included studies) or patients before or after botulinum toxin injection engaging in upper limb rehabilitation programs (with or without the inclusion of botulinum toxin therapy). A rehabilitation program is one that is devised and implemented by a clinician to work towards achievement of identified goals. Participants can be engaging in the rehabilitation program whilst a hospital inpatient, transitioning to home or be community dwelling.
All research studies must be original research, and both conducted and published studies in English within peer-reviewed literature will be considered for this review.
Publication/study exclusion criteria
This review is concerned with outcomes of upper limb spasticity rehabilitation that identify changes in the performance of an activity or participation as defined by the ICF . Studies that measure activity performance and participation will be included. Studies that measure upper limb spasticity rehabilitation outcomes through assessment of upper limb impairments only, including pain, range of movement, contracture and changes in tone, will be excluded. Outcomes that have been modified in any manner or implemented in a language other than English will be excluded.
Search methods for the identification of studies
A search will be conducted in Medical Literature Analysis and Retrieval System Online (MEDLINE), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Excerpta Medica dataBASE (EMBASE). In MEDLINE, a validated search filter for finding studies on measurement properties will be used ; we report the planned MEDLINE search strategy in Additional file 1. CINAHL and EMBASE search strategies are available from the authors on request. Searches with the names of each included instruments (in the title) in combination with the terms for the study population as described in the search strategy [see Additional file 1] will be conducted until each instrument has been searched.
Once all searches have been exhausted, the abstracts will be downloaded into the reference management system EndNote and duplicates deleted. A study deemed as a duplicate will have authors, setting and location, outcome measures implemented, date and duration of study in common.
The eligibility criteria will first be applied to the title and abstract, and if deemed relevant, the full manuscript will be retrieved to determine eligibility of potential studies. The initial screen and selection will be completed independently by the first author with the second author blindly screening a 10 % selection of articles for eligibility. Debate on the inclusion or exclusion of studies will be resolved by an independent third reviewer and discussion between all three reviewers to reach consensus.
Details on studies that were initially selected based on title and abstract, full-text articles that were retrieved and articles included in the review will be documented. Reasons for the exclusion of retrieved full-text articles, particularly in the case of doubtful articles, will also be recorded.
Data will be extracted from selected studies by the first author utilising a standardised data extraction form. This form will record information related to participants, study design, description of botulinum toxin therapy and rehabilitation program (s), outcome measures administered and their classification according to the ICF (activity performance and or participation focus), psychometric properties, study inclusion/exclusion criteria if available and a brief summary of the findings. The second author will crosscheck all COSMIN ratings.
Risk of bias assessment
Studies evaluating the measurement properties of an assessment require high methodological quality with a low risk bias to guarantee that appropriate conclusions are drawn about the properties of the measure . Thus, it is important to evaluate those methodological qualities . This review will apply the COnsensus-Based Standards for the Selection of Health Status Measurement INstruments (COSMIN) checklist with 4-point scale version . This version is recommended by the COSMIN developers for use in systematic reviews of measurement properties. The checklist will be applied to assess the quality of the papers reporting on the psychometric properties of the 27 outcome measures, evaluating whether each study meets the standards for methodological quality with regard to internal consistency, reliability (test-retest, inter-rater and intra-rater reliability), measurement error, content validity (including face validity), structural validity, hypothesis testing, cross-cultural validity, criterion validity, responsiveness, interpretability and generalisability . The 4-point scale will allow a methodological quality rating of either “excellent”, “good”, “fair” or ”poor” to be assigned to the study . The COSMIN checklist was developed in an international Delphi study with the focus of evaluating the methodological quality of studies on measurement properties . The COSMIN checklist is a modular tool, and the measurement properties evaluated in the study will determine which components or “boxes” need to be completed .
Individual assessment items within the outcome measures will be examined to extract meaningful concepts. Those concepts will then be linked to the ICF framework categories of activity performance and or participation following the linking rules suggested by Cieza et al.  [see Additional file 2]. This linking process will enable the extent to which outcomes are valid measures of activity performance and life participation to be determined.
The COSMIN checklist with 4-point scale version  as described above will be applied to the selected studies, as per COSMIN guidelines, to appraise the overall methodological quality of studies. From here, Terwee’s quality criteria for measurement properties  will be applied. Quality criteria for the following nine measurement properties are defined; content validity, internal consistency, criterion validity, construct validity, reproducibility, reliability, responsiveness, floor and ceiling effects and interpretability. This data analysis process will enable conclusions to be drawn regarding the strongest psychometric measure available for the particular purpose of evaluating activity and/or participation outcomes following upper limb rehabilitation. Differences in the psychometric properties of outcome measures for patients with and without upper limb spasticity will be discussed.
This systematic review will provide a comprehensive evaluation of the measurement properties of outcome measures assessing activity performance and participation goals for adults with upper limb spasticity undergoing rehabilitation. The results of this review will provide health professionals with detailed information to guide clinical decision-making when choosing the most appropriate outcome measure for occupational performance. Rehabilitation clinicians and managers will also be provided with information to permit accurate assessment and monitoring of the relationship between rehabilitation and health outcome in these patients.
Cumulative Index to Nursing and Allied Health Literature
Consensus-based Standards for the Selection of Health Measurement Instruments
Excerpta Medica dataBASE
International Classification of Functioning, Disability and Health
Medical Literature Analysis and Retrieval System Online
Lance JW. Symposium synopsis. In: Feldman RG, Young RR, Koella WP, editors. Spasticity: Disordered Motor Control. Miami: Symposia Specialists; 1980. p. 485–94.
Pandyan AD, Gregoric M, Barnes MP, Wood D, Van Wijck F, Buridge J, et al. Spasticity: clinical perception, neurological realities and meaningful measurement. Disabil Rehabil. 2005;27(1/2):2–6.
World Health Organisation. International Classification of Functioning, Disability and Health. Geneva: World Health Organisation; 2001.
Ashford S, Turner-Stokes L. Systematic review of upper-limb function measurement methods in botulinum toxin intervention for focal spasticity. Physiother Res Int. 2013;18(3):178–89. doi:10.1002/pri.1554.
Demetrios M, Khan F, Turner-Stokes L, Brand C, McSweeney S. Multidisciplinary rehabilitation following botulinum toxin and other focal intramuscular treatment for post-stroke spasticity (review). Cochrane Database Syst Rev. 2013;6:CD009689. doi:10.1002/14651858.CD009689.pub2.
Sheean G, Lannin NA, Turner-Stokes L, Rawicki B, Snow BJ. Botulinum toxin assessment, intervention and after-care for upper limb hypertonicity in adults: international consensus statement. Eur J Neurol. 2010;17 Suppl 2:74–93.
Royal College of Physicians, British Society of Rehabilitation Medicine, Chartered Society of Physiotherapy, Association of Chartered Physiotherapists Interested in Neurology. Spasticity in adults: management using botulinum toxin. National guidelines. London: RCP; 2009.
Turner-Stokes L. Goal attainment scaling (GAS) in rehabilitation: a practical guide. Clin Rehabil. 2009;23:362–70. doi:10.1177/0269215508101742.
Playford ED, Siegert R, Levack W, Freeman J. Areas of consensus and controversy about goal setting in rehabilitation: a conference report. Clin Rehabil. 2009;23:334–44. doi:10.1177/0269215509103506.
Rosewilliam S, Roskell CA, Pandyan AD. A systematic review and synthesis of the quantitative and qualitative evidence behind patient-centred goal setting in stroke rehabilitation. Clin Rehabil. 2011;25(6):501–14.
Turner-Stokes L, Fheodoroff K, Jacinto J, Maisonobe P. Results from the Upper Limb International Spasticity Study-II (ULISII): a large international, prospective cohort study investigating practice and goal attainment following treatment with botulinum toxin A in real-life clinical management. BMJ Open. 2013;3:e002771.
Ward AB, Aguilar M, De Beryl Z, Gedin S, Kanovsky P, Molteni F, et al. Use of botulinum toxin type A in management of adult spasticity—a European consensus statement. J Rehabil Med. 2003;35:98–9.
William G, Olver J, de Graaff S, Singer BJ. The use of botulinum toxin type A in the management of adult-onset focal spasticity: a survey of Australian allied health professionals. Aust Occup Ther J. 2012;59:257–64.
Bakheit AM, Zakine B, Maisonobe P, Aymard C, Fhedoroff K, Hefter H, et al. The profile of patients and current practice of treatment of upper limb muscle spasticity with botulinum toxin type A: an international survey. Int J Rehabil Res. 2010;33(3):191–204.
Ashford S, Slade M, Malaprade F, Turner-Stokes L. Evaluation of functional outcome measures for the hemiparetic upper limb: a systematic review. J Rehabil Med. 2008;40:787–95.
Intiso D, Simone V, Di Rienzo F, Santamato A, Russo M, Tolfa M, et al. Does spasticity reduction by botulinum toxin type A improve upper limb functionality in adult post-stroke patients? A systematic review of relevant studies. J Neurol Neurophysiol. 2013;4(4):167.
Sheean GL. Botulinum treatment of spasticity: why is it so difficult to show a functional benefit? Curr Opin Neurol. 2001;14:771–6.
Elia AE, Filippini G, Calandrella D, Albanese A. Botulinum neurotoxins for post-stroke spasticity in adults: a systematic review. Mov Disord. 2009;24(6):801–12.
Cusick A, Lannin NA, Kinnear B: Upper limb spasticity management for patients who have received botulinum toxin A injection: Australian therapy practice. Australian Occupational Therapy Journal, in press.
Turner-Stokes L, Baguley IJ, de Graaff S, Katrak P, Davies L, McCrory P, et al. Goal attainment scaling in the evaluation of treatment of upper limb spasticity with botulinum toxin: a secondary analysis from a double-blind placebo controlled randomised clinical trial. J Rehabil Med. 2010;42:81–9.
Hinderer SR, Gupta S. Functional outcome measures to assess interventions for spasticity. Arch Phys Med Rehabil. 1996;77:1083–9.
Platz T, Eickhof C, Nuyens G, Vuadens P. Clinical scales for the assessment of spasticity, associated phenomena, and function: a systematic review of the literature. Disabil Rehabil. 2005;27(1/2):7–18.
Simpson DM, Gracies JM, Graham HK, Miyasaki JM, Naumann M, Russman B, et al. Assessment: Botulinum neurotoxin for the treatment of spasticity (an evidence-based review): report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology. Neurology. 2008;70:1691–8.
Shaw LC, Price CIM, van Wijck FMJ, Shackley P, Steen N, Barnes MP, et al. Botulinum toxin for the upper limb after stroke (BoTULS) trial: effect on impairment, activity limitation, and pain. Stroke. 2011;42:1371–9.
Terwee CB, Jansma EP, Riphagen II, de Vet HC. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009;18:1115e23.
De Vet HC, Terwee CB, Mokkink LB, Knol DL. Systematic reviews of measurement properties. In: De Vet HCW, Terwee CB, Mokkink LB, Knol DL, editors. Measurement in Medicine: A practical guide. New York: Cambridge University Press; 2011. p. 275–314.
Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7. doi:10.1007/s11136-011-9960-1.
Cieza A, Geyh S, Chatterji S, Kostanjsek N, Ustun B, Stucki G. ICF linking rules: an update based on lessons learned. J Rehabil Med. 2005;37:212–8.
Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42.
The authors declare that they have no competing interests.
SP led the design of the protocol and drafting of the manuscript. NL contributed to the design of the protocol and search strategy and provided critical revision of the manuscript. AC contributed to the design of the protocol and critical revision of the manuscript. KW adapted the search strategy from COSMIN for the searches of EMBASE and CINAHL databases. SA and LTS contributed to the selection of assessments for review and critical revision of the manuscript. All authors read and approved the final manuscript.
The study is being conducted as part of SP’s program of postgraduate study through La Trobe University under the supervision of Associate Professor Natasha Lannin (La Trobe University) and Professor Anne Cusick (University of Wollongong).
About this article
Cite this article
Pike, S., Lannin, N.A., Cusick, A. et al. A systematic review protocol to evaluate the psychometric properties of measures of function within adult neuro-rehabilitation. Syst Rev 4, 86 (2015). https://doi.org/10.1186/s13643-015-0076-5
- Botulinum toxin
- Upper limb
- Activity performance
- Outcome measurement