- Open Access
- Open Peer Review
Determining the number of stimuli required to reliably assess corticomotor excitability and primary motor cortical representations using transcranial magnetic stimulation (TMS): a protocol for a systematic review and meta-analysis
Systematic Reviewsvolume 4, Article number: 107 (2015)
Transcranial magnetic stimulation (TMS) is a technique that can be used to assess corticospinal plasticity. Current TMS practices involve the administration of multiple stimuli over target areas of the participant’s scalp. However, these procedures require 1 to 2 h per assessment. Decreasing the number of stimuli delivered during TMS assessments would improve time efficiency and decrease participant demand. Thus, the aim of this review is to determine the number of TMS stimuli required to reliably measure (1) corticomotor excitability to a target muscle at a single cranial site and (2) the topography of the primary motor cortical representation for a target muscle across multiple cranial sites (termed ‘mapping’).
A systematic review and meta-analysis will be conducted. Electronic databases will be searched using pre-determined search terms to identify relevant studies and evaluate the studies for inclusion and risks of bias. Two independent reviewers will extract the data. Any disagreements will be resolved by a third reviewer. Studies employing single-pulse TMS to measure (1) corticomotor excitability at a single cranial site or (2) the topographic cortical organisation of a target muscle across a number of cranial sites, published before May 2015, will be included if they meet the eligibility criteria. Outcomes will include motor-evoked potential amplitude, map volume, number of active map sites, location of the map centre of gravity, and distance between the centres of gravity of the target muscle and one or more neighbouring muscles.
To our knowledge, this review will be the first to systematically explore the number of TMS stimuli required to reliably measure both corticomotor excitability and the topography of primary motor cortical representations. This research has the capacity to improve the efficiency of TMS, decrease participant demand, and facilitate the use of TMS as an outcome measurement tool in clinical populations.
Systematic review registration
First described by Barker, Jalinous, and Freeston in 1985, transcranial magnetic stimulation (TMS) is a non-invasive, safe, and painless technique used to stimulate the human brain . Initially, TMS was used to investigate central nervous system integrity among neurological populations . However, over the last 15 years, TMS has evolved to include a variety of broader applications. Such applications range from pre-surgical tumour mapping to exploration of cortical reorganisation in pathological populations [2, 3]. Transcranial magnetic stimulation has also been shown to be a valuable tool for predicting functional recovery and investigating the neurophysiological effects of various rehabilitation approaches [4, 5]. However, the time-consuming nature of TMS limits its applicability in clinical practice .
During TMS, a stimulation coil introduces a magnetic field over the target area of the participant’s scalp, inducing a secondary electrical current within cortical tissue . In the case of the motor cortex, the electrical current triggers action potential propagation down the corticospinal tract to contralateral peripheral motor neurons, eliciting a motor response . Electrical potentials generated by muscle cells following motor cortex stimulation are measured as motor-evoked potentials (MEPs) using electromyography. Changes in MEPs over time suggest alterations to the structure and function of a participant’s neuronal network [6, 7].
Current TMS practices involve the administration of multiple stimuli over target areas of the participant’s scalp [3, 8]. Depending on the desired neurophysiological index, stimuli are either applied at a single cranial site, or systematically over a predefined grid, in a process known as mapping. Stimuli are delivered as single pulses, typically with 3 to 5 s between each stimulus . During mapping, four to five stimuli are commonly administered to each cranial site [3, 8]. Single-site analyses at the optimal site, known as the ‘hotspot’, typically utilise ten stimuli [9, 10]. Recording the mean outcome of repeated trials is thought to enhance reliability and enable identification of potential outliers . Although these TMS procedures have been validated using magnetic resonance imaging (MRI), the International Federation of Clinical Neurophysiology highlights that current practices are inefficient and that recommendations were determined arbitrarily .
Administering five stimuli per cranial site to map the cortical representation of a particular muscle requires 1 to 2 h per assessment, restricting its use beyond the research environment . Further, prolonged TMS assessments have been associated with significant increases in participant fatigue and discomfort . This becomes particularly important when investigating neurological populations, where increased metabolic demands and poor baseline levels of endurance may limit adherence . Similarly, prolonged assessments are impractical when exploring conditions involving chronic pain . The importance of reducing data collection time is also evident from the observation that corticospinal excitability fluctuates with participant arousal and concentration, which may vacillate throughout prolonged investigations .
Reducing the number of stimuli delivered to each cranial site during TMS has the potential to improve the efficiency of the procedure while decreasing participant demand . However, lowering the number of stimuli per site may also compromise the procedure’s reliability . No reviews have systematically examined the reliability of outcomes obtained using varying numbers of stimuli per cranial site during TMS . Here, we present the protocol for a review that aims to determine the number of TMS stimuli required to reliably measure (1) corticomotor excitability to a target muscle at a single cranial site and (2) the topography of the primary motor cortical representation for a target muscle across multiple cranial sites (termed ‘mapping’).
This protocol was prepared in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P)  [see Additional file 1] and a measurement tool to assess systematic reviews (AMSTAR) . The protocol has been registered with the International Prospective Register of Systematic Reviews (PROSPERO; registration number CRD42015024579).
How many TMS stimuli are required to reliably measure (1) corticomotor excitability to a target muscle at a single cranial site and (2) the topography of the primary motor cortical representation for a target muscle across multiple cranial sites?
The methods for this systematic review were developed using items from Chapter 13 of the Cochrane Handbook for Systematic Reviews relevant to the reporting of systematic reviews of diagnostic reliability studies . The search strategy was developed in consultation with a librarian with systematic review expertise.
Searches will be conducted in CINAHL, CENTRAL, EMBASE, MEDLINE, Neuroscience Information Framework (NIF), PEDro, PsycINFO, PubMed, Scopus, and Web of Science databases. No limits will be placed upon language or location of publication. Where necessary, articles will be translated to English by an independent interpreter for analysis. Keywords and medical subject headings (MeSH) related to transcranial magnetic stimulation, cortical reorganisation, and reliability will be used. The terms ‘transcranial magnetic stimulation’, ‘cortical reorganisation’, and ‘reliability’ will be used in varying combinations to identify relevant literature. Search strategies will be customised to suit each database [see Additional file 2].
Authors will contact experts in the field of TMS and search Google Scholar for additional studies. Searches of the International Clinical Trials Registry Platform (ICTRP) will be conducted to identify recently completed studies. The reference lists of all relevant articles will be analysed to identify additional trials. Studies from these sources satisfying the eligibility criteria will be included in the systematic review.
Types of participants
To reflect the widespread applicability of TMS, no restrictions will be placed upon participant age or gender. Both healthy and clinical populations will be eligible for inclusion.
Full-text articles published prior to May 2015.
Diagnostic reliability studies comparing the effect of the number of TMS stimuli on the within- or between-session reliability of either (a) corticomotor excitability at a single cranial site or (b) the topographic cortical organisation of a target muscle across multiple cranial sites. Only studies employing single-pulse TMS will be included. The number and position of targeted cranial coordinates are required to be consistent within and between sessions.
Journal or conference abstracts with no associated full-text article
Studies not exploring within- or between-session reliability
Studies investigating within- or between-session reliability of corticomotor excitability or topographic cortical representation where the number of TMS stimuli is not varied
Studies comparing the reliability of TMS outcomes obtained between cortical hemispheres, rather than within or between sessions using the same hemisphere
Studies reporting only the ‘ideal’ number of stimuli at a particular site
Studies presenting reliability predictions based upon mathematical models, rather than experimental testing, as mathematical models are unable to take into account inter-individual fluctuations in corticospinal activity 
Studies investigating the effects of interventions.
Studies employing repetitive TMS to induce, rather than measure, neuroplastic changes.
For investigations of corticomotor excitability at a single cranial site, eligible studies must report motor-evoked potential amplitude. For investigation of the topographic representation (map) of a target muscle across multiple cranial sites, eligible studies must report at least one of the following: motor-evoked potential amplitude, map volume, number of active map sites, location of the map centre of gravity (CoG), or distance between the centres of gravity of the target muscle and one or more neighbouring muscles. These indices represent those most commonly assessed during TMS studies [6, 19].
Search results will be exported to EndNote citation software for automatic duplicate removal. Any duplicates overlooked by the programme will be manually removed. Two independent reviewers will then screen the exported articles for relevance by title and abstract. Potentially relevant papers will be retrieved as full-text articles and assessed according to the eligibility criteria by these two reviewers. An additional reviewer will be consulted should there be any uncertainty or disagreement regarding the eligibility of studies. Excluded studies and reasons for exclusion will be recorded.
Participant characteristics, sample sizes, TMS protocols, neuroplastic indices, and reliability measurements will be extracted by two independent reviewers. Any disagreements will be resolved by a third reviewer. If data are missing, authors will be contacted a maximum of three times, after which the data will be considered irretrievable.
Where studies obtain outcomes in response to multiple TMS stimulation intensities, the intensity closest to 120 % of resting motor threshold (rMT) will be analysed as this value has been shown to be valid and reliable over time . Additionally, 120 % of rMT is the most commonly employed stimulation intensity during TMS investigations . Similarly, in studies using active motor thresholds (aMTs), the analysis closest to 10 % of maximal voluntary contraction will be included [20, 21].
Assessment of methodological quality
Two tiers of methodological quality assessment will be performed. First, the general experimental design of each study will be appraised using a custom appraisal tool. Second, the included studies will be assessed according to the TMS-specific checklist developed by Chipchase et al. .
General experimental design
A custom quality appraisal tool will be utilised to assess general methodological quality [see Additional file 3]. The tool will consist of relevant items from both the Quality Assessment of Reliability Studies (QAREL)  checklist and the guidelines for assessing reliability studies developed by Bialocerkowski et al. . These appraisal tools have been shown to possess good inter-rater reliability . Additionally, they provide insight into both internal and external validity [24, 25].
Two reviewers will independently assess studies that satisfy the eligibility criteria. Items will be scored as either present (1) or absent (0), and a score out of 11 will be obtained via summation. Studies will not be excluded based upon quality. In accordance with QAREL-based recommendations provided by Triano et al. , studies scoring four or less will be deemed to be of low quality, scores between five and seven will be classified as moderate quality, and studies scoring eight or above will be classified as high quality. Any disagreement will be resolved by a third reviewer.
TMS-specific protocol and reporting
The TMS checklist designed by Chipchase et al.  will be used to assess TMS-specific elements of methodological and reporting quality [see Additional file 4]. This checklist was produced via a two-round international Web-based Delphi consensus study. Appraising the TMS protocol itself will enable identification of methodological shortcomings that would not be revealed during assessments of general experimental design. Items will be marked ‘Yes’ or ‘No’ by two independent reviewers based upon whether or not requirements for the corresponding item have been satisfied. Disagreements will be resolved by a third reviewer.
Strategy for data synthesis
Reliability estimates in the form of intraclass correlation coefficients (ICCs) will be pooled using StatsDirect 3 Software. The intraclass correlation coefficient is the most accurate and commonly used indication of the size and direction of association between two variables . Intraclass correlation coefficients for within- and between-session reliability will be interpreted using the following values: less than 0.50 = poor, 0.50 to 0.65 = moderate, 0.65 to 0.80 = good, and greater than 0.80 = excellent . Confidence intervals surrounding ICCs will be calculated using StatsDirect software, and significance will be set at p < 0.05.
A random-effects model will be used during analyses, as methodological heterogeneity is inevitable during manual TMS assessments . The impact of heterogeneity will be calculated using the I2 statistic and interpreted as follows: 0 to 40 % may be unimportant, 30 to 60 % may represent moderate heterogeneity, 50 to 90 % may represent substantial heterogeneity, and 75 to 100 % represents considerable heterogeneity . In cases of substantial methodological or statistical heterogeneity, subgroup analyses will be performed to highlight potential reasons for this heterogeneity. If, following subgroup analyses, substantial statistical heterogeneity remains, results will be synthesised descriptively, rather than statistically.
Analysis of subgroups or subsets
Following initial meta-analyses, comprehensive subgroup analyses will be performed to investigate the influence of methodological inconsistencies and participant variability. Subgroup analyses will be performed according to participant age (greater than or less than 65 years), target region (facial, upper limb, or lower limb musculature), test conditions (active or resting target musculature), population (clinical or healthy), and intersession interval (greater than 72 h or less than 72 h). Additional analyses will also be performed with low-quality studies excluded to determine if general experimental design quality significantly influences the results.
To our knowledge, this review will be the first to systematically explore the number of TMS stimuli required to reliably measure (1) corticomotor excitability to a target muscle at a single cranial site and (2) the topography of the primary motor cortical representation for a target muscle across multiple cranial sites. This research has the capacity to improve the efficiency of TMS, decrease participant demand, and facilitate the use of TMS as an outcome measurement tool in clinical populations. The review will also facilitate further research into this area, identifying potential areas for development.
As database searches will be limited to full-text articles, bias may be introduced through the exclusion of data in grey literature. This ‘publication bias’ may inflate reliability estimates, as studies with desirable or significant results are more likely to be granted publication . Additionally, including only one stimulation intensity per study may reduce the external validity of this review.
Ethics and dissemination
This review does not require ethical approval. Results will be presented at scientific meetings and published in peer-reviewed journals. All publications and presentations related to the study will be authorised and reviewed by the study investigators.
The reviewers have commenced searching relevant studies on the electronic databases. This review is expected to be complete by July 2015.
a measurement tool to assess systematic reviews
centre of gravity
intraclass correlation coefficient
Meta-analysis of Observational Studies in Epidemiology
Preferred Reporting Items for Systematic Reviews and Meta-analyses
Quality Assessment of Reliability Studies
Quality Assessment of Diagnostic Accuracy Studies
transcranial magnetic stimulation
Barker AT, Jalinous R, Freeston IL. Non-invasive magnetic stimulation of human motor cortex. Lancet. 1985;325(8437):1106–7. doi:10.1016/s0140-6736(85)92413-4.
Reinacher P, Priebe H, Blumrich W, Zentner J, Scheufler K. The effects of stimulation pattern and sevoflurane concentration on intraoperative motor-evoked potentials. Anesth Analg. 2006;102(3):888–95. doi:10.1213/01.ane.0000195235.02162.5d.
Peterchev A, Wagner T, Miranda P, Nitsche M, Paulus W, Lisanby S, et al. Fundamentals of transcranial electric and magnetic stimulation dose: definition, selection, and reporting practices. Brain Stimul. 2012;5(4):435–53. doi:10.1016/j.brs.2011.10.001.
Rapisarda G, Bastings E, de Noordhout A, Pennisi G, Delwaide P. Can motor recovery in stroke patients be predicted by early transcranial magnetic stimulation? Stroke. 1996;27(12):2191–6. doi:10.1161/01.str.27.12.2191.
Stinear C. Prediction of recovery of motor function after stroke. Lancet Neurol. 2010;9(12):1228–32. doi:10.1016/s1474-4422(10)70247-7.
Groppa S, Oliviero A, Eisen A, Quartarone A, Cohen LG, Mall V, et al. A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee. Clin Neurophysiol. 2012;123(5):858–82. doi:10.1016/j.clinph.2012.01.010.
Di Lazzaro V, Oliviero A, Pilato F, Saturno E, Dileone M, Mazzone P, et al. The physiological basis of transcranial motor cortex stimulation in conscious humans. Clin Neurophysiol. 2004;115(2):255–66. doi:10.1016/j.clinph.2003.10.009.
Schabrun SM, Hodges PW, Vicenzino B, Jones E, Chipchase LS. Novel adaptations in motor cortical maps: the relationship to persistent elbow pain. Med Sci Sports Exerc. 2014;5(1):1–34. doi:10.1249/mss.0000000000000469.
Mutanen T, Mäki H, Ilmoniemi R. The effect of stimulus parameters on TMS–EEG muscle artifacts. Brain Stimul. 2013;6(3):371–6. doi:10.1016/j.brs.2012.07.005.
Rossi S, Rossini P. TMS in cognitive plasticity and the potential for rehabilitation. Trends Cogn Sci. 2004;8(6):273–9. doi:10.1016/j.tics.2004.04.012.
Raina S. Validity versus reliability. Neurol India. 2015;63(2):291. doi:10.4103/0028-3886.156326.
van de Ruit M, Perenboom M, Grey M. TMS brain mapping in less than two minutes. Brain Stimul. 2015;8(2):231–9. doi:10.1016/j.brs.2014.10.020.
Mead G, Bernhardt J, Kwakkel G. Stroke: physical fitness, exercise, and fatigue. Stroke Res Treat. 2012;1(1):1–2. doi:10.1155/2012/632531.
Löfberg O, Julkunen P, Pääkkönen A, Karhu J. The auditory-evoked arousal modulates motor cortex excitability. Neuroscience. 2014;274:403–8. doi:10.1016/j.neuroscience.2014.05.060.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. doi:10.1186/2046-4053-4-1.
Jagannath V, Mathew J, Asokan G, Fedorowicz Z. Quality assessment of systematic reviews of health care interventions using AMSTAR. Indian Pediatr. 2010;48(5):383–5. doi:10.1007/s13312-011-0080-3.
Higgins J, Green S. Cochrane handbook for systematic reviews of interventions, version 5.0.2. Cochrane Collaboration. 2011. http://www.neuinfo.org/about/index.shtm. Accessed 20 March 2015.
Kamen G. Reliability of motor-evoked potentials during resting and active contraction conditions. Med Sci Sports Exerc. 2004;36(9):1574–9. doi:10.1249/01.mss.0000139804.02576.6a.
Plowman-Prine E, Triggs W, Malcolm M, Rosenbek J. Reliability of transcranial magnetic stimulation for mapping swallowing musculature in the human motor cortex. Clin Neurophysiol. 2008;119(10):2298–303. doi:10.1016/j.clinph.2008.06.006.
Carroll T, Riek S, Carson R. Reliability of the input–output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation. J Neurosci Methods. 2001;112(2):193–202. doi:10.1016/s0165-0270(01)00468-x.
Martin P, Hudson A, Gandevia S, Taylor J. Reproducible measurement of human motoneuron excitability with magnetic stimulation of the corticospinal tract. J Neurophysiol. 2009;102(1):606–13. doi:10.1152/jn.91348.2008.
Chipchase L, Schabrun S, Cohen L, Hodges P, Ridding M, Rothwell J, et al. A checklist for assessing the methodological quality of studies using transcranial magnetic stimulation to study the motor system: an international consensus study. Clin Neurophysiol. 2012;123(9):1698–704. doi:10.1016/j.clinph.2012.05.003.
Lucas N, Macaskill P, Irwig L, Bogduk N. The development of a quality appraisal tool for studies of diagnostic reliability (QAREL). J Clin Epidemiol. 2010;63(8):854–61. doi:10.1016/j.jclinepi.2009.10.002.
Bialocerkowski A, Klupp N, Bragge P. How to read and critically appraise a reliability article. Int J Ther Rehabil. 2010;17(3):114–20. doi:10.12968/ijtr.2010.17.3.46743.
Lucas N, Macaskill P, Irwig L, Moran R, Rickards L, Turner R, et al. The reliability of a quality appraisal tool for studies of diagnostic reliability (QAREL). BMC Med Res Methodol. 2013;13(1):111. doi:10.1186/1471-2288-13-111.
Triano J, Budgell B, Bagnulo A, Roffey B, Bergmann T, Copperstein R, et al. Review of methods used by chiropractors to determine the site for applying manipulation. Chiropr Man Therap. 2013;21(1):36. doi:10.1186/2045-709x-21-36.
Portney L, Watkins M. Foundations of clinical research: application to practice. 2nd ed. Melbourne: Pearson; 2009.
There are no additional acknowledgements or affiliations to report. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
The authors declare that they have no competing interests.
RC, LC, and SMS were each involved in the conception, design, writing, and editing of the study protocol. The final protocol was approved by RC, LC, and SMS.
PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist. Recommended items to address in a systematic review protocol. (DOC 81 kb)
Search strategies. Detailed search strategies that will be utilised to conduct the proposed systematic review. (DOCX 16 kb)
Custom methodological quality checklist. Checklist that will be utilised to assess the general experimental design of included studies. (DOCX 15 kb)
TMS-specific methodology checklist. Checklist that will be utilised to assess the TMS-specific methodological and reporting quality of included studies. (DOCX 15 kb)