We compiled a systematic review protocol using guidance from the Preferred Reporting Items for Systematic reviews and Meta-analyses Protocols (or PRISMA-P) [15]. Our protocol was compiled, reviewed by the team, and peer-reviewed by the Canadian Institutes of Health Research knowledge synthesis committee. Our protocol has been registered in the PROSPERO database (CRD42013004151; available at: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42013004151).
Eligibility criteria
We will include RCTs (including all RCT designs such as crossover, cluster and patient-randomised clinical trials) examining falls prevention interventions amongst adults aged ≥65 years from all settings (e.g., community, acute care). Single and multifactorial interventions will be included, such as exercise, gait and balance training, vitamin D, vision correction, environmental assessment, and medication review. The eligible comparators will include usual care or other falls prevention interventions.
In order to be included, the RCTs will have to examine our primary or secondary outcomes of interest that have been identified by the knowledge users involved with this review. The primary outcomes of interest are the number of injurious falls or number of hospital admissions due to falls. Our secondary outcomes of interest include falls rate, number of fallers, number of emergency room visits due to falls, number of physician visits due to falls, number of fractures, costs, and number of intervention-related harms (e.g., muscle soreness or myocardial infarction related to exercise).
Finally, we will not impose any language, time (i.e., studies of all duration will be included) or year of publication restrictions. Both published and unpublished material will be included. A draft eligibility form is in Appendix 1.
Information sources
We will start by searching electronic databases, such as MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Ageline. The main database search will be supplemented with a comprehensive search for grey literature; difficult to locate or unpublished RCTs. It will be conducted using guidance from a handbook produced by the Canadian Agency for Drugs and Technology in Health (CADTH) on searching for grey literature [16]. Specifically, we will search public health websites (e.g., Health Canada, Public Health Agency of Canada), organizations that produce clinical practice guidelines (e.g., National Institute for Health and Clinical Excellence, Agency for Healthcare Research and Quality: http://www.guidelines.gov), clinical trial registers (e.g., World Health Organization Clinical Trials Search Portal: http://apps.who.int/trialsearch/, which allows searching multiple databases simultaneously), and abstracts and proceedings from meetings (e.g., Canadian Geriatrics Society meetings, American Geriatrics Society meetings, World Injury Prevention conferences). Furthermore, forward citation searching for all included trials will be conducted in Scopus, Web of Science and Google. Experts in the field will be contacted via email to identify relevant trials. Finally, the reference lists of included trials and key studies identified through our search will be searched.
Literature search
Comprehensive searches of the literature will be conducted by our experienced librarian. The literature search will be limited to adults and RCTs using validated filters. The draft literature search for the main (MEDLINE) database will be peer reviewed by another experienced librarian using the Peer Review of Electronic Search Strategies (PRESS) checklist [17]. After this exercise, the literature search will be modified as necessary. The draft literature search for MEDLINE can be found in Appendix 2.
Study selection process
To ensure reliability, a training exercise will be conducted prior to screening. Using the inclusion and exclusion criteria developed a priori, a random sample of 50 citations from the literature search will be screened by all team members independently using our online Synthesi.SR Tool (proprietary online systematic review software developed for our Knowledge Synthesis Center at St. Michael’s Hospital). Inter-rater agreement for study inclusion will be calculated using percent agreement and a second training exercise will be conducted if <95% agreement is observed. When sufficient agreement has been reached, two reviewers will independently screen the citations from the literature search results for inclusion in duplicate. Conflicts will be resolved through team discussion. Two reviewers will independently review the full-text of potentially relevant articles to determine inclusion using a similar process, except that authors are sometimes contacted at this stage if study eligibility is unclear.
Data items
From the included RCTs, we will abstract data on study characteristics (e.g., year of conduct, sample size, setting, intervention and comparator), participant characteristics (e.g., number of patients, mean age and standard deviation, co-morbidities), primary outcomes results (injurious falls, hospital admissions due to falls), and secondary outcomes results (falls rate, number of fallers, fractures, emergency room and physician visits due to falls, intervention-related harms, costs). We will abstract data on the primary and secondary outcomes from each time period reported across the RCTs to examine the effects of the interventions on the outcomes over time. Multiple study publications reporting data from the same study group (i.e., companion reports) will be sorted and abstraction will only be conducted on the RCT reporting the primary outcome of interest and/or longest duration of follow-up. Sufficient data will be collected to allow careful judgment of the homogeneity and similarity of assumptions for meta-analysis, as described in the synthesis section below.
Falls prevention strategies are complex interventions, which are rarely reported in the literature [18]. As such, the authors of the included RCTs will be contacted to obtain pertinent information, including details on the population and setting, the intervention (e.g., ‘dose’, ‘formulation’, schedule, timing of the intervention and monitoring), and how the intervention was delivered. We will ask them to send us any training materials used for the intervention. Authors will also be contacted for data clarifications related to the primary and secondary outcomes, as required.
Data collection process
The data abstraction form will be piloted on a random sample of 10 included trials and modified as required. Data abstraction will only begin when sufficient agreement is noted (i.e., >95% agreement). Two reviewers will independently abstract all of the data using the standardized data abstraction form. All data abstraction will be conducted using our online Synthesi.SR Tool, which provides a summary of conflicts between reviewers and allows these to be rectified directly in the system. Discrepancies will be resolved by discussion amongst the team.
Risk of bias appraisal
We will appraise the risk of bias using the Cochrane Risk of Bias Tool [19]. This tool was developed specifically to assess the internal validity of RCTs. It consists of the following seven criteria: 1) randomization generation, 2) allocation concealment, 3) blinding of outcome assessors, 4) blinding patients and personnel, 5) incomplete outcome data (i.e., withdrawals), 6) selective outcome reporting, and 7) other risks of bias. The final item will include fraudulent results, other methodological flaws in the RCTs, and the potential for industry bias [20].
Synthesis of included studies
The results will first be synthesized descriptively, reporting study characteristics, patient characteristics, risk of bias results, and frequencies of outcomes across the included RCTs. Prior to undertaking meta-analysis, we will assess for statistical, clinical, and methodological heterogeneity. If extensive statistical heterogeneity is observed (e.g., I2 statistic > 60%) or if substantial clinical or methodological heterogeneity is noted (based on the team’s clinical and methodological expertise), we will conduct meta-regression analysis. The meta-regression will explore the influence of factors including baseline effect sizes [source of statistical heterogeneity]; age, frailty, comorbidities, setting (e.g., rehabilitation, community) [sources of clinical heterogeneity]; and trial design [source of methodological heterogeneity] on the meta-analysis results. We will explore the effects of sources of clinical and methodological heterogeneity on our results via subgroup analysis. Subgroups that we may explore include risk of bias (high versus low), attrition rate (<10% versus ≥10%), and setting (e.g., acute care, community, long-term care and rehabilitation settings). We will use established methods for adjusting cluster-randomised clinical trials for meta-analysis with patient-randomised clinical trials [21, 22]. For crossover trials, we will include the data from the period before the cross over was completed. Both meta-analysis and meta-regression will be analyzed using SAS version 9.2. Missing data from the included RCTs (e.g., unreported standard deviations or standard errors) will be included in our analysis using established imputation methods [23].
Although the assumptions underlying network meta-analysis are similar to standard meta-analysis techniques, there are additional issues of comparability that need to be considered [24]. For example, the included studies should be homogeneous across patient populations, study design, and methodological characteristics. Furthermore, the effect of all treatments included in the model should be generalizable across all included studies to ensure validity of results. If deemed feasible and appropriate, network meta-analysis will be conducted to derive the combined outcome for all possible pairwise comparisons between interventions, including those that were not directly compared within an RCT, as well as rank the efficacy and safety among all available interventions [25–27]. To examine the additive effects of multifactorial interventions, an additive mixed treatment comparison with covariates will be conducted [28]. The covariates will be coded as binary (yes/no or 1/0). For example, a trial comparing exercise, balance assessment, and medication reconciliation versus no intervention will be coded as 1,1,1, versus 0,0,0. This approach can tease out the effectiveness of single component interventions compared to multifactorial interventions [28].
The consistency of the results from direct evidence (i.e., head-to-head trials from pairwise meta-analysis) versus indirect evidence (i.e., from network meta-analysis) will be compared using the “node-splitting” method. This method separates trial-level evidence for a particular comparison (or node) by ‘direct’ and ‘indirect’ and the results are compared [29]. To facilitate the practicality of treatment comparisons, median rankings will be used as point estimates of intervention efficacy and safety. We will also use the graphic method proposed by Salanti et al. to assist in the interpretation of the results from the network meta-analysis [30]. Mixed treatment meta-regression will be performed to examine the influence of study level covariates using the method proposed by Nixon et al. [31]. The network meta-analysis will be conducted in WinBUGS; [25] a Bayesian software program used to build complex statistical models using the Markov Chain Monte Carlo simulation. To distinguish between significant and non-significant treatment efficacies, 95% credible intervals will be established using the 2.5 and 97.5 percentiles obtained via Markov Chain Monte Carlo simulation of 100,000 iterations. The 95% credible interval will be interpreted in a similar manner as confidence intervals are interpreted when they are derived using standard meta-analysis methods [25]. Sensitivity analysis will be conducted to examine the effects of imputations for missing data [32], and the use of different “priors” that were employed in the Bayesian network meta-analysis.
