Skip to main content

The best treatment option(s) for adult and elderly patients with chronic primary musculoskeletal pain: a protocol for a systematic review and network meta-analysis

Abstract

Background

Chronic primary musculoskeletal pain (CPMP) is one subcategory of the new classification of chronic primary pain for the upcoming ICD-11, defined as chronic pain in the muscles, bones, joints, or tendons that persists or recurs for more than 3 months and is associated with significant emotional distress or functional disability. An array of pharmacological, psychological, physical, complementary, and rehabilitative interventions is available for CPMP, for which previous research has demonstrated varying effect sizes with regard to effectiveness in pain reduction and other main outcomes. This highlights the need for the synthesis of all available evidence. The proposed network meta-analysis will compare all available interventions for CPMP to determine the best treatment option(s) with a focus on efficacy and safety of interventions.

Methods

We are interested in comparing interventions of the following types: psychological, pharmacological, physical, complementary, and rehabilitative interventions. We will include all randomized controlled trials that compare one intervention with another, or with a control group, in the treatment of CPMP. Primary efficacy outcomes will be pain intensity, emotional distress, and functional disability. Safety outcomes extracted will include proportion of patients with treatment-emergent adverse events, unwanted events, or drop-out rates due to side effects. Published and unpublished trials will be sought through the search of all relevant databases and trial registries. At least two independent reviewers of the team will select the references and extract data independently. We will assess the risk of bias of each individual study using the Cochrane risk of bias assessment tool. We will conduct a network meta-analysis to synthesize all evidence for each outcome. We will fit our model primarily within a Bayesian framework.

Discussion

CPMP is a disabling condition for which several interventions exist. To our knowledge, this is the first network meta-analysis to systematically compare all available evidence. This is required by national health institutions to inform their decisions about the best available treatment option(s) with regard to efficacy and safety outcomes.

Systematic review registration

PROSPERO CRD42018096114

Peer Review reports

Background

In 2015, an International Association for the Study of Pain (IASP) Task Force, comprised of pain experts from across the globe, suggested a new and pragmatic classification of chronic pain for the upcoming ICD-11. This classification includes seven categories, among them chronic primary pain (CPP). This new definition was created because the etiology for most pain conditions is unknown; CPP describes a condition where pain itself is the disease [1]. CPP is defined as pain in one or more bodily regions that persists or recurs for more than 3 months, is associated with significant emotional distress or functional disability, and is not better explained by another chronic pain condition [2]. CPP consists of the following subcategories: chronic widespread pain, complex regional pain syndrome, chronic primary headache and orofacial pain, and chronic primary visceral pain, as well as chronic primary musculoskeletal pain [2]. These different categories share the following multiple similarities that allow to summarize them under the umbrella term CPP [3,4,5]: (1) high comorbidity rates between chronic pain conditions [6], (2) similar interventions across syndromes [7], (3) similar psychiatric comorbidities (i.e., mainly anxiety and depression) [8], (4) similar societal challenges (e.g., sick days and use of health care services) [9], and (5) the syndromes share genetic factors [10].

The ICD-11 defines chronic primary musculoskeletal pain (CPMP) as chronic pain in the muscles, bones, joints, or tendons that is characterized by significant emotional distress (i.e., anxiety, anger, frustration, and depressed mood) or functional disability. CPMP poses a major problem of public health, considering its high prevalence and refractory nature [11]. Low back pain and neck pain contribute to 1694 years lost to disability per 100,000 persons annually. The two disorders rank first and fourth as global causes of years lived with disability [12]. These conditions also reduce quality of life by interfering with work [11]. From a societal perspective, chronic musculoskeletal pain creates substantial costs for healthcare systems, disability insurance, and work absenteeism [13].

Interventions for CPMP include pharmacotherapy, psychotherapeutic interventions, physiological therapies, rehabilitative interventions, and complementary medicine. Pharmacotherapy plays an important role in alleviating pain for these patients. Non-steroidal anti-inflammatory drugs (NSAIDs), muscle relaxants, and opioid analgesics are some of the most common classes of drugs provided [14]. With regard to effectiveness, pharmacological interventions show small to moderate effect sizes in recent meta-analyses compared to placebo [14, 15]. For example, Gabapentinoids showed minimal improvement of pain when compared to placebo [16], and opioids showed better pain reduction for chronic low back pain than placebo, with standardized mean differences (SMDs) ranging from − 0.55 for tramadol to − 2.47 for transdermal buprenorphine, and strong opioids showed an SMD of − 0.43 [17]. A network meta-analysis looking at paracetamol, NSAIDs, and opioid analgesics is underway [18]. Additionally, vitamin D supplementation recently showed no effect on pain (SMD = 0.004) [19]. It has been argued that the small effects might be attributed to the choice of the primary outcome measure (i.e., pain reduction) in pharmacological trials [20]; other outcomes such as quality of life might show greater gains.

Non-pharmacological interventions also reveal small to moderate effect sizes. For example, psychological interventions such as CBT and self-regulatory treatments proved superior to several control groups for pain intensity (SMD = 0.41), pain interference (SMD = 0.23), and quality of life (SMD = 0.41) at post-treatment [21]. Along similar lines, a recent meta-analysis reported that a self-management group for patients suffering from chronic musculoskeletal pain revealed a moderate, significant effect size (SMD = 0.28) when compared to a range of control groups (such as waitlist control, treatment as usual, exercise, and education); however, the difference was not clinically significant [22]. Other meta-analyses in the field of self-management training [23], multidisciplinary biopsychosocial rehabilitation programs [24], and pain education approaches [25] draw similar conclusions. For physiological and alternative approaches, it has been shown that Tai Chi is more effective than no treatment or usual care in reducing pain in the short-term (SMD = 0.66) [26]. Comparable effects have been found for exercise interventions [27], whereas no significant differences in pain were found between a Pilates and a control group [28]. In terms of complementary approaches, patients suffering from nonspecific chronic low back pain reported a clinically meaningful reduction in levels of pain when participating in an acupuncture group [29]. However, it should be noted that the pharmacological and non-pharmacological studies did not use similar follow-up periods. Drug studies usually have shorter timeframes than psychological ones, and a direct comparison of the results is not advisable.

Besides effectiveness, the safety of a specific intervention informs clinicians and guidelines. For example, Tapentadol has shown to be associated with a 2.7-fold increase in the risk of discontinuation treatment due to adverse effects [15]. Non-pharmacological interventions may also cause unwanted events but are, however, less frequent and less likely to be reported. A meta-analysis of multicomponent treatments (i.e., those that involve a combination of physical, psychological, educational, and/or work-related components) looked at adverse events, but the included studies reported adverse events in a manner that did not enable comparisons between groups [24]. Other available meta-analyses in the field of CPMP did not report safety outcomes, and to date, no network meta-analysis is available (except for one protocol [18] that looks at paracetamol, NSAIDs, and opioid analgesics for chronic low back pain).

Aging and disability are both related to an increased occurrence of chronic pain [30]. Despite increasing awareness of the prevalence of pain among older adults, under-treatment remains an issue, not only because of polypharmacy and comorbid diseases, but also because of commonly held pain myths which are assumed to complicate care in the elderly [31]. To the best of our knowledge, meta-analyses focusing on this vulnerable age category are scarce, with the exception of one recent meta-analysis, examining exercise interventions in elderly patients suffering from chronic musculoskeletal pain [32].

Previous research on efficacy and safety data has predominantly focused on the comparison of one medication versus a control group or has applied a comparative design (e.g., the effectiveness and safety of one medication compared to another) [18]. However, from the perspective of patients and doctors selecting treatments, it would be preferable to compare multiple interventions simultaneously, and to quantify the likely range of outcomes when selective between the highly diverse interventions available. Here, the proposed network meta-analysis uniquely allows to address these points.

Objectives

The objective of this systematic review and network meta-analysis (NMA) is to synthesize all available data from RCTs on interventions delivered for CPMP in adult patients, including the elderly. Our aims are to (1) compare the efficacy (i.e., changes in pain, emotional distress, and functional disability) and (II) safety (i.e., treatment-emergent adverse events, serious adverse events, unwanted events) of all available interventions with each other and various control groups. We will focus on post-treatment assessment, medium-term, as well as long-term follow-up, whereby long-term outcomes will be considered primary. Our analysis will incorporate information on the relative efficacy of control interventions used within included studies, and, where feasible, we will include control conditions within the matrix of treatment comparisons (e.g., treatment-as-usual) [33].

Methods and analysis

Protocol and registration

Our protocol is registered on PROSPERO (CRD42018096114) and is available from http://www.crd.york.ac.uk/PROSPERO.

Eligibility criteria for considering studies for review

Types of studies

We will include RCTs that make head-to-head comparisons of two different interventions as well as those that compare an intervention with a control group (e.g., waitlist control, treatment as usual, or placebo) in the treatment of CPMP. We will include protocols and conference papers. Cross-over trials will be included if the results of the first administration are reported separately. Further, randomized single control studies (e.g., the comparison of different dosages of one intervention) will be included. Finally, prophylactic interventions will be included and form a separate network as long as they target CPMP.

We will not include case-control studies, post hoc analyses, or secondary analyses (for example those reporting the cost-effectiveness of an intervention where the primary study focused on the efficacy of the intervention).

Types of participants

Adult patients, including the elderly, of both sexes, with a primary diagnosis that can now be classified as CPMP [2] will be included. We will classify chronic pain syndromes as CPMP according to the 11th revision of the International Classification of Diseases (ICD-11). Hence, the syndromes to be included are now established as chronic low back pain, chronic cervical pain, chronic thoracic pain, and chronic limb pain. In line with the ICD-11, we will only include patients suffering from pain that occurs in one or more bodily regions and that persists or recurs for more than 3 months. Also, the diagnosis of CPMP is appropriate independently of identified biological or psychological contributors unless another diagnosis would better account for the presenting symptoms. Other ICD-11 chronic pain diagnoses, namely chronic cancer-related pain, chronic postsurgical or posttraumatic pain, chronic neuropathic pain, chronic secondary headache or orofacial pain, chronic secondary visceral pain, and chronic secondary musculoskeletal pain, will be excluded. Additionally, according to the ICD-11, CPMP is characterized by significant emotional distress or functional disability. However, the studies we include will rely on older diagnostic criteria of chronic pain (e.g., DSM-III, DSM-III-R, DSM-IV, DSM-5, and ICD-10). Therefore, we will also include patients suffering from chronic pain where no additional information regarding their emotional or functional disability is provided.

Studies in which patients have a diagnosis of acute pain (MG31), headache disorders (8A80-8A8Z), or a chronic primary pain syndrome other than CPMP will not be included. Also, studies reporting on chronic secondary pain (i.e., where pain is a symptom of an underlying condition) will be excluded.

Types of interventions

We will include all available interventions for CPMP. This includes (but is not limited to) psychological interventions such as CBT, pharmacological agents such as opioid and non-opioid analgesics, physical interventions such as exercise therapy, complementary interventions such as acupuncture, and rehabilitative interventions such as multidisciplinary inpatient treatment. We will not only focus on face-to-face interventions, but also e-health and m-health approaches. We do not require the treatment to have a minimal length or to be provided by a trained clinician. See the “Data extraction” section for more details on what will be extracted from individual studies.

Efficacy outcome measures

For each efficacy outcome, we plan to conduct a separate NMA.

Primary outcomes

The choice of our primary outcomes is in line with recommendations for clinical trials studying chronic pain (IMMPACT initiative) [34] as well as recent (network) meta-analyses and reviews [15, 18, 35, 36] in the field. Furthermore, we decided to focus on self-reported measures of pain rather than on clinical measurements (e.g., walking test) or objective measures (e.g., measure of pain threshold) since first, self-report measures provide the “gold standard” in the assessment of pain outcomes, and second, to acknowledge the inherently subjective nature of pain. Finally, we intend to prioritize global scores over syndrome-specific scores since the definition of CPMP includes various syndromes [37].

Global pain intensity and the global measurement of pain will be our primary outcomes. We will extract both outcomes where both are reported. We plan to model them jointly, because we assume that these two measurements correspond to the same underlying construct, and relative intervention effects on each will be correlated. We will conduct an additional sensitivity analysis in which these outcomes will be analyzed separately.

  1. 1.

    Global pain intensity. Pain intensity may be measured with a continuous self-report scale (e.g., visual analog scale (VAS) or numeric rating scale (NRS)), or an ordinal scale with greater than six levels (we will consider such ordinal scales to exhibit continuous properties). It should be noted that for the majority of clinical trials of CPP treatments, a measure of pain intensity will provide the primary outcome measure [34].

  2. 2.

    Global measurement of pain. A rating scale within a composite measure of pain (e.g., Short-Form McGill Pain Questionnaire (SF-MPQ) [38]).

In line with the core definition of CPMP (i.e., significant emotional distress or functional disability), we decided to define emotional distress as well as functional disability as additional primary outcomes of interest. Furthermore, the evaluation of additional symptoms other than pain is clinically important since studies reveal that pain intensity and (physical) functioning are only modestly associated [39].

  1. 3.

    Emotional distress. Emotional distress includes symptoms such as depression, anxiety, and anger in CPMP patients [40]. Questionnaires such as the Beck Depression Inventory (BDI) [33], the Hospital Anxiety and Depression Scale (HADS) [41], and the Profile of Mood States (POMS) [42] will be extracted.

  2. 4.

    Generic measures of functional disability. Measurements of the interference of pain with physical functioning. Use of global questionnaires such as the Multidimensional Pain Inventory (MPI) [43] as well as the Interference Scale or the Brief Pain Inventory (BPI) [44, 45] with pain interference items (i.e., general activity, mood, walking ability, work, relations with other people, sleep, enjoyment of life) will be extracted.

Secondary outcomes

The secondary outcomes are as follows:

  1. 1.

    Syndrome-specific pain. Self-rated questionnaires which specifically measure pain in the field of CPMP (e.g., the Aberdeen Back Pain Scale, ABPS for chronic low back pain) [46].

  2. 2.

    Syndrome-specific measures of physical functioning. Self-rated questionnaires which specifically evaluate physical functioning for patients suffering from chronic musculoskeletal pain (e.g., Roland and Morris Back Pain Disability Scale for chronic low back pain) [47].

  3. 3.

    Health-related quality of life (HRQOL). The SF-36 Health Survey [48] is the most commonly used generic measure of HRQOL. However, we will also extract other questionnaires such as the EuroQol EQ-5D [49].

  4. 4.

    Participant ratings of global improvement and satisfaction with treatment. This category will evaluate all aspects of patients’ health and assesses if there has been an improvement or decline in clinical status (e.g., Patient Global Impression of Change scale, PGIC) [50].

Safety outcome measures

For each safety outcome, we plan to conduct a separate NMA.

Besides efficacy outcomes, we are also interested in safety outcomes, including treatment-emergent adverse events (TEAEs; e.g., dizziness and nausea), serious adverse events (SAEs; e.g., increased risk of suicidal thoughts and behavior), and other unwanted events (UEs). Adverse effects are described broadly as any of “adverse event,” “adverse drug reaction,” “side effect,” “toxic effect,” “complication,” and “unwanted events” that are associated with the treatment under investigation (may it be pharmacotherapy, psychotherapeutic interventions, complementary medicine, physiological therapies, or rehabilitative interventions). We will also extract the length of the treatment period for each study in order to calculate the results for the safety outcomes. We will evaluate three different reporting methods of safety outcomes:

  1. 1.

    We will extract the proportion of patients who experience TEAEs, SAEs, or UEs during the treatment period (i.e., the latest reported time point).

  2. 2.

    We will also extract the mean number of TEAEs, SAEs, or UEs per patient across all reported symptoms (i.e., for the latest reported time point).

  3. 3.

    Finally, study discontinuation due to TEAEs, SAEs, or UEs is also defined as an additional safety outcome measure.

Search strategy and study selection

Searches for published RCTs will be undertaken in the following electronic databases: MEDLINE, Embase, PsycINFO, Cochrane Central, and Web of Science. A first search has been conducted until April 5th, 2018, and will be updated in October 2019. In addition to that, we will search for unpublished RCTs at clinicaltrials.gov and OpenTrials. It is important to include unpublished data, since publication bias leads to exaggerated effect sizes [51] and reporting bias can bias NMA-based estimates of treatment efficacy and modify ranking of interventions [52]. Studies will be identified using search terms for chronic pain in general (chronic pain) and specific syndromes (such as fibromyalgia, complex regional pain syndromes, and irritable bowel syndrome). See Additional file 1 for the search strategy. Note that the search strategy also includes all other categories of chronic primary pain (i.e., chronic widespread pain, complex regional pain syndrome, chronic primary headache and orofacial pain, and chronic primary visceral pain). The reasons for this are twofold: first, we wanted to make sure that our search is exhaustive and second, the long-term goal of this project is to build up a database on all existing interventions for chronic primary pain syndromes. Also, due to the high number of search returns, each individual chronic primary pain category will be reported separately. No data limits or language restrictions will be applied to any of the searches. Results will be exported into Covidence, a web-based software by Cochrane to assist with the production of systematic reviews (https://www.covidence.org).

A research team (i.e., around 15 people) will independently review references and abstracts retrieved by the search and exported into Covidence (https://www.covidence.org/home). The team will have a regular exchange (i.e., once in a month) in order to clarify uncertainties and questions related to the screening process. Each paper will be evaluated by two raters independently. Conflicts will be resolved by the two primary investigators (CL and HK). Consensus will be reached by a weekly consensus meeting of the two primary investigators. If no consensus can be reached, an independent expert in the field of CPP will be contacted. We will obtain the full text of all articles included by reference and abstract screening and use the same eligibility criteria to determine inclusion or exclusion. In case of missing information on these domains in the article, the trial authors may be contacted to obtain further information.

Data extraction

Each article/study-report will be read, evaluated for completeness of the data and be rated for quality by two independent reviewers (for quality ratings, see details below). A structured data extraction form will be designed and pilot-tested for 30 randomly selected studies. Information extracted will include study design (type of study such as parallel or cross-over design, blinding of treatment, special inclusion criteria, special population [if 80% or more of the sample share a particular characteristic]), study characteristics (such as lead author, publication year, journal, sponsor, region and setting, number of clinical sites), participant characteristics (such as diagnostic criteria, age, sex, pain syndrome, pain intensity, comorbidities, duration of symptoms, age of onset), intervention details (such as type of intervention, drug dose and dosing schedule [fixed vs. flexible], length of intervention, training of provider, co-intervention/prophylaxis), control group details (such as type of control group, placebo lead-in yes/no, control dose and dosing schedule, length of control), analysis details (such as intent-to-treat, secondary or sub-analysis), and outcome measures (see the “Outcome measurements” section). After the extraction, we will carefully check and decide which demographics will be used for moderator analyses.

Outcome measurements

We will extract means (Ms) and standard deviations (SDs) for our continuous outcomes (i.e., efficacy outcomes). We will also extract the number of events (E) as well as the total occurrences (N) for each trial arm for our categorical outcomes (i.e., safety outcomes). We will extract data in a wide format (i.e., single study per row) for both continuous and categorical outcomes (i.e., efficacy and safety outcomes). Two review authors (HK and CL) will ascertain that the data are entered correctly into the final data set. We will evaluate whether published and unpublished studies provide different values, using sensitivity analysis.

Missing outcome data

If SDs are not provided for the continuous outcomes, we will calculate them from standard errors (SEs), confidence intervals [53] or other measures [54, 55]. If we are unable to calculate SDs, we will impute the mean of SDs from studies using the same outcome measure [56]. It should be noted that the SDs depend on the scale of measurement which can vary between studies even for the same outcome measure (e.g., VAS for global pain intensity ranging from 1 to 10 or from 1 to 100). Therefore, we will adapt the imputation procedure: SDs vs. Ms will be plotted across all studies where SDs are provided. This will allow us to use a regression model to more precisely predict the SDs in studies without them, based on the reported mean in that particular study. We will conduct an additional sensitivity analysis (all studies, including those with imputed SDs vs. only those studies that report SDs). If the sensitivity analysis reveals a significant difference, we will exclude studies with missing SDs. If the N is missing in the table of analysis, we will use the N of the descriptive statistics (e.g., patients randomized). In the absence of continuous outcome data, we will extract the number of patients that fulfilled a certain criterion (e.g., pain relief) and the number of patients that did not fulfill the respective criterion [57].

Unit of analysis issues

For cross-over studies, only the first period before the cross will be extracted [58]. In the case of cluster-randomized trials, we will extract data that takes the clustering into account (i.e., adjusted results). If such adjusted data are not available, we will extract unadjusted results and will adjust the sample size by dividing it with the design effect.

Length of trial

In line with other meta-analytical approaches [24, 59], we will extract outcome assessment data for a maximum of three time periods and group them for the purposes of disseminating the analyses: post-treatment assessment (i.e., measured at the time point closest to the end of treatment), medium-term (i.e., at least 3 months but less than 12 months), and long-term follow-up (i.e., 12 months or more) measured from end of treatment. We considered long-term outcomes to be primary.

Risk of bias in individual studies

We will assess risk of bias in the included studies using the revised Cochrane risk of bias assessment tool [60]. The assessment will be performed by two raters independently. Any disagreements will be resolved by the primary investigators (HK and CL). The risk of bias will be evaluated across the following domains: generation of allocation sequence, allocation concealment, blinding of study personnel and participants, blinding of outcome assessor, attrition, selective outcome reporting, and other domains, including sponsorship bias. We will conduct a sensitivity analysis and compare all studies vs. only those studies with low or moderate risk of bias (i.e., excluding those with high risk of bias). We will pilot-test the risk of bias assessment procedure on a small number of studies.

Statistical synthesis of study data

Characteristics of included studies and information flow in the network

We will present descriptive statistics for each trial, including study characteristics (e.g., lead author, publication year, journal, sponsor, region and setting, number of clinical sites) as well as participant characteristics (e.g., diagnostic criteria, age, sex, pain syndrome, pain intensity, comorbidities, duration of symptoms, age of onset). We will also describe the types of comparisons. The available results will be presented in a network diagram for each outcome. The size of the nodes will reflect the amount of evidence accumulated for each treatment (total number of patients), the breadth of each edge will be proportional to the inverse of the variance of the summary effect of each direct treatment comparison, and the color of each edge will represent the risk of bias assessment.

Pairwise meta-analyses

For the pairwise meta-analyses, mean differences of efficacy outcomes (e.g., global pain intensity) between treatment groups will be estimated for each study and will be pooled using a standardized mean difference (SMD). In the case where we have pre- and post-measures only and we want to calculate the standard error (SE) for change from baseline, a correlation coefficient (r) is needed. In a first step, we will set r = 0.5. However, we will conduct additional sensitivity analyses, defining r = {0.3, 0.7} [56]. For our safety outcome measures, we will calculate odds ratios (ORs) with 95% credible intervals.

Heterogeneity will be assessed by calculating the Q statistic, the τ2, and the I2, a transformation of Q that indicates the proportion of observed variance that can be attributed to heterogeneity rather than sampling error [61, 62]. However, we will prefer Q statistic and the τ2 over the I2, as the I2 is sensitive to sample size [63].

Network meta-analyses

For the network meta-analyses of efficacy and safety outcomes, we will employ Bayesian methods, using JAGS via R [64] using the R package gemtc (https://cran.r-project.org/web/packages/gemtc/gemtc.pdf). Initially, we will set broad, pessimistic, weakly informative priors for treatment effects [65]. With regard to the nodes in the network, we plan to start by splitting the interventions and plot networks in a first step. However, this might lead to unconnected networks. Therefore, in a second step, we will discuss ways to classify the interventions into groups, relying on clinical experts in this area to gain an understanding of the different interventions. Other aspects such as length of sessions and training of treatment providers will be included as regression covariates. In a sensitivity analysis, we will contrast this with models using priors elicited from the study team and our clinical colleagues.

We will calculate SMDs for our continuous efficacy outcomes and risk (i.e., probability of study discontinuation due to side effects) or rate ratios (i.e., proportion of patients who experience TEAEs or SAEs and the mean number of TEAEs and SAEs across all participants) for our categorical safety outcomes. We will present the summary SMD as well as ORs for all pairwise comparisons in a league table. We will use random-effects models rather than fixed-effects models because we expect that the included studies will be heterogeneous [66]. However, we will assess heterogeneity and if there is no evidence of heterogeneity, then a fixed-effects model will be more appropriate.

In case of an unconnected network (e.g., two separate networks for pharmacological and psychological interventions), we plan to conduct and report results from each of the networks analyzed separately. We will also use these statistical models to summarize likely efficacy outcomes (and uncertainty in those outcomes) from the perspective of a patient choosing between interventions.

Assessment of the transitivity assumption

The transitivity assumption states that we can combine the direct evidence from AC and BC studies to learn (indirectly) about the comparison AB [67]. We will evaluate the assumption of the transitivity by epidemiologic judgment considering three of its “equivalent expressions” by Salanti [68, 69]: (1) whether studies are comparable in terms of the distribution of effect modifiers, (2) whether the direct and indirect treatment effects are in statistical agreement (via an assessment for inconsistency), and (3) whether participants included in the network could in principle be randomized to any of the treatments.

Subgroup analyses

We will test whether some interventions have treatment effects dependent on what syndrome and age group (i.e., adults vs. the elderly) they are being applied in. This will add an extra level of hierarchy.

Sensitivity analyses

Multiple sensitivity analyses are planned and described in the according paragraphs. In short, these include (1) double-blind vs. unblinded trials (this is especially important for pharmacological interventions since there, double-blinding is possible and preferable), (2) global pain intensity vs. global measurement of pain (we will analyze them separately in this sensitivity analysis in order to justify the pooling across the two outcomes), (3) published vs. unpublished, (4) imputed vs. reported SD, (5) high risk of bias vs. low risk of bias, and (6) different levels of correlation coefficients where SE for change from baseline has to be calculated (r = 0.5 vs. r = {0.3, 0.7}).

Assessment of inconsistency

Transitivity will be tested with a statistical evaluation of consistency, i.e., the agreement between direct and indirect estimates. We will apply local as well as global methods to detect inconsistency [67, 70, 71]. Local methods focus on the inconsistency of a specific treatment comparison, whereas global methods check for inconsistency in the entire network of evidence [67, 71]. We will apply a local method to test for incoherence, namely node-splitting, which compares direct and indirect relative treatment effects using a Z-test [71] and a p value of test for disagreement (direct versus indirect evidence). In order to calculate node-splitting, we will use the R package gemtc. We will also evaluate consistency in the entire network by calculating the unrelated mean effects (UME) model [71]. In this model, the consistency equations are completely removed, which is equivalent to a series of independent meta-analyses for each pairwise contrast while sharing a common heterogeneity variance. The fit of this model is then compared with the standard consistency NMA model [67]. It should be noted that for both local and global approaches, it will only be possible to assess inconsistency if the network presents with closed loops.

Tests for inconsistency are known to have low power [53, 72], and it has been reported that 10% of evidence loops are expected to be inconsistent in the medical literature [73]. Accordingly, we plan to interpret the statistical inference regarding inconsistency with caution.

Selection bias

We will apply the comparison-adjusted [53] as well as the contour-enhanced [74] funnel plots in order to investigate whether the results in imprecise trials significantly differ from those in more precise trials. Additionally, we will perform a meta-regression with a measure of study precision as a covariate, which allows us to explore if there are issues with small study effects [75].

Discussion

The introduction of the new CPMP classification in the ICD-11 both enables and requires the generation of new evidence regarding its treatment options. The network meta-analytic approach will synthesize already existing syndrome-specific evidence in a single network, resulting in efficacy and safety outcomes of multiple interventions for CPMP. Also, evidence gaps in the new field of CPMP will be identified.

National organizations such as the National Institute for Health and Care Excellence require the synthesis of all available evidence from existing studies to inform their decisions about the best available treatment option(s) with regard to safety and efficacy outcomes [76]. The planned network meta-analysis will provide these organizations, clinicians, and patients with this information, potentially helping to inform clinical guidelines.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

ABPS:

Aberdeen Back Pain Scale

BDI:

Beck Depression Inventory

BPI:

Brief Pain Inventory

CPMP:

Chronic primary musculoskeletal pain

CPP:

Chronic primary pain

DSM-III, III-R, IV, 5:

Diagnostic and Statistical Manual of Mental Disorders, 3rd edition (1980), 3rd edition—revised (1987), 4th edition (1994), 5th edition (2013)

E :

Number of events

EQ-5D:

EuroQol 5 Dimension

HADS:

Hospital Anxiety and Depression Scale

HRQOL:

Health-related quality of life

IASP:

International Association for the Study of Pain

ICD-11:

International Classification of Diseases, 11th revision

IMMPACT:

Initiative on Methods, Measurement, and Pain Assessment in Clinical Trial

MPI:

Multidimensional Pain Inventory

Ms:

Means

N :

Total number for each trial arm

NMA:

Network meta-analysis

NRS:

Numeric rating scale

NSAIDs:

Non-steroidal anti-inflammatory drugs

OR:

Odds ratio

PGIC:

Patient Global Impression of Change scale

POMS:

Profile of Mood States

RCT:

Randomized controlled trial

SAEs:

Serious adverse events

SDs:

Standard deviations

SEs:

Standard error

SF-MPQ:

Short-Form McGill Pain Questionnaire

SMD:

Standardized mean differences

TE:

Standardized treatment effect

TEAEs:

Treatment-emergent adverse events

UEs:

Unwanted events

UME:

Unrelated mean effects model

VAS:

Visual analog scale

References

  1. Schechter NL. Functional pain: time for a new name. JAMA Pediatr. 2014;168:693–4.

    Article  PubMed  Google Scholar 

  2. Treede RD, Rief W, Barke A, Aziz Q, Bennett MI, Benoliel R, et al. A classification of chronic pain for ICD-11. Pain. 2015;156:1003–7.

    PubMed  PubMed Central  Google Scholar 

  3. Steingrímsdóttir ÓA, Landmark T, Macfarlane GJ, Nielsen CS. Defining chronic pain in epidemiological studies: a systematic review and meta-analysis. Pain. 2017;158:2092–107.

    Article  PubMed  Google Scholar 

  4. Wessely S, White PD. There is only one functional somatic syndrome. Br J Psychiatry. 2004;185:95–6.

    Article  PubMed  Google Scholar 

  5. Zakrzewska JM. Are headaches different from other chronic pains? J Headache Pain. 2012;13:171.

    Article  PubMed  Google Scholar 

  6. Mills S, Torrance N, Smith BH. Identification and management of chronic pain in primary care: a review. Curr Psychiatry Rep. 2016;18:22.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Chalder T, Willis C. “Lumping” and “splitting” medically unexplained symptoms: is there a role for a transdiagnostic approach? J Ment Health. 2017;26:187–91.

    Article  PubMed  Google Scholar 

  8. Goldberg DP, Reed GM, Robles R, Bobes J, Iglesias C, Fortes S, et al. Multiple somatic symptoms in primary care: a field study for ICD-11 PHC, WHO’s revised classification of mental disorders in primary care settings. J Psychosom Res. 2016;91:48–54.

    Article  PubMed  Google Scholar 

  9. James W. Pragmatism. Cambridge: Harvard University Press; 1975.

    Google Scholar 

  10. Vehof J, Zavos HM, Lachance G, Hammond CJ, Williams FM. Shared genetic factors underlie chronic pain syndromes. Pain. 2014;155:1562–8.

    Article  PubMed  Google Scholar 

  11. Hartvigsen J, Hancock MJ, Kongsted A, Louw Q, Ferreira ML, Genevay S, et al. What low back pain is and why we need to pay attention. Lancet. 2018;391:2356–67.

    Article  PubMed  Google Scholar 

  12. Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2163–96.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Lambeek LC, van Tulder MW, Swinkels ICS, Koppes LLJ, Anema JR, van Mechelen W. The trend in total cost of back pain in the Netherlands in the period 2002 to 2007. Spine. 2011;36:1050–8.

    Article  PubMed  Google Scholar 

  14. Zhang T, Adatia A, Zarin W, Moitri M, Vijenthira A, Chu R, et al. The efficacy of botulinum toxin type A in managing chronic musculoskeletal pain: a systematic review and meta analysis. Inflammopharmacology. 2011;19:21–34.

    Article  CAS  PubMed  Google Scholar 

  15. Santos J, Alarcão J, Fareleira F, Vaz-Carneiro A, Costa J. Tapentadol for chronic musculoskeletal pain in adults. Cochrane Database Syst Rev. 2015;5:CD009923.

    Google Scholar 

  16. Shanthanna H, Gilron I, Rajarathinam M, AlAmri R, Kamath S, Thabane L, et al. Benefits and safety of gabapentinoids in chronic low back pain: a systematic review and meta-analysis of randomized controlled trials. PLoS Med. 2017;14:e1002369.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Chaparro LE, Furlan AD, Deshpande A, Mailis-Gagnon A, Atlas S, Turk DC. Opioids compared with placebo or other treatments for chronic low back pain: an update of the Cochrane review. Spine. 2014;39:556–63.

    Article  PubMed  Google Scholar 

  18. Bagg MK, McLachlan AJ, Maher CG, Kamper SJ, Williams CM, Henschke N, et al. Paracetamol, NSAIDs and opioid analgesics for chronic low back pain: a network meta-analysis. Cochrane Database Syst Rev. 2018;6:CD013045.

    Google Scholar 

  19. Gaikwad M, Vanlint S, Mittinity M, Moseley GL, Stocks N. Does vitamin D supplementation alleviate chronic nonspecific musculoskeletal pain? A systematic review and meta-analysis. Clin Rheumatol. 2017;36:1201–8.

    Article  CAS  PubMed  Google Scholar 

  20. Machado LAC, Kamper SJ, Herbert RD, Maher CG, McAuley JH. Analgesic effects of treatments for non-specific low back pain: a meta-analysis of placebo-controlled randomized trials. Rheumatology. 2008;48:520–7.

    Article  PubMed  CAS  Google Scholar 

  21. Hoffman BM, Papas RK, Chatkoff DK, Kerns RD. Meta-analysis of psychological interventions for chronic low back pain. Health Psychol. 2007;26:1–9.

    Article  PubMed  Google Scholar 

  22. Elbers S, Wittink H, Pool JJM, Smeets RJEM. The effectiveness of generic self-management interventions for patients with chronic musculoskeletal pain on physical function, self-efficacy, pain intensity, and physical activity: a systematic review and meta-analysis. Eur J Pain. 2018;22:1577–96.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Du S, Hu L, Dong J, Xu G, Chen X, Jin S, et al. Self-management program for chronic low back pain: a systematic review and meta-analysis. Patient Educ Couns. 2017;100:37–49.

    Article  PubMed  Google Scholar 

  24. Kamper SJ, Apeldoorn AT, Chiarotto A, Smeets RJEM, Ostelo RWJG, Guzman J, et al. Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: Cochrane systematic review and meta-analysis. BMJ. 2015;350:h444.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Tegner H, Frederiksen P, Esbensen BA, Juhl C. Neurophysiological pain education for patients with chronic low back pain. Clin J Pain. 2018;34:778–86.

    PubMed  Google Scholar 

  26. Hall A, Copsey B, Richmond H, Thompson J, Ferreira M, Latimer J, et al. Effectiveness of tai chi for chronic musculoskeletal pain conditions: updated systematic review and meta-analysis. Phys Ther. 2017;97:227–38.

    Article  PubMed  Google Scholar 

  27. Searle A, Spink M, Ho A, Chuter V. Exercise interventions for the treatment of chronic low back pain: a systematic review and meta-analysis of randomised controlled trials. Clin Rehabil. 2015;29:1155–67.

    Article  PubMed  Google Scholar 

  28. Pereira LM, Obara K, Dias JM, Menacho MO, Guariglia DA, Schiavoni D, et al. Comparing the Pilates method with no exercise or lumbar stabilization for pain and functionality in patients with chronic low back pain: systematic review and meta-analysis. Clin Rehabil. 2012;26:10–20.

    Article  PubMed  Google Scholar 

  29. Lam M, Galvin R, Curry P. Effectiveness of acupuncture for nonspecific chronic low back pain: a systematic review and meta-analysis. Spine. 2013;38:2124–38.

    Article  PubMed  Google Scholar 

  30. Nobili A, Garattini S, Mannucci PM. Multiple diseases and polypharmacy in the elderly: challenges for the internist of the third millennium. J Comorb. 2011;1:28–44.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Thielke S, Sale J, Reid MC. Aging: are these 4 pain myths complicating care? J Fam Pr. 2012;61:666–70.

    Google Scholar 

  32. Park S, Kwon SH. Effect of exercise intervention on chronic musculoskeletal pain in older adults: meta-analysis. Korean J Adult Nurs. 2018;30:455–69.

    Article  Google Scholar 

  33. Beck A, Ward C, Mendelsohn M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–71.

    Article  CAS  PubMed  Google Scholar 

  34. Dworkin RH, Turk DC, Farrar JT, Haythornthwaite JA, Jensen MP, Katz NP, et al. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain. 2005;113:9–19.

    Article  PubMed  Google Scholar 

  35. Eccleston C, Fisher E, Thomas KH, Hearn L, Derry S, Stannard C, et al. Interventions for the reduction of prescribed opioid use in chronic non-cancer pain. Cochrane Database Syst Rev. 2017;11:CD010323.

    PubMed  Google Scholar 

  36. Enthoven W, Roelofs PD, Deyo R, van Tulder M, Koes B. Non-steroidal anti-inflammatory drugs for chronic low back pain. Cochrane Database Syst Rev. 2016;2:CD012087.

    PubMed  Google Scholar 

  37. Melidis C, Denham SL, Hyland ME. A test of the adaptive network explanation of functional disorders using a machine learning analysis of symptoms. Biosystems. 2018;165:22–30.

    Article  PubMed  Google Scholar 

  38. Melzack R. The short-form McGill pain questionnaire. Pain. 1987;30:191–7.

    Article  CAS  PubMed  Google Scholar 

  39. Turk DC. Clinical effectiveness and cost-effectiveness of treatments for patients with chronic pain. Clin J Pain. 2002;18:355–65.

    Article  PubMed  Google Scholar 

  40. Fernandez E. Anxiety, depression, and anger in pain: research findings and clinical options. Texas: Advanced Psychological Resources; 2002.

  41. Snaith RP. The hospital anxiety and depression scale. Health Qual Life Outcomes. 2003;1:29.

    Article  PubMed  PubMed Central  Google Scholar 

  42. McNair DM, Lorr M, Droppleman LF. Profile of mood states. San Diego: Educational and Industrial Testing Service; 1971.

    Google Scholar 

  43. Kerns RD, Turk DC, Rudy TE. The west haven-yale multidimensional pain inventory (WHYMPI). Pain. 1985;23:345–56.

    Article  CAS  PubMed  Google Scholar 

  44. Cleeland CS, Ryan KM. Pain assessment: global use of the brief pain inventory. Ann Acad Med Singap. 1994;23:129–38.

    CAS  PubMed  Google Scholar 

  45. Cleeland CS, Nakamura Y, Mendoza TR, Edwards KR, Douglas J, Serlin RC. Dimensions of the impact of cancer pain in a four country sample: new information from multidimensional scaling. Pain. 1996;67:267–73.

    Article  CAS  PubMed  Google Scholar 

  46. Ruta DA, Garratt AM, Wardlaw D, Russell IT. Developing a valid and reliable measure of health outcome for patients with low back pain. Spine. 1994;19:1887–96.

    Article  CAS  PubMed  Google Scholar 

  47. Roland M, Morris R. A study of the natural history of back pain: part I development of a reliable and sensitive measure of disability in low-back pain. Spine. 1983;8:141–4.

    Article  CAS  PubMed  Google Scholar 

  48. Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Med Care. 1992:30:473–83.

  49. Williams A. The role of the EuroQol instrument in QALY calculations: University of York Centre for Health Economics. Economics Discussion Paper [Internet]; 1995.

  50. Guy W. ECDEU assessment manual for psychopharmacology (DHEW Publication No. ADM 76–338). Washington, DC: US Government Printing Office; 1976.

  51. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–60.

    Article  CAS  PubMed  Google Scholar 

  52. Furukawa TA, Akechi T, Azuma H, Okuyama T, Higuchi T. Evidence-based guidelines for interpretation of the Hamilton Rating Scale for Depression. J Clin Psychopharmacol. 2007;27:531–4.

    Article  PubMed  Google Scholar 

  53. Furukawa TA, Salanti G, Atkinson LZ, Leucht S, Ruhe HG, Turner EH, et al. Comparative efficacy and acceptability of first-generation and second-generation antidepressants in the acute treatment of major depression: protocol for a network meta-analysis. BMJ Open. 2016;6:e010919.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Lipsey, MW, Wilson, DB. Practical meta-analysis. Thousand Oaks, CA: Sage; 2001.

  55. Higgins JPT, Deeks JJ. Chapter 7: selecting studies and collecting data. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011): The Cochrane Collaboration; 2011. Available from www.handbook.cochrane.org.

  56. Higgins JPT, Deeks JJ, Altman DG. Chapter 16: special topics in statistics. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011): The Cochrane Collaboration; 2011. Available from www.handbook.cochrane.org.

  57. Schünemann HJ, Oxman AD, Vist GE, Higgins JPT, Deeks JJ, Glasziou P, et al. Interpreting results and drawing conclusions. In: Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Wiley, 2008:359-87.

  58. Spineli LM, Higgins JP, Cipriani A, Leucht S, Salanti G. Evaluating the impact of imputations for missing participant outcome data in a network meta-analysis. Clin Trials. 2013;10:378–88.

    Article  PubMed  Google Scholar 

  59. Hayden JA, Cartwright JL, Riley RD. Exercise therapy for chronic low back pain: protocol for an individual participant data meta-analysis. Syst Rev. 2012;1:64.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.

  61. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Br Med J. 2003;327:557.

    Article  Google Scholar 

  62. Cochran WG. The combination of estimates from different experiments. Biometrics. 1954;10:101–29.

    Article  Google Scholar 

  63. Borenstein M, Higgins JP, Hedges LV, Rothstein HR. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res Synth Methods. 2017;8:5–18.

    Article  PubMed  Google Scholar 

  64. Plummer M, Stukalov A, Denwood M. rjags: Bayesian graphical models using MCMC (R package version 4-6) [Computer software manual]; 2016.

    Google Scholar 

  65. The Stan Development Repository. Prior Choice Recommendations. 2019. https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations. Accessed 20 Apr 2019.

  66. Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549.

    Article  PubMed  Google Scholar 

  67. Efthimiou O, Debray TP, van Valkenhoef G, Trelle S, Panayidou K, Moons KG, et al. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods. 2016;7:236–63.

    Article  PubMed  Google Scholar 

  68. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3:80–97.

    Article  PubMed  Google Scholar 

  69. Linde K, Rücker G, Schneider A, Kriston L. Questionable assumptions hampered interpretation of a network meta-analysis of primary care depression treatments. J Clin Epidemiol. 2016;71:86–96.

    Article  PubMed  Google Scholar 

  70. Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta-analysis. PLoS One. 2014;9:e99682.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  71. Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials. Med Decis Mak. 2013;33:641–56.

    Article  Google Scholar 

  72. Veroniki AA, Mavridis D, Higgins JP, Salanti G. Characteristics of a loop of evidence that affect detection and estimation of inconsistency: a simulation study. BMC Med Res Methodol. 2014;14:106.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Veroniki AA, Vasiliadis HS, Higgins JP, Salanti G. Evaluation of inconsistency in networks of interventions. Int J Epidemiol. 2013;42:332–45.

    Article  PubMed  Google Scholar 

  74. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol. 2008;61:991–6.

    Article  PubMed  Google Scholar 

  75. Rothstein HR, Sutton AJ, Borenstein M. Publication bias in meta analysis: prevention, assessment and adjustments. Chichester: Wiley; 2005.

    Book  Google Scholar 

  76. Riley RD, Jackson D, Salanti G, Burke DL, Price M, Kirkham J, et al. Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. Br Med J. 2017;358:j3932.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the PainNet team (in alphabetical order): Beth Allen, Antonia Barke, Anna Becker, Kiara Bodonyi, Sarah Bürgler, Daniel Graham, Marc Inderbinen, Anna Kharko, Beatrice Korwisi, Linda Kost, and Colleen Pawliuk. We would especially like to thank Linda Kost and Marc Inderbinen for their help in preparing the manuscript. We would also like to thank Dr. Heike Gerger and Dr. Joe Kossowsky for their conceptual support.

Funding

Cosima Locher, PhD, received funding for this project from the Swiss National Science Foundation (SNSF): P400PS_180730 (title: Overcoming Classificatory and Methodological Hurdles to Improve Treatment of Chronic Primary Pain: A Network Meta-Analytic Approach).

Author information

Authors and Affiliations

Authors

Contributions

HK and CL designed the study and drafted the protocol, will lead the data extraction and analysis, and will write the publication. CL and HK contributed equally to this manuscript. NJW and BW revised the protocol and will assist with the data analysis and provide input on the working of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Cosima Locher.

Ethics declarations

Ethics approval and consent to participate

This systematic review and NMA does not require ethical approval.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1:

Search strategy.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koechlin, H., Whalley, B., Welton, N.J. et al. The best treatment option(s) for adult and elderly patients with chronic primary musculoskeletal pain: a protocol for a systematic review and network meta-analysis. Syst Rev 8, 269 (2019). https://doi.org/10.1186/s13643-019-1174-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-019-1174-6

Keywords