A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions
Systematic Reviews volume 9, Article number: 97 (2020)
Network meta-analysis (NMA) has rapidly grown in use during the past decade for the comparison of healthcare interventions. While its general use in the comparison of conventional medicines has been studied previously, to our awareness, its use to assess complementary and alternative medicines (CAM) has not been studied. A scoping review of the literature was performed to identify systematic reviews incorporating NMAs involving one or more CAM interventions.
An information specialist executed a multi-database search (e.g., MEDLINE, Embase, Cochrane), and two reviewers performed study selection and data collection. Information on publication characteristics, diseases studied, interventions compared, reporting transparency, outcomes assessed, and other parameters were extracted from each review.
A total of 89 SR/NMAs were included. The largest number of NMAs was conducted in China (39.3%), followed by the United Kingdom (12.4%) and the United States (9.0%). Reviews were published between 2010 and 2018, with the majority published between 2015 and 2018. More than 90 different CAM therapies appeared at least once, and the median number per NMA was 2 (IQR 1–4); 20.2% of reviews consisted of only CAM therapies. Dietary supplements (51.1%) and vitamins and minerals (42.2%) were the most commonly studied therapies, followed by electrical stimulation (31.1%), herbal medicines (24.4%), and acupuncture and related treatments (22.2%). A diverse set of conditions was identified, the most common being various forms of cancer (11.1%), osteoarthritis of the hip/knee (7.8%), and depression (5.9%). Most reviews adequately addressed a majority of the PRISMA NMA extension items; however, there were limitations in indication of an existing review protocol, exploration of network geometry, and exploration of risk of bias across studies, such as publication bias.
The use of NMA to assess the effectiveness of CAM interventions is growing rapidly. Efforts to identify priority topics for future CAM-related NMAs and to enhance methods for CAM comparisons with conventional medicine are needed.
Systematic review registration
The use of complementary and alternative medicine (CAM) interventions is common [1,2,3,4,5,6], and the number of randomized controlled trials (RCT) and systematic reviews related to CAM interventions have previously been shown to be on the rise [7, 8]. As physicians are sometimes hesitant to discuss the use of CAM therapies with patients due to a lack of comfort in addressing related questions, there is a need to ensure rigorous scientific evidence of their benefits and harms is available [9, 10]. Past research has suggested that reviews of CAM interventions have been associated with certain areas of strength and weakness in terms of rigor relative to systematic reviews of other types of interventions , and challenges regarding clinical trial design and priority setting have also been identified .
Network meta-analysis (NMA) is a generalization of traditional pairwise meta-analysis [13, 14] and the use of NMA has grown rapidly in recent years [15,16,17]. NMA is of considerable value to researchers, analysts, and decision-makers when dealing with clinical scenarios requiring the comparison of multiple alternative therapies, as well as scenarios where there exists both direct and indirect evidence of relevance to the research question at hand [18, 19]. Methodologic research related to the conduct of NMA has also grown rapidly, and its use is also supported by helpful implementation tools including reporting guidance, overviews of adapted procedures for judging the strength of evidence, and published considerations for critical appraisal by decision-makers [20,21,22,23].
While the use of NMA for the comparison of pharmacologic interventions is common in the literature , the frequency of and approaches to its use for the evaluation of benefits and harms of complementary and alternative medicine (CAM) interventions—whether against each other or relative to other non-CAM interventions—to our awareness, has not been studied. In order to inform comparisons between traditional and complementary therapies, NMA represents a potentially valuable tool to establish relative benefits and harms. In the current study, we used a scoping review approach to establish the extent of published NMAs involving CAM interventions in the literature, to assess their objectives as well as clinical and methodologic characteristics, and to judge the current level of reporting transparency based upon criteria of the PRISMA Extension Statement for Network Meta-Analysis . This information will be of value to establish what topics have been assessed in existing NMAs in the literature, thereby helping to prioritize both research topics as well as methodologic approaches for NMAs involving CAM interventions moving forward for interested physicians, decision-makers, and patients alike. Findings from this review will be informative for researchers and stakeholders seeking to prioritize future topics for CAM-related NMAs and may also allow for the identification of conditions wherein future randomized controlled trials of CAM interventions may be informative in the derivation of comparisons with traditional medical interventions.
A protocol for this review was drafted a priori by members of the authorship team. The protocol is available from the University of Ottawa Library’s Online Repository (available from https://www.ruor.uottawa.ca/handle/10393/35658). This review has been prepared in consideration of the guidance provided by the PRISMA extension statement for scoping reviews as well as the Joanna Briggs Institute [24, 25].
Literature review, eligibility criteria and study selection
Published NMAs involving CAM interventions were identified for the current review using a combination of two approaches. First, three co-authors (AV, PR, ACT) maintained a database of all published NMAs published between 1999 and 2015 based upon a multi-database search strategy (including Medline, Embase, and the Cochrane Database) updated on a quarterly basis, with screening of citations and full texts performed by two independent reviewers; details of the search strategy used to establish the database are provided in Additional file 1. An update of the search was performed on May 29, 2018, with analogous techniques for screening of titles/abstracts and full-text articles used to identify and include relevant review articles. From the perspective of identifying reviews including NMA, studies selected for inclusion in the database were required to: (a) have used a valid comparison method (such as adjusted indirect comparison, Bayesian model, meta-regression, multivariate meta-analysis, graph theoretical approach); (b) included a minimum of 4 interventions in the network of evidence studied; (c) included a greater number of studies than there were nodes in the network; and (d) included data from RCTs only. For the purposes of the current review, studies identified from the above screening procedures were also reviewed in additional detail in terms of their included interventions to identify reviews that involved one or more CAM interventions; a listing of CAM therapies used during screening is provided in Additional file 2. Screening for reviews incorporating CAM interventions was performed by two independent reviewers (MP, SW). Articles which were focused upon statistical methods investigations relative to NMA were not included in the current review. The process of study selection was summarized using a flow diagram. Only English language reviews were included.
Data collection procedures
A detailed list of information was gathered from each included study that met the study objectives. This information included publication characteristics (i.e., authorship list, year, and journal of publication), core features of each review (e.g., aspects of the research question addressed including study population and endpoints assessed, CAM therapies evaluated), characteristics of each review’s network geometry (including whether only CAM interventions were compared in isolation, or whether CAM and non-CAM interventions were established as comparators; and underlying numbers of studies and patients informing analysis); and statistical aspects of analyses performed (including choice of framework [Bayesian vs frequentist], assessment of the consistency assumption, and reporting of secondary measures of treatment effect). The completeness of reporting for each SR/NMA was assessed using the checklist from the PRISMA Extension Statement for NMA . This checklist addresses the 27 core items included in the PRISMA Statement  and also addresses 5 additional items specific to the reporting of network meta-analyses (including methods and reporting of findings for each of network geometry inspection and assessment of the appropriateness of the consistency assumption, as well as presentation of a network diagram of the available evidence).
Charting the data
A descriptive approach to summarize the core study characteristics was prepared, along with structured tables and figures to identify salient points of differences noted across studies. A heat map was generated to present the geographic distribution of published reviews (based upon affiliation of each study’s lead author), while a word cloud was prepared to assess the relative frequencies with which different CAM interventions were studied in the set of included NMAs. Trends over time in the number of NMAs published with regard to different clinical conditions were reviewed. Bar graphs were generated to evaluate the proportions of included studies adequately addressing individual items of the PRISMA NMA Checklist related to abstract and introduction, methods, results, discussion, and funding status, respectively. Changes in the completeness of reporting were also assessed by year of publication to establish whether the proportions of studies assessed to be of adequate reporting transparency or review methods were improving over time.
Identified literature and general characteristics
In total, literature searching for this review identified a total of 3948 unique abstracts, 90 of which were retained as eligible network meta-analyses that included one or more CAM interventions according to the criteria described earlier [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115]. Figure 1 presents a summary of the study selection process. Table 1 presents a detailed summary of the core characteristics of the included reviews, including patients’ indication, numbers of studies (and patients) analyzed, endpoints evaluated, key methods used, and review funding.
Year of publication amongst the included reviews ranged from 2010 to 2018 (median 2017; Fig. 2). A total of 35 (39%) were conducted in China, 11 (12.4%) were conducted in the United Kingdom, 8 (9.0%) were conducted in the United States, and 6 (6.7%) were conducted in Germany; 4 (4.5% per country) were conducted in each of Canada, Switzerland, and Thailand, 3 (3.4%) were conducted in each of the Netherlands and Hong Kong, 2 were conducted in each of Italy (2.3%), Malaysia (2.3%), and Brazil (2.3%), and single reviews were conducted in Korea, Sweden, and Greece (see Table 1); Fig. 3 presents a heat map summarizing the distribution of nations producing the set of included NMAs. Funding was public for 57 reviews (64.0%), private/industry sponsored for 3 reviews (3.3%), and no funding was available for support for 12 reviews (13.5%) (see Table 1); funding was unreported for 17 reviews (19.1%).
Patient indications and outcomes studied
Table 2 provides a listing of the patient indications that were studied within the included reviews, as well as data regarding both the totality of reviews per indication and the evolution of reviews with CAM interventions between 2010 and 2018. A total of 10 were from the realm of mental health, addressing topics such as depressive disorder, post-traumatic stress disorder, post-natal depression, treatment-resistant depression, obsessive-compulsive disorder, psychotic disorders, panic disorder, and attention deficit hyperactivity disorder [34, 35, 39, 45, 60, 75, 76, 80, 93, 108]. A total of 11 reviews related to cancer were identified, including NMAs of interventions for gastrointestinal cancer, pancreatic cancer, acute promyelocytic leukemia non-small cell lung cancer, neurotoxicity from chemotherapy, and cancer-related fatigue [37, 54, 66, 74, 82, 89, 95, 105, 107, 112, 113]. Osteoarthritis (including prostatitis) was the subject of 10 reviews [28,29,30, 40, 42, 73, 78, 101, 103, 114], gastrointestinal infections/disorders were the subject of 6 reviews [61, 64, 65, 67, 68, 115], cardiovascular disease in 4 reviews [46, 69, 99, 102], topics related to pregnancy, childbirth and newborn health in 4 reviews [47, 62, 71, 83], and a variety of other clinical indications were assessed in 3 or fewer reviews. The number of NMAs overall increased notably from earlier to later years.
The outcomes studied within each review were also collected. While a narrative overview of these endpoints is not provided here given the extensive nature of this information, a detailed listing for each review has been included in Table 1.
Interventions reviewed and network geometry
A total of 51 reviews (56.7%) considered more than one form of CAM intervention (median 2; IQR 1–4; range 1–18). A total of 17 reviews (19.1%) involved comparisons between CAM interventions only [28, 35, 37, 40, 43, 46, 47, 55, 61, 62, 65, 68, 71, 87, 102, 106, 116], while the remaining 72 (80.9%) also involved comparisons with general medical interventions (Table 1). Figure 4 presents a word cloud summarizing the types of CAM interventions that were identified within the included set of review articles. Dietary supplements (n = 42) and vitamins and minerals (n = 35) appeared in the largest number of reviews, followed by acupuncture and related treatments (n = 20), electrical stimulation (n = 20), East Asian herbal medicines (n = 19), herbal medicines (n = 18), and magnetic stimulation (n = 10); all other interventions were assessed in fewer than 10 reviews. The total number of nodes per evidence network (both CAM and non-CAM interventions) ranged from 3 to 32 (median 8). The total number of patients ranged from 288 to 86,393 (median 3146; IQR 1710 to 8488) for the 82 reviews where this information was available; the numbers of studies ranged from 5 to 283 (median 27; IQR 20 to 55).
Statistical methods and completeness of reporting
Amongst the included reviews, 60 (66.7%) performed analyses using a Bayesian model for NMA while the remaining 29 (32.2%) used a frequentist approach (see Table 1). Consideration of the appropriateness of the consistency assumption was discussed in 70 (78.7%) reports. In addition to reporting of primary findings using approaches such as tables, forest plots, and league tables, a total of 63 (70.8%) NMAs reported either values of Surface Under the Cumulative Ranking (SUCRA) curve, rank-o-grams of probabilities, the probability of being best for each treatment or an average/median ranking per intervention in terms of secondary measures of summary effect (see Table 1).
With regard to the completeness of reporting, the proportion of included NMAs adequately addressing each of the 32 items from the PRISMA NMA Checklist is summarized in Fig. 5 (an overview of the PRISMA NMA Checklist is provided in Additional file 3, while Additional file 4 contains a detailed account of the study-specific assessments). For twenty checklist items (but only one of the 5 added checklist items specific to NMA), reporting was judged to be adequate for 80% or more of the reviews assessed; this included core elements of the abstract, introduction, and methods (specification of eligibility criteria, search information sources, process for study selection, methods for data collection, variables extracted, risk of bias appraisal methods, principal summary measures, methods for meta-analysis), as well as certain components of the findings and discussion sections (numeric details of study selection, provision of a network graph, presentation of study characteristics, presentation of risk of bias data, summary data related to included studies, appraisal of the risk of bias across studies, a summative overview of findings, discussion of study limitations and interpretations).
Several other checklist items were associated with less common completeness of reporting. Amongst the 89 included reviews, only 44 (49.4%) identified the report as a systematic review incorporating a NMA (Checklist Item 1). Few studies adequately reported whether a review protocol existed, and where to access the protocol (Checklist Item 5; 37/89 or 41.6%). A full electronic search strategy for at least one database was provided by only 58 of 89 included studies (65.2%; Checklist Item 8), while totals of 55 (61.8%) and 61 (68.5%) studies addressed methodologic details related to the risk of bias assessments across studies (e.g., publication bias, Checklist Item 15) and details of additional analyses (Checklist Item 16); regarding the latter two elements, reporting was also less complete within the results of the included reviews (Checklist Items 22 and 23). Funding and funder roles were also inconsistently reported (Checklist Item 27). With regard to Checklist Items S1–S5 that are specific to NMA, only one exceeded 80% adequate reporting across the included reviews (Checklist Item S3—provision of a network graph). Methods used to explore network geometry (Checklist Item S1), methods to assess for inconsistency of direct and indirect evidence (Checklist Item S2), description of the traits of the evidence network (Checklist Item S4) and findings from analyses checking for inconsistency (Checklist Item S5) were adequately reported in totals of 34.8%, 73.0%, 56.8%, and 69.3%, respectively.
In reviewing the distribution of the median (IQR) total number of PRISMA items reported over time, findings suggest that the reporting transparency of network meta-analyses has improved slightly over time in NMAs with CAM interventions. In 22 included reviews published prior to 2016 (date chosen in relation to the publication of the PRISMA extension statement for NMA in June 2015), the median (interquartile range) number of items addressed out of 32 (i.e., 27 core items and 5 NMA-related items) was 25 (IQR 23-27.5). In the set of 67 reviews published since the start of 2016, the corresponding median was 26 (IQR 24-28). Totals of 41 (61.2%) reviews published in 2016 and afterward adequately addressed 25 or more checklist items, while the corresponding total amongst those published in 2015 and earlier was 7 (31.8%). With regard to NMA-specific reporting items (S1–S5), improvements were noted in the more recent category of publications for S2 regarding inconsistency methods (79.7% versus 56.0%), S3 regarding provision of network diagrams (96.9% versus and 84.0%) and S5 regarding findings from inconsistency evaluations (70.3% versus 64.0%), while the proportions of studies for S1 and S4 regarding assessment of network geometry patterns were similar across time periods.
The growth of NMA as an incrementally important knowledge synthesis methodology for the comparison of healthcare interventions is well established . While its value in informing the comparison of multiple pharmacologic therapies, in particular, is well known, the use of NMA in evaluating the benefits of CAM interventions, to our awareness, has not previously been studied. In the current scoping review, we have enhanced the current understanding of its history of use in the CAM realm.
Several interesting findings were identified in the context of this scoping review. First, the annual frequency of NMAs incorporating one or more CAM interventions has risen considerably since 2010, aligning with the type of relative growth observed with NMAs in general. While the largest number of reviews included in this study was produced in China, the diverse range of countries represented was geographically diverse, corroborating the use of NMA to be global in nature. The range of CAM interventions studied and the assortment of medical diagnoses in which they were assessed were also diverse, with certain most common approaches to treatment (including dietary supplements, vitamins, minerals, and East Asian herbal medicines) being observed. From a design perspective, the current review suggests that in many cases, CAM interventions were considered either in separation from conventional medicine (compared only with other CAM therapies) or only a very limited amount of CAM therapies were included in comparisons with conventional medicine. The rationale for both occurrences may potentially be driven by the uncertainty of many researchers as to the benefits that CAM interventions as a whole may potentially offer patients; other plausible rationale may include the stages of disease assessed in reviews (CAM therapies may be tried earlier or later in different cases), the types of benefits that are sought by physicians and patients (e.g., symptom relief versus the impact on disease progression), or concerns regarding potential differences in patient populations (i.e., the potential for systematic differences between those agreeing to receive CAM versus non-CAM interventions). Strategies to enhance their inclusion may, therefore, require greater collaboration amongst CAM experts and producers of systematic reviews to establish more diverse research teams, in particular at the design phase of systematic reviews, to grow the list of comparators for consideration; however, this may not address all existing challenges.
The collection of systematic reviews incorporating NMAs identified in this scoping review offers opportunities in several directions. From the perspective of planning future research, the listing of included reviews may allow organizations with a focus in CAM interventions to (a) identify clinical diagnoses considered highly amenable to CAM therapies where no prior NMA has been conducted, allowing for plans to address a current knowledge gap; (b) identify reviews for high priority indications where a comprehensive comparison amongst CAM therapies has not yet been conducted, with past reviews focused upon only a very limited selection; (c) identify reviews for high priority indications where there remains a clear need to derive treatment comparisons between CAM and conventional medicines; and (d) to consider possible conditions wherein future randomized trials of CAM therapies may be informative. While not discussed in detail in the text of this review, the summary table of past reviews also lists the considered outcomes from past NMAs for consideration by multiple audiences to allow thought as to ways existing information might be helpful or to enhance plans for future research in syntheses related to clinical areas assessed in prior reviews. Surveys indicate that the most commonly used CAM therapies in the US are non-vitamin, non-mineral dietary therapies , and this is consistent with the relative prominence of dietary supplements observed in this scoping review. The next most commonly used CAM therapies are deep breathing exercises, yoga, chiropractic or osteopathic manipulation and meditation, and more recent US research also indicates that the percentage of persons using yoga, meditation or chiropractic therapies is increasing . These therapies appeared less often in NMAs, and with increased use, these therapies may be a focus of future research comparisons. CAM therapies are used by a large proportion of people diagnosed with chronic conditions , particularly musculoskeletal pain conditions such as arthritis . Although many people who use CAM do so for musculoskeletal pain or mental health , many people with musculoskeletal pain conditions who use CAM do not use the CAM to treat pain . Likewise, some of the most commonly used CAM modalities such as dietary supplements or yoga are most frequently used for “wellness” reasons rather than treatment of a condition . Identifying where appropriate CAM therapies could be incorporated into NMAs, therefore, cannot rely only upon the prevalence of use, but rather will also consult with researchers and clinicians to identify gaps in the NMA literature. This scoping review may assist in this identification.
In reviewing the completeness and transparency of reporting of the set of included NMAs, several weaknesses were identified relative to both core elements of PRISMA as well as certain elements specific to the PRISMA Extension statement for NMA; this aligns with past evaluations of published NMAs , and efforts to enhance both elements are needed. From a methodologic perspective, further research considering specific elements that relate to the conduct and assumptions underlying NMA may also be relevant. For example, the appropriateness of “lumping” control groups (such as different forms of sham therapy, placebo, and waitlist controls) requires consideration and has been shown previously to potentially introduce bias into the findings of NMAs based upon differential event rates or mean values between sources of control [124,125,126,127]. Furthermore, careful consideration as to whether the study populations enrolled in trials of CAM interventions may differ in important ways relative to those enrolled in trials of conventional methods may also present challenges to the transitivity assumption. In our analyses that looked at trends in reporting completeness based upon PRISMA NMA over time, the median (and IQR) numbers of elements addressed were similar before and after 2016, though the proportions of studies before and after this date that addressed totals of > 25 items (61.2% versus 31.8%) and > 30 items (4.5% versus 0%) both were improved in the latter group.
There are certain limitations to this review to be noted. First, while this scoping review set out to map the conditions studied, CAM interventions evaluated, reporting completeness and other elements, judgements as to the appropriateness of methods for NMA and the completeness of interventions compared in NMAs (from a clinical relevance perspective) were not drawn; while of interest, these were considered to be beyond the goals for this research. Second, while certain characteristics of the NMAs were associated with failures to provide related information within the article text, we did not contact authors for these details, instead, we rely upon what was described only in the article. Last, we did not search registration records for ongoing systematic reviews that may be oriented toward the comparison of CAM therapies or involve comparisons between CAM and conventional medical interventions, and thus the data presented here may underestimate the extent of ongoing NMA evaluations involving CAM therapies.
The application of NMA methods to inform comparisons of CAM interventions has grown rapidly in recent years, and the diversity of interventions assessed and conditions studied is diverse. Given the prevalence of use of CAM interventions, particularly for musculoskeletal conditions and mental health, future efforts to incorporate comparisons in NMAs with conventional medicines and to identify and address the methodologic challenges of NMA in this setting are worthwhile for the comprehensive identification and comparison of treatment options. This review may serve as a starting point from which future research initiatives related to the evaluation of CAM interventions can be prioritized.
A completed PRISMA for Scoping Reviews Checklist is provided in Additional file 5 to document the completeness of reporting of this review.
Availability of data and materials
All data generated or analyzed during this study are included in this published article [and its supplementary information files].
Complementary and alternative medicine
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Randomized controlled trial
Surface under the cumulative ranking
Barnes PM, Bloom B, Nahin RL. Complementary and alternative medicine use among adults and children: United States, 2007. Natl. Health Stat. Rep. 2008:1–23.
Johnson PJ, Ward A, Knutson L, Sendelbach S. Personal use of complementary and alternative medicine (CAM) by U.S. health care workers. Health Serv. Res. 2012;47:211–27.
Okoro CA, Zhao G, Li C, Balluz LS. Use of complementary and alternative medicine among USA adults with functional limitations: for treatment or general use? Complement. Ther. Med. 2011;19:208–15.
Fouladbakhsh JM, Stommel M. Gender, symptom experience, and use of complementary and alternative medicine practices among cancer survivors in the U.S. cancer population. Oncol. Nurs. Forum. 2010;37:E7–E15.
Grzywacz JG, et al. Older adults’ use of complementary and alternative medicine for mental health: findings from the 2002 National Health Interview Survey. J. Altern. Complement. Med. N. Y. N. 2006;12:467–73.
Hoerster KD, Butler DA, Mayer JA, Finlayson T, Gallo LC. Use of conventional care and complementary/alternative medicine among US adults with arthritis. Prev. Med. 2012;54:13–7.
Institute of Medicine (US) Committee on the use of complementary and alternative medicine by the american public. Complementary and Alternative Medicine in the United States. US: National Academies Press; 2005.
Treister-Goltzman Y, Peleg R. Trends in publications on complementary and alternative medicine in the medical literature. J. Complement. Integr. Med. 2015;12:111–5.
Fischer FH, et al. High prevalence but limited evidence in complementary and alternative medicine: guidelines for future research. BMC Complement. Altern. Med. 2014;14:46.
Ng JY, Liang L, Gagliardi AR. The quantity and quality of complementary and alternative medicine clinical practice guidelines on herbal medicines, acupuncture and spinal manipulation: systematic review and assessment using AGREE II. BMC Complement. Altern. Med. 2016;16:425.
Turner L, et al. An evaluation of epidemiological and reporting characteristics of complementary and alternative medicine (CAM) systematic reviews (SRs). PloS One. 2013;8:e53536.
Fischer HF, et al. Key issues in clinical and epidemiological research in complementary and alternative medicine--a systematic literature review. Forsch. Komplementarmedizin. 2012;2006(19 Suppl 2):51–60.
Caldwell D, Ades A, Higgins J. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331:897–900.
Catala-Lopez F, Tobias A, Cameron C, Moher D, Hutton B. Network meta-analysis for comparing treatment effects of multiple interventions: an introduction. Rheumatol. Int. 2014;34:1489–96.
Nikolakopoulou A, et al. Characteristics of networks of interventions: a description of a database of 186 published networks. PloS One. 2014;9:e86754.
Zarin W, et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med. 2017;15:3.
Petropoulou M, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J. Clin. Epidemiol. 2017;82:20–8.
Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat. Med. 2004;23:3105–24.
Dias, S., Welton, N., Sutton, A. & Ades, A. NICE DSU Technical Support Document 2:a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. (2011).
Puhan MA, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ. 2014;349:g5630.
Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JPT. Evaluating the quality of evidence from a network meta-analysis. PloS One. 2014;9:e99682.
Jansen JP, et al. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health J. Int. Soc. Pharmacoeconomics Outcomes Res. 2014;17:157–73.
Hutton B, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews incorporating network meta-analyses of healthcare interventions: checklist and explanations. Ann. Intern. Med. 2015;162:777–84.
Tricco A, Lillie E & Zarin W et al. PRISMA Extenstion for Scoping Reviews (PRISMA ScR): checklist and explanations. Ann. Intern. Med. Press (2018).
The Joanna Briggs Institute. The Joanna Briggs Institute Reviewers’ Manual 2015: Methods for JBI Scoping Reviews. (2015).
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA Statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097. Epub 2009 Jul 21.
Imamura, M. et al. Systematic review and economic modelling of the effectiveness and cost-effectiveness of non-surgical treatments for women with stress urinary incontinence. Health Technol. Assess. Winch. Engl. 14, 1–188, iii–iv (2010).
Wandel S, et al. Effects of glucosamine, chondroitin, or placebo in patients with osteoarthritis of hip or knee: network meta-analysis. BMJ. 2010;341:c4675.
Anothaisintawee T, et al. Management of chronic prostatitis/chronic pelvic pain syndrome: a systematic review and network meta-analysis. JAMA. 2011;305:78–86.
Thakkinstian A, Attia J, Anothaisintawee T, Nickel JC. α-blockers, antibiotics and anti-inflammatories have a role in the management of chronic prostatitis/chronic pelvic pain syndrome. BJU Int. 2012;110:1014–22.
Nüesch E, Häuser W, Bernardy K, Barth J, Jüni P. Comparative efficacy of pharmacological and non-pharmacological interventions in fibromyalgia syndrome: network meta-analysis. Ann. Rheum. Dis. 2013;72:955–62.
Cawston H, Davie A, Paget M-A, Skljarevski V, Happich M. Efficacy of duloxetine versus alternative oral therapies: an indirect comparison of randomised clinical trials in chronic low back pain. Eur. Spine J. Off. Publ. Eur. Spine Soc. Eur. Spinal Deform. Soc. Eur. Sect. Cerv. Spine Res. Soc. 2013;22:1996–2009.
Snedecor SJ, et al. Systematic review and meta-analysis of pharmacological therapies for painful diabetic peripheral neuropathy. Pain Pract. Off. J. World Inst. Pain. 2014;14:167–84.
Kriston L, von Wolff A, Westphal A, Hölzel LP, Härter M. Efficacy and acceptability of acute treatments for persistent depressive disorder: a network meta-analysis. Depress. Anxiety. 2014;31:621–30.
Gerger H, et al. Integrating fragmented evidence by network meta-analysis: relative effectiveness of psychological interventions for adults with post-traumatic stress disorder. Psychol. Med. 2014;44:3151–64.
Griebeler ML, et al. Pharmacologic interventions for painful diabetic neuropathy: An umbrella systematic review and comparative effectiveness network meta-analysis. Ann. Intern. Med. 2014;161:639–49.
Wang J-C, Tian J-H, Ge L, Gan Y-H, Yang K-H. Which is the best Chinese herb injection based on the FOLFOX regimen for gastric cancer? A network meta- analysis of randomized controlled trials. Asian Pac. J. Cancer Prev. APJCP. 2014;15:4795–800.
Lewis RA, et al. Comparative clinical effectiveness of management strategies for sciatica: systematic review and network meta-analyses. Spine J. Off. J. North Am. Spine Soc. 2015;15:1461–77.
Linde K, et al. Efficacy and acceptability of pharmacological treatments for depressive disorders in primary care: systematic review and network meta-analysis. Ann. Fam. Med. 2015;13:69–79.
Zeng C, et al. Electrical stimulation for pain relief in knee osteoarthritis: systematic review and network meta-analysis. Osteoarthritis Cartilage. 2015;23:189–202.
Zhu G-Q, et al. Systematic review with network meta-analysis: the comparative effectiveness and safety of interventions in patients with overt hepatic encephalopathy. Aliment. Pharmacol. Ther. 2015;41:624–35.
Zhu X, et al. Comparative effectiveness of glucosamine, chondroitin, acetaminophen or celecoxib for the treatment of knee and/or hip osteoarthritis: a network meta-analysis. Clin. Exp. Rheumatol. 2018;36:595–602.
Dong W, et al. Treatments for shoulder impingement syndrome: a PRISMA systematic review and network meta-analysis. Med. 2015;94:e510.
Devoe DJ, Peterson A, Addington J. Negative Symptom Interventions in Youth at Risk of Psychosis: A Systematic Review and Network Meta-analysis. Schizophr. Bull. 2018;44:807–23.
Slade E, et al. Treatments for bulimia nervosa: a network meta-analysis. Psychol. Med. 2018;48:2629–36.
Zhang G-X, et al. Different network pharmacology mechanisms of Danshen-based Fangjis in the treatment of stable angina. Acta Pharmacol. Sin. 2018;39:952–60.
van den Akker CHP, et al. Probiotics for Preterm Infants: A Strain-Specific Systematic Review and Network Meta-analysis. J. Pediatr. Gastroenterol. Nutr. 2018;67:103–22.
Grant MD, et al. Menopausal Symptoms: Comparative Effectiveness of Therapies. US: Agency for Healthcare Research and Quality; 2015.
Wei Y, Liao H, Ye J. Therapeutic effects of various therapeutic strategies on non-exudative age-related macular degeneration: A PRISMA-compliant network meta-analysis of randomized controlled trials. Medicine (Baltimore). 2018;97:e10422.
Fu X, et al. A mixed treatment comparison on efficacy and safety of treatments for spasticity caused by multiple sclerosis: a systematic review and network meta-analysis. Clin. Rehabil. 2018;32:713–21.
Freeman SC, et al. Component network meta-analysis identifies the most effective components of psychological preparation for adults undergoing surgery under general anesthesia. J. Clin. Epidemiol. 2018;98:105–16.
Tsikopoulos K, Mavridis D, Georgiannos D, Vasiliadis HS. Does Multimodal rehabilitation for ankle instability improve patients’ self-assessed functional outcomes? A network meta-analysis. Clin. Orthop. 2018;476:1295–310.
Lee A, et al. Therapeutic approaches to atrophic vaginitis in postmenopausal women: a systematic review with a network meta-analysis of randomized controlled trials. J. Menopausal Med. 2018;24:1–10.
Hilfiker R, et al. Exercise and other non-pharmaceutical interventions for cancer-related fatigue in patients during or after cancer treatment: a systematic review incorporating an indirect-comparisons meta-analysis. Br. J. Sports Med. 2018;52:651–8.
Xie D, et al. Effects of five types of selenium supplementation for treatment of Kashin-Beck disease in children: a systematic review and network meta-analysis. BMJ Open. 2018;8:e017883.
Pang B, et al. Different intervention strategies for preventing type 2 diabetes mellitus in China: A systematic review and network meta-analysis of randomized controlled trials. Diabetes Obes. Metab. 2018;20:718–22.
Sarri G, Pedder H, Dias S, Guo Y, Lumsden MA. Vasomotor symptoms resulting from natural menopause: a systematic review and network meta-analysis of treatment effects from the National Institute for Health and Care Excellence guideline on menopause. BJOG Int. J. Obstet. Gynaecol. 2017;124:1514–23.
Feng F, et al. Can music improve sleep quality in adults with primary insomnia? A systematic review and network meta-analysis. Int. J. Nurs. Stud. 2018;77:189–96.
Yeh M-L, Ko S-H, Wang M-H, Chi C-C, Chung Y-C. Acupuncture-Related Techniques for Psoriasis: A Systematic Review with Pairwise and Network Meta-Analyses of Randomized Controlled Trials. J. Altern. Complement. Med. N. Y. N. 2017;23:930–40.
Gartlehner, G. et al. Nonpharmacological versus pharmacological treatments for adult patients with major depressive disorder. (Agency for Healthcare Research and Quality (US), 2015).
Wen J, et al. Probiotics in 14-day triple therapy for Asian pediatric patients with Helicobacter pylori infection: a network meta-analysis. Oncotarget. 2017;8:96409–18.
Khaing W, et al. Calcium and Vitamin D Supplementation for Prevention of preeclampsia: a systematic review and network meta-analysis. Nutrients. 2017;9.
Häggman-Henrikson B, et al. Pharmacological treatment of oro-facial pain - health technology assessment including a systematic review with network meta-analysis. J. Oral Rehabil. 2017;44:800–26.
Ho RST, et al. Acupuncture and related therapies used as add-on or alternative to prokinetics for functional dyspepsia: overview of systematic reviews and network meta-analysis. Sci. Rep. 2017;7:10320.
Feng J-R, et al. Efficacy and safety of probiotic-supplemented triple therapy for eradication of Helicobacter pylori in children: a systematic review and network meta-analysis. Eur. J. Clin. Pharmacol. 2017;73:1199–208.
Fu X, et al. Efficacy of Drug Interventions for chemotherapy-induced chronic peripheral neurotoxicity: a network meta-analysis. Front. Neurol. 2017;8:223.
Feng J-S, Li J-Y, Chen X-Y, Yang Z, Li S-H. Strategies for Preventing endoscopic recurrence of crohn’s disease 1 year after surgery: a network meta-analysis. Gastroenterol. Res. Pract. 2017;2017(7896160).
Wang F, et al. Probiotics in Helicobacter pylori eradication therapy: systematic review and network meta-analysis. Clin. Res. Hepatol. Gastroenterol. 2017;41:466–75.
Yang H, Wu X, Wang M. The effect of three different meditation exercises on hypertension: a network meta-analysis. Evid.-Based Complement. Altern. Med. ECAM. 2017;2017(9784271).
Liang J-H, et al. Comparison of multiple interventions for older adults with Alzheimer disease or mild cognitive impairment: A PRISMA-compliant network meta-analysis. Medicine (Baltimore). 2018;97:e10744.
Khan TM, Wu DB-C, Dolzhenko AV. Effectiveness of fenugreek as a galactagogue: a network meta-analysis. Phytother. Res. PTR. 2018;32:402–12.
Li X, et al. Acupuncture for myofascial pain syndrome: a network meta-analysis of 33 randomized controlled trials. Pain Physician. 2017;20:E883–902.
Qin Z, et al. Network meta-analysis of the efficacy of acupuncture, alpha-blockers and antibiotics on chronic prostatitis/chronic pelvic pain syndrome. Sci. Rep. 2016;6:35737.
Huang J, et al. Induction treatments for acute promyelocytic leukemia: a network meta-analysis. Oncotarget. 2016;7:71974–86.
Skapinakis P, et al. Pharmacological and psychotherapeutic interventions for management of obsessive-compulsive disorder in adults: a systematic review and network meta-analysis. Lancet Psychiatry. 2016;3:730–9.
Morrell CJ, et al. A systematic review, evidence synthesis and meta-analysis of quantitative and qualitative studies evaluating the clinical effectiveness, the cost-effectiveness, safety and acceptability of interventions to prevent postnatal depression. Health Technol. Assess. Winch. Engl. 2016;20:1–414.
Kasatpibal N, et al. Effectiveness of probiotic, prebiotic, and synbiotic therapies in reducing postoperative complications: a systematic review and network meta-analysis. Clin. Infect. Dis. Off. Publ. Infect. Dis. Soc. Am. 2017;64:S153–60.
Woods B, et al. Cost-effectiveness of adjunct non-pharmacological interventions for osteoarthritis of the knee. PloS One. 2017;12:e0172749.
Su X, et al. Comparative effectiveness of 12 treatment strategies for preventing contrast-induced acute kidney injury: a systematic review and bayesian network meta-analysis. Am. J. Kidney Dis. Off. J. Natl. Kidney Found. 2017;69:69–77.
Catalá-López F, et al. The pharmacological and non-pharmacological treatment of attention deficit hyperactivity disorder in children and adolescents: A systematic review with network meta-analyses of randomised trials. PloS One. 2017;12:e0180355.
Westby MJ, Dumville JC, Soares MO, Stubbs N, Norman G. Dressings and topical agents for treating pressure ulcers. Cochrane Database Syst. Rev. 2017;6:CD011947.
Zhang D, Wu J, Liu S, Zhang X, Zhang B. Network meta-analysis of Chinese herbal injections combined with the chemotherapy for the treatment of pancreatic cancer. Medicine (Baltimore). 2017;96:e7005.
Yu W, et al. Preventing necrotizing enterocolitis by food additives in neonates: a network meta-analysis revealing the efficacy and safety. Medicine (Baltimore). 2017;96:e6652.
van Nooten F, Treur M, Pantiri K, Stoker M, Charokopou M. Capsaicin 8% patch versus oral neuropathic pain medications for the treatment of painful diabetic peripheral neuropathy: a systematic literature review and network meta-analysis. Clin. Ther. 2017;39:787–803.e18.
Wang T, Xu C, Pan K, Xiong H. Acupuncture and moxibustion for chronic fatigue syndrome in traditional Chinese medicine: a systematic review and meta-analysis. BMC Complement. Altern. Med. 2017;17:163.
Sekercioglu N, et al. Effects of different phosphate lowering strategies in patients with CKD on laboratory outcomes: A systematic review and NMA. PloS One. 2017;12:e0171028.
Amaral MA, et al. Network meta-analysis of probiotics to prevent respiratory infections in children and adolescents. Pediatr. Pulmonol. 2017;52:833–43.
Zhang W, Deng X, Liu C, Wang X. Intravesical treatment for interstitial cystitis/painful bladder syndrome: a network meta-analysis. Int. Urogynecology J. 2017;28:515–25.
Dulai PS, et al. Chemoprevention of colorectal cancer in individuals with previous colorectal neoplasia: systematic review and network meta-analysis. BMJ. 2016;355:i6188.
Sawangjit R, et al. Comparative efficacy of interventions on nonalcoholic fatty liver disease (NAFLD): A PRISMA-compliant systematic review and network meta-analysis. Medicine (Baltimore). 2016;95:e4529.
Wang H-L, et al. Tripterygium wilfordii Hook F versus conventional synthetic disease-modifying anti-rheumatic drugs as monotherapy for rheumatoid arthritis: a systematic review and network meta-analysis. BMC Complement. Altern. Med. 2016;16:215.
Chung VCH, et al. Chinese Herbal medicine and salmeterol and fluticasone propionate for chronic obstructive pulmonary disease: systematic review and network meta-analysis. Medicine (Baltimore). 2016;95:e3702.
Pompoli A, et al. Psychological therapies for panic disorder with or without agoraphobia in adults: a network meta-analysis. Cochrane Database Syst. Rev. 2016;4:CD011004.
Rochwerg B, et al. Treatment of idiopathic pulmonary fibrosis: a network meta-analysis. BMC Med. 2016;14:18.
Wu X, et al. Chinese herbal medicine for improving quality of life among nonsmall cell lung cancer patients: overview of Systematic Reviews and Network Meta-Analysis. Medicine (Baltimore). 2016;95:e2410.
Dong W, et al. Injection therapies for lateral epicondylalgia: a systematic review and Bayesian network meta-analysis. Br. J. Sports Med. 2016;50:900–8.
Steenhuis LA, Nauta MH, Bocking CLH, Pijnenborg GHM. Treating depressive symptoms in psychosis: a network meta-analysis on the effects of non-verbal therapies. PloS One. 2015;10:e0140637.
Lehert P, Villaseca P, Hogervorst E, Maki PM, Henderson VW. Individually modifiable risk factors to ameliorate cognitive aging: a systematic review and meta-analysis. Climacteric J. Int. Menopause Soc. 2015;18:678–89.
Dong H, et al. Efficacy of Supplementation with B vitamins for stroke prevention: a network meta-analysis of randomized controlled trials. PloS One. 2015;10:e0137533.
Loveman E, et al. Comparing new treatments for idiopathic pulmonary fibrosis--a network meta-analysis. BMC Pulm. Med. 2015;15:37.
Kongtharvonskul J, et al. Efficacy and safety of glucosamine, diacerein, and NSAIDs in osteoarthritis knee: a systematic review and network meta-analysis. Eur. J. Med. Res. 2015;20:24.
Han S-Y, Hong Z-Y, Xie Y-H, Zhao Y, Xu X. Therapeutic effect of Chinese herbal medicines for post stroke recovery: A traditional and network meta-analysis. Medicine (Baltimore). 2017;96:e8830.
MacPherson, H. et al. Acupuncture for chronic pain and depression in primary care: a programme of research. (NIHR Journals Library, 2017).
Howarth, E. et al. IMPRoving Outcomes for children exposed to domestic ViolencE (IMPROVE): an evidence synthesis. (NIHR Journals Library, 2016).
Zhang D, et al. Comparative efficacy and safety of Chinese herbal injections combined with the FOLFOX regimen for treating gastric cancer in China: a network meta-analysis. Oncotarget. 2017;8:68873–89.
Muñoz Fernández, S. S., Ivanauskas, T. & Lima Ribeiro, S. M. Nutritional Strategies in the Management of Alzheimer Disease: Systematic Review With Network Meta-Analysis. J. Am. Med. Dir. Assoc. 18, 897.e13-897.e30 (2017).
Zhang D, et al. Which are the best Chinese herbal injections combined with XELOX regimen for gastric cancer?: A PRISMA-compliant network meta-analysis. Medicine (Baltimore). 2018;97:e0127.
Linde K, Rücker G, Schneider A, Kriston L. Questionable assumptions hampered interpretation of a network meta-analysis of primary care depression treatments. J. Clin. Epidemiol. 2016;71:86–96.
Palmer SC, et al. Phosphate-binding agents in adults with ckd: a network meta-analysis of randomized trials. Am. J. Kidney Dis. Off. J. Natl. Kidney Found. 2016;68:691–702.
Reinecke H, et al. Analgesic efficacy of opioids in chronic pain: recent meta-analyses. Br. J. Pharmacol. 2015;172:324–33.
Di T, et al. Injections of ginkgo in the treatment of cerebral infarction: a systematic review and network Meta-analysis. J. Tradit. Chin. Med. 2018;38:1–11.
Wei X, et al. Efficacy of traditional chinese medicines in preventing oxaliplatin-induced peripheral neurotoxicity in cancer patients: a network meta-analysis. Chin. Herb. Med. 2017;9:161–8.
Ma Y, et al. Can polysaccharide K improve therapeutic efficacy and safety in gastrointestinal cancer? a systematic review and network meta-analysis. Oncotarget. 2017;8:89108–18.
Corbett MS, et al. Acupuncture and other physical treatments for the relief of pain due to osteoarthritis of the knee: network meta-analysis. Osteoarthritis Cartilage. 2013;21:1290–8.
Cai J, et al. Comparative efficacy and tolerability of probiotics for antibiotic-associated diarrhea: Systematic review with network meta-analysis. United Eur. Gastroenterol. J. 2018;6:169–80.
Cai L, et al. Efficacy and safety of adalimumab in Chinese patients with moderate-to-severe plaque psoriasis: results from a phase 3, randomized, placebo-controlled, double-blind study. J. Eur. Acad. Dermatol. Venereol. 2017;31:89–95.
Clarke TC, Black LI, Stussman BJ, Barnes PM, Nahin RL. Trends in the use of complementary health approaches among adults: United States, 2002-2012. Natl. Health Stat. Rep. 2015:1–16.
Saydah SH, Eberhardt MS. Use of complementary and alternative medicine among adults with chronic diseases: United States 2002. J. Altern. Complement. Med. N. Y. N. 2006;12:805–12.
Zhang Y, et al. Complementary and Alternative Medicine Use by U.S. Adults with Self-Reported Doctor-Diagnosed Arthritis: Results from the 2012 National Health Interview Survey. PM R. 2019;11:1059–69.
Sharp D, et al. Complementary medicine use, views, and experiences: a national survey in England. BJGP Open. 2018;2:bjgpopen18X101614.
Clarke TC, Nahin RL, Barnes PM, Stussman BJ. Use of complementary health approaches for musculoskeletal pain disorders among adults: United States, 2012. Natl. Health Stat. Rep. 2016:1–12.
Stussman BJ, Black LI, Barnes PM, Clarke TC, Nahin RL. Wellness-related use of common complementary health approaches among adults: United States, 2012. Natl. Health Stat. Rep. 2015:1–12.
Chambers JD, et al. An assessment of the methodological quality of published network meta-analyses: a systematic review. PloS One. 2015;10:e0121715.
Bannuru R, Osani M, Vaysbrot E, McAlindon T. Are all sugar pills created equal? a systematic review and network meta-analysis. Osteoarthritis Cartilage. 2018;26:S27.
Furukawa TA, et al. Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta Psychiatr. Scand. 2014;130:181–92.
Zhu Z, et al. Comparison of psychological placebo and waiting list control conditions in the assessment of cognitive behavioral therapy for the treatment of generalized anxiety disorder: a meta-analysis. Shanghai Arch. Psychiatry. 2014;26:319–31.
Hyde AJ, May BH, Xue CC, Zhang AL. Variation in placebo effect sizes in clinical trials of oral interventions for management of the behavioral and psychological symptoms of dementia (BPSD): a Systematic Review and Meta-Analysis. Am. J. Geriatr. Psychiatry Off. J. Am. Assoc. Geriatr. Psychiatry. 2017;25:994–1008.
Research reported in this publication was supported by the National Center for Complementary and Integrative Health of the National Institutes of Health under award number R24AT001293. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. ACT is funded by a Tier 2 Canada Research Chair in Knowledge Synthesis.
Ethics approval and consent to participate
Consent for publication
BH has previously received funding from Eversana Incorporated (previously Cornerstone Research Group) for methodologic advice related to systematic reviews and meta-analysis. No other authors have conflicts to declare. AV is a Senior Editor for Systematic Reviews. She had no involvement in the peer review or publication processes for this manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Literature Search Strategies for the review are provided
The list of eligible CAM Interventions is provided
A copy of the PRISMA NMA Extension Checklist is provided for reference
The completed PRISMA-NMA Assessments for the included studies are provided
A completed PRISMA for Scoping Reviews Checklist for the current review is provided
About this article
Cite this article
Pratt, M., Wieland, S., Ahmadzai, N. et al. A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions. Syst Rev 9, 97 (2020). https://doi.org/10.1186/s13643-020-01328-3