Skip to main content

Does type of funding affect reporting in network meta-analysis? A scoping review of network meta-analyses

Abstract

Background

Evidence has shown that private industry-sponsored randomized controlled trials (RCTs) and meta-analyses are more likely to report intervention-favourable results compared with other sources of funding. However, this has not been assessed in network meta-analyses (NMAs).

Objectives

To (a) explore the recommendation rate of industry-sponsored NMAs on their company’s intervention, and (b) assess reporting in NMAs of pharmacologic interventions according to their funding type.

Methods

Design: Scoping review of published NMAs with RCTs.

Information Sources: We used a pre-existing NMA database including 1,144 articles from MEDLINE, EMBASE and Cochrane Database of Systematic Reviews, published between January 2013 and July 2018.

Study Selection: NMAs with transparent funding information and comparing pharmacologic interventions with/without placebo.

Synthesis: We captured whether NMAs recommended their own or another company’s intervention, classified NMAs according to their primary outcome findings (i.e., statistical significance and direction of effect), and according to the overall reported conclusion. We assessed reporting using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension to NMA (PRISMA-NMA) 32-item checklist. We matched and compared industry with non-industry NMAs having the same research question, disease, primary outcome, and pharmacologic intervention against placebo/control.

Results

We retrieved 658 NMAs, which reported a median of 23 items in the PRISMA-NMA checklist (interquartile range [IQR]: 21–26). NMAs were categorized as 314 publicly-sponsored (PRISMA-NMA median 24.5, IQR 22–27), 208 non-sponsored (PRISMA-NMA median 23, IQR 20–25), and 136 industry/mixed-sponsored NMAs (PRISMA-NMA median 21, IQR 19–24). Most industry-sponsored NMAs recommended their own manufactured drug (92%), suggested a statistically significant positive treatment-effect for their drug (82%), and reported an overall positive conclusion (92%). Our matched NMAs (25 industry vs 25 non-industry) indicated that industry-sponsored NMAs had favourable conclusions more often (100% vs 80%) and were associated with larger (but not statistically significantly different) efficacy effect sizes (in 61% of NMAs) compared with non–industry-sponsored NMAs.

Conclusions

Differences in completeness of reporting and author characteristics were apparent among NMAs with different types of funding. Publicly-sponsored NMAs had the best reporting and published their findings in higher impact-factor journals. Knowledge users should be mindful of this potential funding bias in NMAs.

Peer Review reports

Background

Industry-sponsored randomised clinical trials (RCTs) are often conducted for regulatory purposes [1]. Results that are unfavourable to a study’s funder can pose considerable financial risks to companies. The cost to develop a new intervention, including expenditures on drugs that fail to reach the market, can range between $1–2 billion USD dollars [2]. Consequently, empirical evidence has shown that private industry-sponsored RCTs tend to report more favourable efficacy (or safety) results and conclusions for the sponsoring company’s intervention compared to RCTs with non-conflicted sources of funding [3, 4]. Several efforts in reducing industry-sponsorship bias have been made, including RCT registration, publication of RCT results in registries, and mandatory disclosure of the sponsor’s role in journal publications, among others [5, 6]. Also, reporting of RCT sponsorship and authors’ conflicts of interests (COIs) in meta-analyses has improved over years [7].

Pressure to make marginal products look novel, more effective and safer than they are, may result in biases in industry-sponsored studies. Sponsorship bias (or funding bias) refers to the tendency of a scientific study to support the interests of the sponsoring company's interventions. [8, 9]. Previous research revealed the impact of sponsorship bias in RCTs [10] and systematic reviews with meta-analysis [11,12,13,14]. Empirical evidence showed that private industry-sponsored RCTs reported favourable efficacy results and conclusions, and fewer harms for the sponsoring company’s intervention compared to RCTs with non-conflicted sources of funding [3, 4, 15, 16]. Industry-sponsored systematic reviews with meta-analysis are often produced by company employees, consultancies/contactors hired by the companies, or by authors with industry ties [17], and the findings aligned with sponsor interests [14, 18,19,20,21].

Although sponsorship bias may exist in systematic reviews with network meta-analysis (NMA) [8, 22, 23], to date, no study has evaluated it at the NMA level. NMAs are very attractive for decision making and they are increasingly used when developing clinical practice guidelines and health technology assessments [24, 25]. This is because NMAs can compare the efficacy and safety of all interventions in a single model and can rank the interventions according the specific condition from best to worst [26]. NMAs can help decision makers choose among competing interventions, and hence have a policy impact. This heightens the concerns about conflicts of interest. However, the completeness of reporting in NMA is low with a small yearly improvement [27]. It is unclear what the impact of funding source is on the reporting of NMA. To determine whether the results and conclusions of NMAs are trustworthy, we need to examine whether NMAs supported by private manufacturers compared to not-for-profit and no funded NMAs lead to more favourable findings.

In this study, we aimed to (a) explore the recommendation rate of private industry-sponsored network meta-analyses on their own company’s experimental intervention, and (b) assess reporting in NMAs of pharmacologic interventions according to their funding type (industry, public, or non-sponsored).

Methods

Study design

This is a scoping review, which aims to evaluate research practices and is based on the JBI (formerly Joanna Briggs Institute) methods manual [28], and followed the Preferred Items for Systematic Reviews and Meta-analysis (PRISMA) extension for scoping reviews [29] (Additional file 2). Our methods are briefly described below, and are detailed in Appendix 1 (Additional file 1) and our previous publications [27, 30,31,32].

We used our previous collection of NMAs with RCTs published between January 2013 and July 2018 [27]. We included networks of RCTs with at least four intervention nodes and a number of studies larger than the number of nodes compared (e.g., if four nodes were compared then five studies had to be included), where the authors had conducted an adjusted indirect comparison or NMA [33]. These decisions were initially made to ensure sufficient data were available per NMA for the evaluation of treatment efficacy or safety, and to assess characteristics in networks with complex evidence structure that were not captured in previous studies [34, 35]. We developed a predefined data abstraction form in an Excel spreadsheet (see data abstraction form in Appendix 1 (Additional file 1)) [27], and pilot tested it using five purposefully chosen NMAs. One investigator extracted data and another verified the data for accuracy.

We assessed the reporting completeness of the NMAs using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension to NMA (PRISMA-NMA) [36] by funding type. The PRISMA statement was first published in 2009 [37] and updated in 2020 [38] to incorporate systematic review methods advances, and was developed to promote transparent reporting in published systematic reviews. The PRISMA extension for NMA was published in 2015 to capture reporting in methods developed specifically for NMA [36].

Statistical analysis

We performed a descriptive analysis by NMA funding type, and used frequencies and percentages for discrete variables, and median and interquartile range (IQR) for continuous variables. We visually assessed whether NMA publication according to funding type changed over time using a stacked bar plot. For our analysis, we combined industry- with mixed-sponsored NMAs (i.e., funded by both public organizations and industry) in a single group (henceforth called industry-sponsored, as for each NMA, some industry sponsorship was provided).

We explored whether the funding type influenced reporting and journal publication by comparing the underlying distributions of the PRISMA-NMA score and journal impact factor. We presented forest plots of pharmacologic intervention effects and their 95% confidence intervals (CIs) for the underlying industry-sponsored pharmacologic interventions according to the effect size used (e.g., log odds ratio). We compared the distributions of the PRISMA-NMA score by the authors’ conclusion about the primary pharmacologic intervention’s outcome according to the calculated effect size (i.e., (non-)significantly positive/negative/neutral) and by overall conclusion across all outcomes (positive, negative, neutral, indeterminate [Appendix 1 (Additional file 1)] [39]). We also assessed reporting of COIs and their types for the first and senior author in the NMA using barplots (see Appendix 1 (Additional file 1) types of COIs).

We matched industry/mixed- with publicly sponsored NMAs/ NMAs with no funding having: the same research question, disease, primary outcome, and pharmacologic intervention compared with placebo/control. We compared the effect sizes of the matched industry- and non-industry (publicly or not funded) sponsored NMAs using forest plots and by obtaining the differences in log odds ratios and mean differences of the sponsored intervention between the matched NMAs.

Patients and public involvement

No patients or public were involved in the study.

Results

Of the total 1,144 NMAs published between 2013 and 2018, we screened 907 NMAs for relevant funding information after removing 237 NMAs with non-pharmacological interventions. Of the screened NMAs, 248 did not report funding information, though in 55 cases this was identified from the PROSPERO record (International Prospective Register of Systematic Reviews). We contacted the remaining 192 NMA corresponding authors, of whom 55 authors responded to our emails; we were unable to locate authors’ contact information for 33 NMAs. We then screened the 769 NMAs for eligibility according to our inclusion criteria and included 658 NMAs categorized as: 131 industry-sponsored, 5 mixed-sponsored (i.e., resulting in a total of 136 NMAs including at least one industry sponsor), 314 publicly-sponsored, and 208 non-sponsored NMAs (i.e., with no funding for the review; Appendix 2 (Additional file 1)).

Characteristics of the 658 NMAs

According to the country of corresponding author, the majority of NMAs (198 [30%]) were conducted in China, followed by the United States (113 [17%]) and the United Kingdom (106 [16%]; Appendix 3 (Additional file 1)). Most publicly- (161 [51%]) and non-sponsored (99 [44%]) NMAs were conducted in Asia, while most industry-sponsored NMAs were conducted in Europe (71 [52%]) followed by North America (52 [38%]; Table 1). Oncology was the most prevalent condition explored in NMAs (99 [15%]), especially in publicly- (50 [16%]) and non-sponsored (32 [15%]) NMAs, while rheumatology was the most frequent condition explored in industry-sponsored (21 [15%]) NMAs. Network characteristics, including number of treatments, studies, and participants, did not vary between the different funding types, on average. The number of pharmacologic interventions manufactured by the funder and assessed in the industry-sponsored NMAs ranged between zero and five.

Table 1 Characteristics of included network meta-analyses, categorized by sponsorship status

The number of private industry-sponsored NMAs decreased with year of publication (e.g., 39% in 2013 vs 14% in 2018), whereas non-sponsored NMAs increased (e.g., 16% in 2013 vs 44% in 2018) and the amount of publicly-sponsored NMAs remained unchanged (e.g., 45% in 2013 vs 44% in 2018; Fig. 1). NMAs were published in journals with a median impact factor 3.86 (IQR 2.72–5.90). Private industry-sponsored NMAs were published in journals with lower impact factors (median 3.00 [IQR 2.27–4.16]) compared to publicly- and non-sponsored NMAs (publicly-sponsored NMAs: median 4.39 [IQR 2.97– 7.09], non-sponsored NMAs: median 3.74 [IQR 2.69–5.24]; Appendix 4 (Additional file 1), on average.

Fig. 1
figure 1

Stacked bar plot of published network meta-analyses over year of publication and by type of funding.

Note: Industry- and mixed-sponsored studies were combined in one group

Author COI characteristics by funding type

Across all NMAs (n = 658)

The median number of authors in a NMA was seven (IQR 5–9). On average and across NMAs, 15% of the total authors included in a NMA were affiliated with the study funder or with any for-profit company. Similarly, 17% of authors declared any financial COIs in a NMA, on average. Overall, 136 (21%) NMAs included industry employees (Appendix 5 (Additional file 1)).

Industry-sponsored NMAs (n = 136)

The median number of authors in an industry-sponsored NMA was six (IQR 5–8). Most NMAs (108 [79%]) included industry employees. In 24 (18%) NMAs, the first author was affiliated with the funder, and they declared financial COIs with the funder in 103 (76%) NMAs (Appendix 5 (Additional file 1)). Similarly, the senior author was affiliated in 58 (43%) NMAs with the funder, and they declared any financial COI in 108 (79%) NMAs. Of the 24 (18%) first authors who were funder-affiliated, one (4%) author did not report their funder COI, and of the 58 (43%) senior authors who were funder-affiliated, seven (12%) authors did not report their funder COI.

The median number of authors affiliated with any funder was two (IQR 1–4), the number of authors affiliated with any for-profit company was five (IQR 2–6), and declared financial COIs was five (IQR 2–7). The most frequent type of financial funder-COI was employment for both first (19/24 [79%]) and senior (48/58 [83%]) funder-affiliated authors, and consultancy for both first (48/112 [43%]) and senior (35/78 [45%]) non-funder affiliated authors (Appendix 6 (Additional file 1)).

Publicly-sponsored NMAs (n = 314)

The median number of authors in a publicly-sponsored NMA was seven (IQR 6–9). In 58 (18%) NMAs the first author was affiliated with the public organization funder, and they declared any financial COI in 13 (4%) NMAs. Similarly, the senior author was affiliated in 57 (18%) NMAs with the public organization funder, and they declared financial funder COIs in 11 (3%) NMAs. Of the 58 first and 57 senior funder-affiliated authors none reported any COIs with the public funder.

Reporting completeness of NMAs by funding type

The median PRISMA-NMA score was 21 (IQR 19–24) in industry-sponsored NMAs, 23 (IQR 20–25) in non-sponsored NMAs, and 24.5 (IQR 22–27) in publicly-sponsored NMAs (total NMAs 23 [IQR 21–26], Fig. 2). Overall, reporting in NMAs of efficacy did not differ from NMAs of safety (median PRISMA-NMA score: efficacy 23 [IQR 21–26], safety 23 [IQR 20–26]). However, as expected, the median PRISMA-NMA scores differed by funding type in efficacy and safety NMAs (efficacy: industry-sponsored 21 [IQR 19–24], non-sponsored 23 IQR [20,21,22,23,24,25], publicly-sponsored 25 [IQR 22–27]; safety: industry-sponsored 20 [IQR 19–21], non-sponsored 23 IQR [20,21,22,23,24,25], publicly-sponsored 24 [IQR 21–26]). Of the total 658 NMAs, 243 (37%) did not report a within-study risk of bias assessment (i.e., present data on study risk of bias and outcome level assessment as suggested in the PRISMA-NMA), and this was mainly observed in 79/136 (58%) industry-sponsored NMAs (non-sponsored NMAs: 77/208 (37%); publicly-sponsored NMAs: 87/314 (28%); see Appendix 7 (Additional file 1)).

Fig. 2
figure 2

Histogram of PRISMA-NMA score by type of funding.

Note: The PRISMA-NMA is a 32-item checklist

The PRISMA-NMA score did not differ by conclusion about the sponsored pharmacologic intervention in the primary outcome in industry-sponsored NMAs (Appendix 8 (Additional file 1)). However, the PRISMA-NMA score was higher in indeterminate overall NMA conclusions (median 24.5 [IQR 23–26]) compared to neutral and positive conclusions (median 21 [IQR 19–24]) in industry-sponsored NMAs (Appendix 9 (Additional file 1)).

Treatment recommendation in 136 private industry-sponsored NMAs

Most (88 [65%]) industry-sponsored NMAs only recommended their own company’s intervention, 7 (5%) NMAs only recommended another company's drug, and 37 (27%) recommended both their own and another company's interventions, while 5 (3%) NMAs made no overall drug recommendations or suggested their drug is equivalent to their comparator(s) (Appendix 7 (Additional file 1)). Overall, 92% of the 136 NMAs recommended their company’s experimental intervention alone or in combination with another company’s drug. The primary outcome regarding the sponsored intervention was significantly positive in 112 NMAs (82%), and the overall NMA conclusion about the sponsored intervention was positive in 125 NMAs (92%).

Treatment effects were predominantly statistically significant irrespective of the year of publication; see absolute z-scores of the sponsored intervention compared to placebo/control in Appendix 10 (Additional file 1). Effect sizes along with their 95% CIs of the sponsored interventions are presented in Appendix 11 (Additional file 1).

Matched comparison of industry-sponsored and non-industry sponsored NMAs

We were able to match 50 datasets (25 industry- with 25 non-industry sponsored datasets), using 36 unique NMAs; some NMAs reported or funded multiple pharmacologic interventions of interest (Appendix 12 (Additional file 1)). The private industry-sponsored NMAs had favourable conclusions more often than the non–industry-sponsored NMAs (in 100% vs 80% positive overall conclusions in the matched NMAs).

Of the 50 matched NMAs, we were able to compare 36 NMAs, resulting in 18 pairs of logORs or MDs. The remaining 14 NMAs, i.e., 7 paired datasets, did not report the same effect sizes and could not be compared. Eleven (61%) industry-sponsored NMAs were associated with larger efficacy effect sizes for the primary outcome, but these did not differ importantly compared with non-industry sponsored NMA effect sizes (Appendix 12, Appendix 13 (Additional file 1)). Of the 11 industry-sponsored NMAs, six (55%) assessed potential risks of bias in individual RCTs compared to the nine (82%) non-industry sponsored NMAs. The mean PRISMA-NMA score was lower for industry-sponsored NMAs compared to non-industry sponsored NMAs (22 vs 24).

Discussion

Research focused on a specific drug which is funded by the manufacturer of the drug has found to be more prone to producing results favourable to the manufacturer’s intervention [15]. Our results showed that NMAs recommended their company’s experimental intervention alone or in combination with another company’s drug (92%), reported a statistically-significant effect estimate (82%) of their primary intervention’s outcome, and a positive conclusion (92%) across all outcomes. NMAs funded by drug companies were more likely to suggest favourable conclusions to the sponsor's product, and to estimate larger, but not significantly different, efficacy effect sizes for the primary outcome compared with NMAs funded by other sources. Possible reasons for more favourable results in industry-sponsored NMAs compared to non-industry sponsored NMAs may include implicit/unconscious biases that we are not able to measure, and the lack of assessing RCT risk of bias. Other reasons may include differences in the network geometries (including the number of nodes, studies, and participants) due to differences in the conduct of systematic review (e.g., literature searches, screening and data abstraction process), overall research question, including objectives of the review, PICO (i.e., patient, intervention, comparator, outcome) and eligibility criteria (e.g., population age, disease severity, performance status, and treatment doses), year of study publication, types of analyses and statistical modelling performed in the matched NMAs. Consideration of node making in a network is an important aspect, and relevant decisions, number of included interventions and nodes, along with rationale, should be presented at the protocol and registration stage. However, industry sponsorship did not appear to significantly influence effect sizes and their interpretation. Our conclusions are firmly supported by our data. In addition to reporting, future studies should explore whether quality and risk of bias is accounted in interpretation and conclusion of NMAs.

Reporting of NMAs of pharmacologic interventions can differ according to their source of funding (industry, public or non-sponsored NMAs). Of the three funding types, publicly-sponsored NMAs had the best reporting and published their findings in the highest impact-factor journals. Similarly, risk of bias assessments for the included RCTs were more frequently reported in publicly-sponsored NMAs than private industry-sponsored NMAs (72% vs 42%). However, the lead authors (first or senior) of the 314 publicly-sponsored NMAs, who were affiliated with the study’s public sponsor only rarely (first authors [4%]; senior author [3%]) declared their financial COIs with the public sponsor in the COI section. This assessment was only based on what was reported in the manuscript, and not on the author ICMJE forms submitted to the underlying journals. In the 136 industry-sponsored NMAs, senior authors were more frequently affiliated with the sponsor than first authors (43% vs 18%), but both declared their financial COIs with the industry sponsor in 3 out of 4 published NMAs.

This is the largest empirical study used to evaluate reporting in NMAs with different funding sources. We followed the JBI methods manual for conducting our scoping review [28] and the PRISMA extension for scoping reviews for reporting [29]. Overall, our results are aligned with previous studies comparing industry-funded and non-industry funded pairwise meta-analyses [14, 40]. Reporting completeness was found to be lower in industry funded meta-analyses and conclusions about the sponsored intervention was more favourable [14]. Our study adds to the literature by revealing the impact of industry funding on NMAs compared to pairwise meta-analyses, using a much larger sample of reviews (658 NMAs vs 39 meta-analyses [14] or 175 Cochrane reviews [40]) and not limiting by publication year or journal.

Selective reporting of outcomes in trial publications is common [41]. Outcomes in published manuscripts frequently do not match the pre-specified primary outcome in trial protocols, with positive outcomes more commonly presented in publications [42]. Industry-funded trials are significantly more likely to recommend the experimental intervention (odds ratio 5.3 95% CI [2.0–14.4]) [43]. When unpublished data from regulatory agencies were added to published meta-analyses, the summary efficacy estimates were lower in 46% of the reviews [44]. Similarly, serious adverse events were also inconsistently reported in published trials compared with trial registry data [45]. A qualitative study revealed that trial investigators were influenced by industry sponsors to selectively report outcomes [46]. Supplementing published data with trial registry results and the grey literature can help improve the quality of systematic reviews and NMAs [47].

Our study has a few limitations. First, we did not blind investigators to the information abstracted regarding the review authors, their affiliations, and their funding status. To avoid potential biases, we verified abstracted data with a second reviewer. Second, reporting assessment was conducted based on the information reported in the final publication, and we did not explore reporting in original protocols. Nevertheless, in our assessment, we considered all available supplementary files and appendices. Third, our database included NMAs published to July 2018, and we may have missed recent well-conducted and reported NMAs. However, our previous study showed that there are no major differences in completeness of reporting before and after the PRISMA-NMA publication, except for the items associated with NMA [27]. Also, given that industry-sponsored NMAs decreased over time, we expect that no major differences would be seen in our results if more recent NMAs would have been included in the database. In this study we assessed the largest number of reviews regarding recommendation rate of private industry-funded interventions. Fourth, in our matched NMAs we used repeated data when a NMA assessed multiple funded interventions. However, this was balanced in the two groups (five industry vs six non-industry repeated NMAs across the 50 matched NMAs), and hence we expect that this will not importantly impact our findings.

Conclusions

Private industry sponsorship is a major source of funding for medical research, evidence synthesis and NMAs. However, decision-makers, such as patient partners, healthcare providers, policy-makers, and guideline-developers should be cautious when evaluating the results of industry-sponsored NMAs, since reporting completeness can be low. Also, COIs can influence the findings and conclusions of an NMA. Guidelines on how to report funding for the conducted NMA, and on how to report authors’ information, including their affiliations, funding, and financial COIs, are needed. For example, employees of the study sponsor should provide an interpretation of how their employment is relevant to or can impact the underlying research and interventions assessed along with their employment declaration. This would increase transparency and make findings better interpretable, irrespective NMA funding type.

Availability of data and materials

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Abbreviations

RCTs:

Randomised clinical trials

COIs:

Conflict of interests

NMAs:

Network meta-analyses

JBI:

Joanna Briggs Institute

PRISMA:

Preferred Items for Systematic Reviews and Meta-analysis

PRISMA-NMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension to NMA

IQR:

Interquartile range

PROSPERO:

International Prospective Register of Systematic Reviews

References

  1. Rosefsky JB. Results of clinical trials sponsored by for-profit vs nonprofit entities. JAMA. 2003;290(23):3070–1; author reply 1-2.

    Article  CAS  PubMed  Google Scholar 

  2. Austin D, Hayford T. Research and Development in the Pharmaceutical Industry. Congressional Budget Office report. 2021.

    Google Scholar 

  3. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2017;2:MR000033.

    PubMed  Google Scholar 

  4. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–65.

    Article  PubMed  Google Scholar 

  5. DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Arch Otolaryngol Head Neck Surg. 2005;131(6):479–80.

    Article  PubMed  Google Scholar 

  6. Schwartz LM, Woloshin S, Zheng E, Tse T, Zarin DA. ClinicalTrials.gov and Drugs@FDA: A Comparison of Results Reporting for New Drug Approval Trials. Ann Intern Med. 2016;165(6):421–30.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Benea C, Turner KA, Roseman M, Bero LA, Lexchin J, Turner EH, et al. Reporting of financial conflicts of interest in meta-analyses of drug trials published in high-impact medical journals: comparison of results from 2017 to 2018 and 2009. Syst Rev. 2020;9(1):77.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Naci H, Dias S, Ades AE. Industry sponsorship bias in research findings: a network meta-analysis of LDL cholesterol reduction in randomised trials of statins. BMJ. 2014;349:g5741.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Holman B, Elliott K. The promise and perils of industry-funded science. Philos Compass. 2018;13:e12544.

    Article  Google Scholar 

  10. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325(7358):249.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Fugh-Berman A, McDonald CP, Bell AM, Bethards EC, Scialli AR. Promotional tone in reviews of menopausal hormone therapy after the Women’s Health Initiative: an analysis of published articles. PLoS Med. 2011;8(3):e1000425.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Haidich AB, Pilalas D, Contopoulos-Ioannidis DG, Ioannidis JP. Most meta-analyses of drug interventions have narrow scopes and many focus on specific agents. J Clin Epidemiol. 2013;66(4):371–8.

    Article  PubMed  Google Scholar 

  13. Ioannidis JP. The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q. 2016;94(3):485–514.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Jorgensen AW, Maric KL, Tendal B, Faurschou A, Gotzsche PC. Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: differences in methodological quality and conclusions. BMC Med Res Methodol. 2008;8:60.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326(7400):1167–70.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Fabbri A, Lai A, Grundy Q, Bero LA. The influence of industry sponsorship on the research agenda: a scoping review. Am J Public Health. 2018;108(11):e9–16.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Hampson LA, Joffe S, Fowler R, Verter J, Emanuel EJ. Frequency, type, and monetary value of financial conflicts of interest in cancer clinical research. J Clin Oncol. 2007;25(24):3609–14.

    Article  PubMed  Google Scholar 

  18. Bero L, Oostvogel F, Bacchetti P, Lee K. Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others. PLoS Med. 2007;4(6):e184.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Dunn AG, Arachi D, Hudgins J, Tsafnat G, Coiera E, Bourgeois FT. Financial conflicts of interest and conclusions about neuraminidase inhibitors for influenza: an analysis of systematic reviews. Ann Intern Med. 2014;161(7):513–8.

    Article  PubMed  Google Scholar 

  20. Yank V, Rennie D, Bero LA. Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ. 2007;335(7631):1202–5.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ebrahim S, Bance S, Athale A, Malachowski C, Ioannidis JP. Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. J Clin Epidemiol. 2016;70:155–63.

    Article  PubMed  Google Scholar 

  22. Bero L. Bias related to funding source in statin trials. BMJ. 2014;349:g5949.

    Article  PubMed  Google Scholar 

  23. Naci H, Dias S, Ades T. No evidence of industry sponsorship bias in statin trials. BMJ. 2014;349:g6579.

    Article  PubMed  Google Scholar 

  24. Kanters S, Ford N, Druyts E, Thorlund K, Mills EJ, Bansback N. Use of network meta-analysis in clinical guidelines. Bull World Health Organ. 2016;94(10):782–4.

    Article  PubMed  PubMed Central  Google Scholar 

  25. World Health Organisation. WHO recommends life-saving interleukin-6 receptor blockers for Covid-19 and urges producers to join efforts to rapidly increase. 2021. Available from: https://www.who.int/news/item/06-07-2021-who-recommends-life-saving-interleukin-6-receptor-blockers-for-Covid-19-and-urges-producers-to-join-efforts-to-rapidly-increase-access.

    Google Scholar 

  26. Li T, Puhan MA, Vedula SS, Singh S, Dickersin K, Ad Hoc Network Meta-analysis Methods Meeting Working G. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med. 2011;9:79.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Veroniki AA, Tsokani S, Zevgiti S, Pagkalidou I, Kontouli KM, Ambarcioglu P, et al. Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis. Syst Rev. 2021;10(1):246.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13:141–6.

    Article  PubMed  Google Scholar 

  29. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

    Article  PubMed  Google Scholar 

  30. Petropoulou M, Nikolakopoulou A, Veroniki AA, Rios P, Vafaei A, Zarin W, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2017;82:20–8.

    Article  PubMed  Google Scholar 

  31. Zarin W, Veroniki AA, Nincic V, Vafaei A, Reynen E, Motiwala SS, et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med. 2017;15(1):3.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Nikolakopoulou A, Chaimani A, Veroniki AA, Vasiliadis HS, Schmid CH, Salanti G. Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS One. 2014;9(1):e86754.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997;50(6):683–91.

    Article  CAS  PubMed  Google Scholar 

  34. Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003;326(7387):472.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Song F, Xiong T, Parekh-Bhurke S, Loke YK, Sutton AJ, Eastwood AJ, et al. Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study. BMJ. 2011;343:d4909.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777–84.

    Article  PubMed  Google Scholar 

  37. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Tricco AC, Tetzlaff J, Pham B, Brehaut J, Moher D. Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study. J Clin Epidemiol. 2009;62(4):380-6 e1.

    Article  PubMed  Google Scholar 

  40. Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ. 2006;333(7572):782.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372(9634):201.

    Article  PubMed  Google Scholar 

  42. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.

    Article  CAS  PubMed  Google Scholar 

  43. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290(7):921–8.

    Article  PubMed  Google Scholar 

  44. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202.

    Article  PubMed  Google Scholar 

  45. Wong EK, Lachance CC, Page MJ, Watt J, Veroniki A, Straus SE, et al. Selective reporting bias in randomised controlled trials from two network meta-analyses: comparison of clinical trial registrations and their respective publications. BMJ Open. 2019;9(9):e031138.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Morrow RL, Mintzes B, Gray G, Law MR, Garrison S, Dormuth CR. Industry Sponsor Influence in Clinical Trial Reporting in Canada: a Qualitative Interview Study. Clin Ther. 2022;44(3):374–88.

    Article  PubMed  Google Scholar 

  47. McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Susan Le for helping with email preparation to contact NMA authors to obtain funding information, and Brahmleen Kaur who helped format and submit the manuscript. We also thank Sofia Tsokani, Stella Zevgiti, Irene Pagkalidou, Katerina-Maria Kontouli, Pinar Ambarcioglou, and Nikos Pandis for helping with some data abstraction at the pre-existing NMA database.

Funding

This work received no funding.

SES is funded by a Tier 1 Canada Research Chair in Knowledge Translation. ACT is funded by a Tier 2 Canada Research Chair in Knowledge Synthesis. EW is funded by the Clinician Scientist Training Program (University of Toronto) and the Vanier Scholarship (Canadian Institute of Health Research). The funders had no role in the conceptualization, design, data collection, analysis, decision to publish, or the manuscript preparation. All authors were independent from the funders and can take responsibility for the integrity of the data and the accuracy of the data analysis.

Author information

Authors and Affiliations

Authors

Contributions

AAV and SES conceived and designed the study. AAV contacted authors, abstracted/verified data, analysed data, interpreted results, and wrote a draft manuscript. EW, CL, JCMM, and IF, abstracted and verified data. AAV is the guarantor of the study. EW, CL, JCMM, IF, and ACT provided input into the design, interpreted results, and edited the manuscript. All authors read and approved the final manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Corresponding author

Correspondence to Areti Angeliki Veroniki.

Ethics declarations

Ethics approval and consent to participate

This work does not involve any participants and as such ethical approval was not sought.

Consent for publication

Not applicable.

Competing interests

All authors declare that they have no conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Supplementary Online Content (Appendices 1–13). The document includes supplementary information (Appendices 1–13).

Additional file 2.

PRISMA-ScR Checklist for Sponsorship in NMA. PRISMA-ScR checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Veroniki, A.A., Wong, E.K.C., Lunny, C. et al. Does type of funding affect reporting in network meta-analysis? A scoping review of network meta-analyses. Syst Rev 12, 81 (2023). https://doi.org/10.1186/s13643-023-02235-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-023-02235-z

Keywords