Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Association of industry sponsorship and positive outcome in randomised controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study

  • Pascal Probst1, 2,
  • Kathrin Grummich2,
  • Alexis Ulrich1,
  • Markus W Büchler1,
  • Phillip Knebel1, 2 and
  • Markus K Diener1, 2Email author
Contributed equally
Systematic Reviews20143:138

https://doi.org/10.1186/2046-4053-3-138

Received: 8 October 2014

Accepted: 11 November 2014

Published: 27 November 2014

Abstract

Background

Industry sponsorship has been identified as a factor correlating with positive research findings in several fields of medical science. To date, the influence of industry sponsorship in general and abdominal surgery has not been fully studied. This protocol describes the rationale and planned conduct of a systematic review to determine the association between industry sponsorship and positive outcome in randomised controlled trials in general and abdominal surgery.

Methods/design

A literature search in the Cochrane Library, MEDLINE and EMBASE and additional hand searches in relevant citations will be conducted. In order to cover all relevant areas of general and abdominal surgery, a new literature search strategy called multi-PICO search strategy (MPSS) has been developed. No language restriction will be applied. The search will be limited to publications between January 1985 and July 2014. Information on funding source, outcome, study characteristics and methodological quality will be extracted.

The association between industry sponsorship and positive outcome will be tested by a chi-squared test. A multivariate logistic regression analysis will be performed to control for possible confounders, such as number of study centres, multinational trials, methodological quality, journal impact factor and sample size.

Discussion

This study was designed to clarify whether industry-sponsored trials report more positive outcomes than non-industry trials. It will be the first study to evaluate this topic in general and abdominal surgery. The findings of this study will enable surgical societies, in particular, to give advice about cooperation with the industry and disclosure of funding source based on empirical evidence.

Systematic review registration

PROSPERO CRD42014010802

Keywords

Industry bias Industry sponsorship General and abdominal surgery Randomised controlled trial Medical devices Systematic review Science study Health care research

Background

The debate about the presence and extent of inappropriate industry influence on medical professionals began in the 1980s [1, 2]. Shortly thereafter, a positive association between industry funding and positive research outcomes was first shown for pharmaceutical clinical trials [3]. This topic, referred to as industry bias, has meanwhile been studied in many different medical disciplines, and in 2012, a Cochrane review showed a relative risk of 1.32 (95% confidence interval 1.21–1.44) for industry-funded studies to report a positive outcome in a meta-analysis of 48 primary studies. One possible explanation for this difference between industry-funded trials and those with independent funding was found to be conclusions not justified by the study data. Other quality characteristics, such as risk of bias, did not differ between the two groups [4].

To our knowledge, 11 studies [515] have been published on the potential effect of industry sponsorship in various surgical disciplines. Five of these studies demonstrated a significant association of industry sponsorship and positive research outcome [68, 12, 13], and six studies did not show an association [5, 911, 14, 15] (Table 1). Only two, Lubowitz et al. for orthopaedic surgery and Sun et al. for surgery in otorhinolaryngology (ORL), searched trials systematically [9, 14]. The remaining nine studies selected their samples from one or more arbitrarily defined and mostly high-impact journals. This non-systematic approach represents the principal methodological limitation of existing studies into the effect of industry funding, as the sample might not be representative for the whole population of clinical trials in surgery.
Table 1

Summary of 11 studies of the association between industry funding and positive research outcome across different surgical disciplines

Author (reference)

Discipline

Search strategy

Investigated period

Included study type (n)

Positive outcomes industry vs. independent

Leopold [8]

Orthopaedics

3 journals

12 months (1999–2000)

All (n = 301)

79% vs. 64% p = 0.0390

Ezzet [6]

Orthopaedics

3 journals, 2 congresses

12 months (2001–2002)

All (n = 173)

86% vs. 24% p < 0.0001

Bhandari [5]

Different fields of surgery

8 journals

18 months (1999–2001)

RCT (n = 87)

81% vs. 68% p = 0.4385

Shah [13]

Orthopaedics

1 journal

19 months (2002–2003)

All (n = 527)

73% vs. 44% p < 0.0001

Lynch [10]

Orthopaedics

1 journal (submitted manuscripts)

17 months (2004–2005)

All (n = 208)

74% vs. 70% p = 0.7070

Okike [12]

Orthopaedics

2 congresses

2001 + 2002

All (n = 494)

98% vs. 88% p = 0.0258

Lubowitz [9]

Orthopaedics

MEDLINE

Open–2005

CCT/RCT (n = 23)

100% vs. 86% p = 0.6637

Yao [15]

ORL

4 journals

60 months (2000–2005)

RCT (n = 202)

81% vs. 79% p = 0.8538

Khan [7]

Orthopaedics

5 journals

24 months (2002–2004)

RCT (n = 100)

85% vs. <45% p < 0.0001

Momeni [11]

Plastic surgery

3 journals

15 years (1990–2005)

CCT/RCT (n = 63)

74% vs. 64% p = 0.5900

Sun [14]

ORL

MEDLINE, EMBASE, CINAHL, CENTRAL

1960–2010

RCT (n = 118)

35% vs. 51% p = 0.2067

CCT controlled clinical trial.

General and abdominal surgery comprises a large field of different operations. Abdominal surgery involves operations on organs like stomach, liver, pancreas and gut. Whereas, general surgery involves an inhomogeneous spectrum of operations, e.g. surgery of the thyroid gland, hernias and proctology. Surgery is a field with high potential for innovation because of the constant development of new interventions, especially with regard to medical devices. Implementation of new interventions is commonly justified on the basis of clinical trials. Thus, any industry bias would have a relevant impact on surgical practice. This protocol describes the methods to perform a systematic literature search to find a representative sample of trials for a primary statistical analysis. The influence of industry sponsorship in general and abdominal surgery will be evaluated for the first time.

Methods/design

Research question

This study will firstly aim to determine whether there is an association between industry sponsorship and positive outcome in randomised controlled trials (RCTs) in the field of general and abdominal surgery.

Second, methodological differences between industry- and non-industry-funded RCT will be evaluated in order to explore potential sources of industry bias.

Systematic literature search methodology

One of the major challenges of this study will be the identification of a representative sample of trials within the field of general and abdominal surgery that is potentially at risk for industry bias. Trials without intrinsic commercial interest will have to be excluded from the study sample (Figure 1). Consequently, trials evaluating surgical strategies such as the Shouldice vs. Bassini operation for repair of inguinal hernia will not be investigated as no commercial interest can be directly related to the efficacy of the experimental intervention. On the basis of a preliminary literature screening, trials dealing with medical devices and pharmacological and nutritional interventions represent the vast majority of research with potential industry involvement.
Figure 1

Conceptual visualisation of existing RCT in surgery, divided into three major areas: medical devices, drugs/nutrition and surgical strategy. The grey shading represents the study sample at risk for industry bias.

A research question was formulated according to the Participants, Interventions, Comparisons and Outcomes (PICO) model [16]. Figure 2 shows the preliminary PICO question with the search strategy. Due to the holistic approach, this search strategy was very unspecific, with precision below 3% on the abstract screening. Alteration of search terms did not improve precision substantially. However, abstracts identified by the preliminary search were screened for surgical interventions of interest and classified into 14 subfields (Figure 3). On this basis, a new form of comprehensive literature search was developed, the “multi-PICO search strategy” (MPSS):
Figure 2

Preliminary PICO question and holistic search strategy.

Figure 3

On the basis of a preliminary literature screening, subfields with potential industrial background were recorded. Trials from these subfields will be gathered according to a multi-PICO search strategy.

–First, the preliminary PICO question served as the “master PICO”.

–Second, one or more specific questions called “minor PICOs” were created for every recorded subfield (e.g. Stapler, Figure 4).
Figure 4

Minor PICOs for the subfield Stapler.

Based on the minor PICOs, a systematic literature search will be performed independently by two reviewers following the recommendations of the Cochrane Collaboration [16]. The following databases will be searched: Cochrane Library, MEDLINE (via PubMed) and EMBASE. A search strategy based on a vocabulary thesaurus (MeSH or Emtree) in combination with text words will be used. Additionally, a hand search in relevant citations will be performed. The search will be limited to the period from January 1985 to July 2014, with the rationale that disclosure of funding source was not demanded before 1985. No language restrictions will be applied.

All details of the performed MPSS are displayed in Additional file 1. Archiving of MPSS followed the same rules as for a single-PICO search strategy [17].

The method of MPSS allowed the creation of a specific search for every subfield of investigation.

Study selection

Articles gathered by the MPSS will be screened for eligibility according to the following criteria.

Inclusion criteria

RCT assessing the efficacy or effectiveness of medical devices and perioperative pharmacological and nutritional interventions with direct relation to the surgical procedure in human patient populations in the field of general and abdominal surgery will be eligible.

Exclusion criteria

RCT without funding information.

RCT assessing neurosurgical, urological, orthopaedic, dental, plastic, cardiothoracic, gynaecologic, dermal, vascular or paediatric surgery and otorhinolaryngological or endoscopic interventions.

Thus, a systematic search for surgical RCT in general and abdominal surgery with potential risk for industry bias will yield a broad and representative sample to answer the primary research question.

Data extraction

The full data extraction sheet is displayed in Additional file 2. Here, only items evaluating the research question are defined (Table 2).
Table 2

Extracted information to evaluate the research question

Funding source

[Industry/independent]

Experimental intervention is reported to be superior to the control intervention

[Yes/no]

Exact p value of the primary endpoint

[n]

Year of publication

[n]

Impact factor of journal

[n]

Region

[National/multinational]

Number of study centres

[n]

Sample size

[n]

Concluded superiority without statisticalsignificance of primary endpoint

[Yes/no]

Risk of bias for primary endpoint according to Cochrane Collaboration’s tool for assessing risk of bias

[Low/high/unclear]

A trial will be classed as industry funded if any funding is explicitly stated, regardless of whether the funding took the form of direct financial support, supply of products for use in the study or the conduct of trial tasks, e.g. data analysis. Trials will be dichotomised according to whether the authors conclude the experimental intervention to be superior to the control intervention. The conclusions drawn by the authors will be compared to the data presented. A conclusion in favour of the experimental intervention based on non-significant differences between groups will be recorded. Risk of bias will be assessed according to the Cochrane Collaboration’s tool for assessing risk of bias [16]. Further study characteristics will be captured for multivariate analysis as stated in the “Statistical analysis” section below.

Data extraction will be performed by two reviewers independently for quality assurance purposes [18]. Discrepancies between the two reviewers will be resolved by a third reviewer, and a final extraction sheet will be determined for database entry. A database monitoring will be performed of 100% of data necessary to evaluate the primary endpoint and a randomly selected 20% of remaining data. Finally, the database will be closed and made available for statistical analysis.

Statistical analysis

A primary statistical analysis will be performed to answer the primary research question regarding the association of industry sponsorship and positive outcome as well as the magnitude of this association. Therefore, trials will be divided into those funded by industry and those not funded by industry. Further, trials will be dichotomised according to whether or not the experimental intervention is reported to be superior to the control group. The chance that industry-funded trials report more positive outcomes is expressed as odds ratio (OR). The null hypothesis (H0) is that industry funding is not associated with a positive trial outcome. The alternative hypothesis (H1) is that industry funding is associated with a positive trial outcome. The significance of association will be tested by means of Fisher’s exact test if at least one value in the contingency table is 5 or below. Pearson’s chi-squared test with Yates’s correction will be used if the total sample size is 60 or less. In all other cases, significance of association will be tested using Pearson’s chi-squared test without Yates’s correction at a level of significance of 5%. Furthermore, a multivariate logistic regression with factors (multinational trials, methodological quality) and covariates (number of study centres, journal impact factor, sample size) will be conducted. An additional analysis will be performed for the three subgroups medical devices, pharmaceuticals and nutrition.

Moreover, by comparison of reported p values of primary endpoints from industry-funded and independently funded trials, a possible industry bias will be quantified. Student’s t-test or the Wilcoxon rank-sum test will be used for exact p values. If 20% of p values are not reported exactly, the p values will be classified and Fisher’s exact test or a chi-squared test will be performed.

Additional data extracted will be presented descriptively. Publication bias will be explored using a funnel plot separately for industry-sponsored trials and non-industry- sponsored trials. Statistical analysis will be performed with R[19].

Discussion

Existing literature about association of industry sponsorship and positive outcome in surgery has major limitations due to the approach to primary trials as mentioned above. Therefore, the presence and extent of such association in randomised controlled trials in general and abdominal surgery remains unexplored.

In the case of the hypothesised association of industry sponsorship and positive outcome, this study will investigate by a multivariate statistical analysis whether industry involvement biases result via standard risk domains or if industry involvement is an independent source of bias as assumed by several studies [4, 20].

The conduct of this study is important, because the detection of an industry bias in surgery would have an impact on future research. The findings of this study will enable surgical societies, in particular, to give advice about cooperation with industry and disclosure of funding source based on empirical evidence.

Authors’ information

PP is a surgical resident and holds an MSc in clinical trial management. KG is a methodological specialist and core member of the surgical systematic review group at the Study Center of the German Surgical Society. PK is a board certified surgeon and head of the surgical clinical trial unit. AU is chief consultant in the surgical department and head of surgical oncology. MWB is full professor of the department of general, visceral and transplantation surgery. MKD is a consultant in the surgical department and head of the Study Center of the German Surgical Society.

Notes

Abbreviations

CCT: 

controlled clinical trial

MPSS: 

multi-PICO search strategy

ORL: 

otorhinolaryngology

PICO: 

Participants Interventions, Comparisons and Outcomes

RCT: 

Randomised controlled trials.

Declarations

Acknowledgements

No additional funding source is available. However, the resources and facilities of the University of Heidelberg were used in conducting this review.

Authors’ Affiliations

(1)
University Hospital Heidelberg, Department of General, Visceral and Transplantation Surgery, University of Heidelberg
(2)
The Study Center of the German Surgical Society (SDGC), University of Heidelberg

References

  1. Relman AS: The new medical-industrial complex. N Engl J Med. 1980, 303: 963-970. 10.1056/NEJM198010233031703.View ArticlePubMedGoogle Scholar
  2. Relman AS: Dealing with conflicts of interest. N Engl J Med. 1984, 310 (18): 1182-1183. 10.1056/NEJM198405033101809.View ArticlePubMedGoogle Scholar
  3. Davidson RA: Source of funding and outcome of clinical trials. J Gen Intern Med. 1986, 1 (3): 155-158. 10.1007/BF02602327.View ArticlePubMedGoogle Scholar
  4. Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L: Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012, 12: MR000033-PubMedGoogle Scholar
  5. Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ: Association between industry funding and statistically significant pro-industry findings in medical and surgical randomised trials. CMAJ. 2004, 170 (4): 477-480.PubMedPubMed CentralGoogle Scholar
  6. Ezzet KA: The prevalence of corporate funding in adult lower extremity research and its correlation with reported results. J Arthroplasty. 2003, 18 (Suppl 1): 138-145.View ArticlePubMedGoogle Scholar
  7. Khan SN, Mermer MJ, Myers E, Sandhu HS: The roles of funding source, clinical trial outcome, and quality of reporting in orthopedic surgery literature. Am J Orthop (Belle Mead NJ). 2008, 37 (12): E205-E212. discussion E212Google Scholar
  8. Leopold SS, Warme WJ, Fritz Braunlich E, Shott S: Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res. 2003, 415: 293-301.View ArticlePubMedGoogle Scholar
  9. Lubowitz JH, Appleby D, Centeno JM, Woolf SK, Reid JB: The relationship between the outcome of studies of autologous chondrocyte implantation and the presence of commercial funding. Am J Sports Med. 2007, 35 (11): 1809-1816. 10.1177/0363546507304720.View ArticlePubMedGoogle Scholar
  10. Lynch JR, Cunningham MR, Warme WJ, Schaad DC, Wolf FM, Leopold SS: Commercially funded and United States based research is more likely to be published; good-quality studies with negative outcomes are not. J Bone Joint Surg Am. 2007, 89 (5): 1010-1018. 10.2106/JBJS.F.01152.View ArticlePubMedGoogle Scholar
  11. Momeni A, Becker A, Bannasch H, Antes G, Blümle A, Stark GB: Association between research sponsorship and study outcome in plastic surgery literature. Ann Plast Surg. 2009, 63 (6): 661-664. 10.1097/SAP.0b013e3181951917.View ArticlePubMedGoogle Scholar
  12. Okike K, Kocher MS, Mehlman CT, Bhandari M: Conflict of interest in orthopaedic research. An association between findings and funding in scientific presentations. J Bone Joint Surg Am. 2007, 89 (3): 608-613. 10.2106/JBJS.F.00994.View ArticlePubMedGoogle Scholar
  13. Shah RV, Albert TJ, Bruegel-Sanchez V, Vaccaro AR, Hilibrand AS, Grauer JN: Industry support and correlation to study outcome for papers published in Spine. Spine. 2005, 30: 1099-1104. 10.1097/01.brs.0000161004.15308.b4.View ArticlePubMedGoogle Scholar
  14. Sun GH, Houlton JJ, MacEachern MP, Bradford CR, Hayward RA: Influence of study sponsorship on head and neck cancer randomised trial results. Head Neck. 2013, 35 (10): 1515-1520.PubMedGoogle Scholar
  15. Yao F, Singer M, Rosenfeld RM: Randomised controlled trials in otolaryngology journals. Otolaryngol Head Neck Surg. 2007, 137 (4): 539-544. 10.1016/j.otohns.2007.07.018.View ArticlePubMedGoogle Scholar
  16. Higgins JPT, Cochrane Handbook: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011, The Cochrane Collaboration, Available from http://www.cochrane-handbook.org. (Accessed 30.10.2014)Google Scholar
  17. Kable AK, Pich J, Maslin-Prothero SE: A structured approach to documenting a search strategy for publication: a 12 step guideline for authors. Nurse Educ Today. 2012, 32 (8): 878-886. 10.1016/j.nedt.2012.02.022.View ArticlePubMedGoogle Scholar
  18. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP: Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006, 59 (7): 697-703. 10.1016/j.jclinepi.2005.11.010.View ArticlePubMedGoogle Scholar
  19. R Core Team. R: A Language and Environment for Statistical Computing. 2013, Vienna, Austria: R Foundation for Statistical Computing, http://www.R-project.org. (accessed 30.10.2014)Google Scholar
  20. Roseman M, Turner EH, Lexchin J, Coyne JC, Bero LA, Thombs BD: Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ. 2012, 345: e5155-10.1136/bmj.e5155.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© Probst et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement