- Open Access
Association of industry sponsorship and positive outcome in randomised controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study
Systematic Reviews volume 3, Article number: 138 (2014)
Industry sponsorship has been identified as a factor correlating with positive research findings in several fields of medical science. To date, the influence of industry sponsorship in general and abdominal surgery has not been fully studied. This protocol describes the rationale and planned conduct of a systematic review to determine the association between industry sponsorship and positive outcome in randomised controlled trials in general and abdominal surgery.
A literature search in the Cochrane Library, MEDLINE and EMBASE and additional hand searches in relevant citations will be conducted. In order to cover all relevant areas of general and abdominal surgery, a new literature search strategy called multi-PICO search strategy (MPSS) has been developed. No language restriction will be applied. The search will be limited to publications between January 1985 and July 2014. Information on funding source, outcome, study characteristics and methodological quality will be extracted.
The association between industry sponsorship and positive outcome will be tested by a chi-squared test. A multivariate logistic regression analysis will be performed to control for possible confounders, such as number of study centres, multinational trials, methodological quality, journal impact factor and sample size.
This study was designed to clarify whether industry-sponsored trials report more positive outcomes than non-industry trials. It will be the first study to evaluate this topic in general and abdominal surgery. The findings of this study will enable surgical societies, in particular, to give advice about cooperation with the industry and disclosure of funding source based on empirical evidence.
Systematic review registration
The debate about the presence and extent of inappropriate industry influence on medical professionals began in the 1980s [1, 2]. Shortly thereafter, a positive association between industry funding and positive research outcomes was first shown for pharmaceutical clinical trials . This topic, referred to as industry bias, has meanwhile been studied in many different medical disciplines, and in 2012, a Cochrane review showed a relative risk of 1.32 (95% confidence interval 1.21–1.44) for industry-funded studies to report a positive outcome in a meta-analysis of 48 primary studies. One possible explanation for this difference between industry-funded trials and those with independent funding was found to be conclusions not justified by the study data. Other quality characteristics, such as risk of bias, did not differ between the two groups .
To our knowledge, 11 studies [5–15] have been published on the potential effect of industry sponsorship in various surgical disciplines. Five of these studies demonstrated a significant association of industry sponsorship and positive research outcome [6–8, 12, 13], and six studies did not show an association [5, 9–11, 14, 15] (Table 1). Only two, Lubowitz et al. for orthopaedic surgery and Sun et al. for surgery in otorhinolaryngology (ORL), searched trials systematically [9, 14]. The remaining nine studies selected their samples from one or more arbitrarily defined and mostly high-impact journals. This non-systematic approach represents the principal methodological limitation of existing studies into the effect of industry funding, as the sample might not be representative for the whole population of clinical trials in surgery.
General and abdominal surgery comprises a large field of different operations. Abdominal surgery involves operations on organs like stomach, liver, pancreas and gut. Whereas, general surgery involves an inhomogeneous spectrum of operations, e.g. surgery of the thyroid gland, hernias and proctology. Surgery is a field with high potential for innovation because of the constant development of new interventions, especially with regard to medical devices. Implementation of new interventions is commonly justified on the basis of clinical trials. Thus, any industry bias would have a relevant impact on surgical practice. This protocol describes the methods to perform a systematic literature search to find a representative sample of trials for a primary statistical analysis. The influence of industry sponsorship in general and abdominal surgery will be evaluated for the first time.
This study will firstly aim to determine whether there is an association between industry sponsorship and positive outcome in randomised controlled trials (RCTs) in the field of general and abdominal surgery.
Second, methodological differences between industry- and non-industry-funded RCT will be evaluated in order to explore potential sources of industry bias.
Systematic literature search methodology
One of the major challenges of this study will be the identification of a representative sample of trials within the field of general and abdominal surgery that is potentially at risk for industry bias. Trials without intrinsic commercial interest will have to be excluded from the study sample (Figure 1). Consequently, trials evaluating surgical strategies such as the Shouldice vs. Bassini operation for repair of inguinal hernia will not be investigated as no commercial interest can be directly related to the efficacy of the experimental intervention. On the basis of a preliminary literature screening, trials dealing with medical devices and pharmacological and nutritional interventions represent the vast majority of research with potential industry involvement.
A research question was formulated according to the Participants, Interventions, Comparisons and Outcomes (PICO) model . Figure 2 shows the preliminary PICO question with the search strategy. Due to the holistic approach, this search strategy was very unspecific, with precision below 3% on the abstract screening. Alteration of search terms did not improve precision substantially. However, abstracts identified by the preliminary search were screened for surgical interventions of interest and classified into 14 subfields (Figure 3). On this basis, a new form of comprehensive literature search was developed, the “multi-PICO search strategy” (MPSS):
–First, the preliminary PICO question served as the “master PICO”.
–Second, one or more specific questions called “minor PICOs” were created for every recorded subfield (e.g. Stapler, Figure 4).
Based on the minor PICOs, a systematic literature search will be performed independently by two reviewers following the recommendations of the Cochrane Collaboration . The following databases will be searched: Cochrane Library, MEDLINE (via PubMed) and EMBASE. A search strategy based on a vocabulary thesaurus (MeSH or Emtree) in combination with text words will be used. Additionally, a hand search in relevant citations will be performed. The search will be limited to the period from January 1985 to July 2014, with the rationale that disclosure of funding source was not demanded before 1985. No language restrictions will be applied.
The method of MPSS allowed the creation of a specific search for every subfield of investigation.
Articles gathered by the MPSS will be screened for eligibility according to the following criteria.
RCT assessing the efficacy or effectiveness of medical devices and perioperative pharmacological and nutritional interventions with direct relation to the surgical procedure in human patient populations in the field of general and abdominal surgery will be eligible.
RCT without funding information.
RCT assessing neurosurgical, urological, orthopaedic, dental, plastic, cardiothoracic, gynaecologic, dermal, vascular or paediatric surgery and otorhinolaryngological or endoscopic interventions.
Thus, a systematic search for surgical RCT in general and abdominal surgery with potential risk for industry bias will yield a broad and representative sample to answer the primary research question.
A trial will be classed as industry funded if any funding is explicitly stated, regardless of whether the funding took the form of direct financial support, supply of products for use in the study or the conduct of trial tasks, e.g. data analysis. Trials will be dichotomised according to whether the authors conclude the experimental intervention to be superior to the control intervention. The conclusions drawn by the authors will be compared to the data presented. A conclusion in favour of the experimental intervention based on non-significant differences between groups will be recorded. Risk of bias will be assessed according to the Cochrane Collaboration’s tool for assessing risk of bias . Further study characteristics will be captured for multivariate analysis as stated in the “Statistical analysis” section below.
Data extraction will be performed by two reviewers independently for quality assurance purposes . Discrepancies between the two reviewers will be resolved by a third reviewer, and a final extraction sheet will be determined for database entry. A database monitoring will be performed of 100% of data necessary to evaluate the primary endpoint and a randomly selected 20% of remaining data. Finally, the database will be closed and made available for statistical analysis.
A primary statistical analysis will be performed to answer the primary research question regarding the association of industry sponsorship and positive outcome as well as the magnitude of this association. Therefore, trials will be divided into those funded by industry and those not funded by industry. Further, trials will be dichotomised according to whether or not the experimental intervention is reported to be superior to the control group. The chance that industry-funded trials report more positive outcomes is expressed as odds ratio (OR). The null hypothesis (H0) is that industry funding is not associated with a positive trial outcome. The alternative hypothesis (H1) is that industry funding is associated with a positive trial outcome. The significance of association will be tested by means of Fisher’s exact test if at least one value in the contingency table is 5 or below. Pearson’s chi-squared test with Yates’s correction will be used if the total sample size is 60 or less. In all other cases, significance of association will be tested using Pearson’s chi-squared test without Yates’s correction at a level of significance of 5%. Furthermore, a multivariate logistic regression with factors (multinational trials, methodological quality) and covariates (number of study centres, journal impact factor, sample size) will be conducted. An additional analysis will be performed for the three subgroups medical devices, pharmaceuticals and nutrition.
Moreover, by comparison of reported p values of primary endpoints from industry-funded and independently funded trials, a possible industry bias will be quantified. Student’s t-test or the Wilcoxon rank-sum test will be used for exact p values. If 20% of p values are not reported exactly, the p values will be classified and Fisher’s exact test or a chi-squared test will be performed.
Additional data extracted will be presented descriptively. Publication bias will be explored using a funnel plot separately for industry-sponsored trials and non-industry- sponsored trials. Statistical analysis will be performed with R.
Existing literature about association of industry sponsorship and positive outcome in surgery has major limitations due to the approach to primary trials as mentioned above. Therefore, the presence and extent of such association in randomised controlled trials in general and abdominal surgery remains unexplored.
In the case of the hypothesised association of industry sponsorship and positive outcome, this study will investigate by a multivariate statistical analysis whether industry involvement biases result via standard risk domains or if industry involvement is an independent source of bias as assumed by several studies [4, 20].
The conduct of this study is important, because the detection of an industry bias in surgery would have an impact on future research. The findings of this study will enable surgical societies, in particular, to give advice about cooperation with industry and disclosure of funding source based on empirical evidence.
PP is a surgical resident and holds an MSc in clinical trial management. KG is a methodological specialist and core member of the surgical systematic review group at the Study Center of the German Surgical Society. PK is a board certified surgeon and head of the surgical clinical trial unit. AU is chief consultant in the surgical department and head of surgical oncology. MWB is full professor of the department of general, visceral and transplantation surgery. MKD is a consultant in the surgical department and head of the Study Center of the German Surgical Society.
controlled clinical trial
multi-PICO search strategy
Participants Interventions, Comparisons and Outcomes
Randomised controlled trials.
Relman AS: The new medical-industrial complex. N Engl J Med. 1980, 303: 963-970. 10.1056/NEJM198010233031703.
Relman AS: Dealing with conflicts of interest. N Engl J Med. 1984, 310 (18): 1182-1183. 10.1056/NEJM198405033101809.
Davidson RA: Source of funding and outcome of clinical trials. J Gen Intern Med. 1986, 1 (3): 155-158. 10.1007/BF02602327.
Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L: Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012, 12: MR000033-
Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ: Association between industry funding and statistically significant pro-industry findings in medical and surgical randomised trials. CMAJ. 2004, 170 (4): 477-480.
Ezzet KA: The prevalence of corporate funding in adult lower extremity research and its correlation with reported results. J Arthroplasty. 2003, 18 (Suppl 1): 138-145.
Khan SN, Mermer MJ, Myers E, Sandhu HS: The roles of funding source, clinical trial outcome, and quality of reporting in orthopedic surgery literature. Am J Orthop (Belle Mead NJ). 2008, 37 (12): E205-E212. discussion E212
Leopold SS, Warme WJ, Fritz Braunlich E, Shott S: Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res. 2003, 415: 293-301.
Lubowitz JH, Appleby D, Centeno JM, Woolf SK, Reid JB: The relationship between the outcome of studies of autologous chondrocyte implantation and the presence of commercial funding. Am J Sports Med. 2007, 35 (11): 1809-1816. 10.1177/0363546507304720.
Lynch JR, Cunningham MR, Warme WJ, Schaad DC, Wolf FM, Leopold SS: Commercially funded and United States based research is more likely to be published; good-quality studies with negative outcomes are not. J Bone Joint Surg Am. 2007, 89 (5): 1010-1018. 10.2106/JBJS.F.01152.
Momeni A, Becker A, Bannasch H, Antes G, Blümle A, Stark GB: Association between research sponsorship and study outcome in plastic surgery literature. Ann Plast Surg. 2009, 63 (6): 661-664. 10.1097/SAP.0b013e3181951917.
Okike K, Kocher MS, Mehlman CT, Bhandari M: Conflict of interest in orthopaedic research. An association between findings and funding in scientific presentations. J Bone Joint Surg Am. 2007, 89 (3): 608-613. 10.2106/JBJS.F.00994.
Shah RV, Albert TJ, Bruegel-Sanchez V, Vaccaro AR, Hilibrand AS, Grauer JN: Industry support and correlation to study outcome for papers published in Spine. Spine. 2005, 30: 1099-1104. 10.1097/01.brs.0000161004.15308.b4.
Sun GH, Houlton JJ, MacEachern MP, Bradford CR, Hayward RA: Influence of study sponsorship on head and neck cancer randomised trial results. Head Neck. 2013, 35 (10): 1515-1520.
Yao F, Singer M, Rosenfeld RM: Randomised controlled trials in otolaryngology journals. Otolaryngol Head Neck Surg. 2007, 137 (4): 539-544. 10.1016/j.otohns.2007.07.018.
Higgins JPT, Cochrane Handbook: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011, The Cochrane Collaboration, Available from http://www.cochrane-handbook.org. (Accessed 30.10.2014)
Kable AK, Pich J, Maslin-Prothero SE: A structured approach to documenting a search strategy for publication: a 12 step guideline for authors. Nurse Educ Today. 2012, 32 (8): 878-886. 10.1016/j.nedt.2012.02.022.
Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP: Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006, 59 (7): 697-703. 10.1016/j.jclinepi.2005.11.010.
R Core Team. R: A Language and Environment for Statistical Computing. 2013, Vienna, Austria: R Foundation for Statistical Computing, http://www.R-project.org. (accessed 30.10.2014)
Roseman M, Turner EH, Lexchin J, Coyne JC, Bero LA, Thombs BD: Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ. 2012, 345: e5155-10.1136/bmj.e5155.
No additional funding source is available. However, the resources and facilities of the University of Heidelberg were used in conducting this review.
The authors declare that they have no competing interests.
PP developed the study concept, wrote the first draft of the protocol and wrote the first draft of the protocol publication. KG developed the search strategy and gave methodological advice. MWB and AU provided scientific input for the study’s background and rationale. PK and MKD helped to develop the study concept and gave methodological advice. All authors read and approved the final manuscript.
Phillip Knebel and Markus K Diener contributed equally to this work.
Electronic supplementary material
Additional file 1: Detailed MPSS. Detailed information on the multi-PICO search strategy including search terms for MEDLINE. (PDF 93 KB)
Additional file 2: Extraction sheet. Standardised data collection form that will be used to extract data from included studies. (PDF 289 KB)
About this article
Cite this article
Probst, P., Grummich, K., Ulrich, A. et al. Association of industry sponsorship and positive outcome in randomised controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study. Syst Rev 3, 138 (2014). https://doi.org/10.1186/2046-4053-3-138
- Industry bias
- Industry sponsorship
- General and abdominal surgery
- Randomised controlled trial
- Medical devices
- Systematic review
- Science study
- Health care research