Skip to content

Advertisement

  • Research
  • Open Access
  • Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Quality ratings of reviews in overviews: a comparison of reviews with and without dual (co-)authorship

  • Dawid Pieper1Email author,
  • Andreas Waltering2,
  • Jakob Holstiege3 and
  • Roland Brian Büchter2
Systematic Reviews20187:63

https://doi.org/10.1186/s13643-018-0722-9

Received: 3 May 2017

Accepted: 26 March 2018

Published: 24 April 2018

Abstract

Background

Previous research shows that many authors of Cochrane overviews were also involved in some of the included systematic reviews (SRs). This type of dual (co-)authorship (DCA) may be a conflict of interest and a potential source of bias. Our objectives were to (1) additionally investigate DCA in non-Cochrane overviews; (2) investigate whether there is an association between DCA and quality assessments of SRs in Cochrane and non-Cochrane overviews.

Methods

We selected a sample of Cochrane (n = 20) and non-Cochrane (n = 78) overviews for analysis. We extracted data on the number of reviews affected by DCA and whether quality assessment of included reviews was conducted independently. Differences in mean quality scores between SRs with and without DCA were calculated in each overview. These differences were standardized (using the standardized mean difference (SMD)) and meta-analyzed using a random effects model.

Results

Forty out of 78 non-Cochrane overviews (51%) and 18 out of 20 Cochrane overviews (90%) had included at least one SR with DCA. For Cochrane overviews, a median of 5 [interquartile range (IQR) 2.5 to 7] SRs were affected by DCA (median of included reviews 10). For non-Cochrane overviews a median of 1 [IQR 0 to 2] of the included SRs were affected (median of included reviews 14). The meta-analysis showed a SMD of 0.58 (95% confidence interval (CI) 0.27 to 0.90) indicating higher quality scores in reviews with overlapping authors. The test for subgroup differences shows no evidence of a difference between Cochrane (SMD 0.44; 95% CI 0.07 to 0.81) and non-Cochrane overviews (SMD 0.62; 95% CI 0.06 to 1.17).

Conclusions

Many authors of overviews also often have an authorship on one or more of the underlying reviews. Our analysis shows that, on average, authors of overviews give higher quality ratings to SRs in which they were involved themselves than to other SRs. Conflict of interest is one explanation, but there are several others such as reviewer expertise. Independent and blinded reassessments of the reviews would provide more robust evidence on potential bias arising from DCA.

Keywords

Conflict of interestOverviewSystematic reviewMethods

Background

Overviews of systematic reviews (henceforth termed overviews) attempt to systematically retrieve, assess, and synthesize the results of multiple systematic reviews (SRs) for a given condition or public health problem [1]. The number of published overviews is rapidly increasing [2, 3].

“Systematic reviewers” has become a term for people conducting SRs. We would expect systematic reviewers to also be involved in the conduct of overviews. Thus, authors of overviews might include SRs into their overviews which they have (co-)authored. We employ the term dual (co-)authorship (DCA) to describe this scenario [4]. Such an overlap in authorship can be considered a competing interest and raises questions regarding conflicts of interests. In theory, several steps in the conduct of an overview may be biased by DCA such as formulating inclusion criteria, conducting quality assessments, interpreting data, drawing conclusions, or dealing with competing reviews. Experts in a given field might be more likely to participate in an overview, while being enthusiastic for specific interventions, or have strong views about its effectiveness, for instance. Their opinion might also be bias by financial conflicts of interest. For example, a recent analysis found review sponsorship and authors’ financial conflicts of interest to introduce bias affecting the outcomes of reviews that could not be explained by other sources of bias [5].

In a sample of 197 Cochrane reviews, 28 (14%) were affected by DCA. DCA was mentioned in 68% (19/28) of the cases as a potential conflict of interest [6]. Our former study found that most (90%) Cochrane overviews were affected by DCA (i.e., at least one of the included reviews was affected by DCA) [4]. In 9/18 (50%) Cochrane overviews with DCA, quality assessment was not conducted independently (i.e., at least one person who (co-)authored the review was involved in quality assessment). To the best of our knowledge, no such data are available for non-Cochrane overviews. Furthermore, our former analysis focused only on the prevalence and management of DCA.

In this study, our objectives are to (1) investigate DCA in non-Cochrane overviews; (2) investigate whether there is an association between DCA and quality assessments of SRs in Cochrane and non-Cochrane overviews.

Methods

There was no a priori protocol for the study.

Given that our study had two objectives, the methods and results section are each separated into two parts. The first deals with the analysis of DCA in non-Cochrane overviews and a comparison with Cochrane overviews. The second describes a comparison of quality assessments of reviews with and without DCA using meta-analytical methods. The second part of the analysis comprises data of Cochrane and non-Cochrane reviews. The data on Cochrane overviews is taken from our former study [4].

DCA in non-Cochrane overviews

To allow for the comparability of results, the methods for the analysis of non-Cochrane overviews followed the same methods as our former study on Cochrane overviews [4]. In brief, we searched MEDLINE(via Pubmed) with a precision-maximizing search strategy (overview[ti] AND reviews[ti]) for overviews published from 2010 to September 2015. Our definition of an overview followed criteria outlined as follows [7]:
  1. 1.)

    Overviews should contain a clearly formulated objective designed to answer a specific clinical research question, typically about a healthcare intervention.

     
  2. 2.)

    Overviews should intend to search for and include only systematic reviews (with or without meta-analyses).

     
  3. 3.)

    Overviews should use explicit and reproducible methods to identify multiple systematic reviews that meet their inclusion criteria and to assess the methodological quality of these systematic reviews.

     
  4. 4.)

    Overviews should intend to collect, analyze, and present the descriptive characteristics of their included systematic reviews (and their primary studies) and the quantitative outcome data contained within the systematic reviews.

     

Protocols were excluded. In cases where updates were published, we used the most recent version. Overview selection was performed applying liberal acceleration (i.e., all titles and abstracts were screened by one reviewer; those deemed not relevant were verified by a second person for exclusion). All data were extracted by one person and verified by a second person. Data were extracted on the same items as in our former study [4]. Data were analyzed descriptively as frequencies or medians and interquartile ranges (IQR). To compare Cochrane and non-Cochrane overviews we used the Mann-Whitney U statistic and calculated odds ratios with 95% confidence intervals.

Comparison of quality assessments of reviews with and without DCA

We used meta-analytical methods to compare the quality assessments of the included SRs with versus without DCA. For this, we extracted data on quality assessments of the SRs from the overviews. The methodological quality of the included SRs was assessed in the overviews using various tools and was reported in different ways.

The Assessment of Multiple Systematic Reviews (AMSTAR) tool [8], R(evised)-AMSTAR [9] and the Overview Quality Assessment Questionnaire (OQAQ) [10] were frequently used to assess the methodological quality of SRs. AMSTAR consists of 11 items, each of which is categorized into a standardized set of four possible responses: “yes”, “no”, “cannot answer”, or “not applicable” [8]. The OQAQ was used in the development of AMSTAR. R-AMSTAR was developed to quantify the methodological quality by assigning a quality score to each SR ranging from 11 to 44, with higher scores indicating higher quality [9]. The OQAQ consists of 10 items, the first nine of which focus on methodological aspects of the scientific quality of a SR, while the last item provides an overall assessment based on an ordinal scale ranging from 1 to 7, with higher scores indicating less flaws (i.e., higher quality) [10]. The first nine questions each have three possible responses: “yes,” “no,” or “partial/cannot tell.”

For AMSTAR, a total score can be derived by summing up the number of “yes” items. This was done when authors did not present an overall score. Where overall scores were reported, these were extracted along with information on how a score was calculated to account for modifications of the original tool. In this regard, R-AMSTAR and the OQAQ were treated in the same way. In cases where authors applied or reported the results of the quality assessment on an ordinal scale (i.e., high, medium, low quality) we assigned numerical values, i.e., “high” was given a score of three, “medium” was given a score of 2, and low was given a score of 1, so that a higher value indicates a higher methodological quality. All data extractions were performed by one person and checked by a second for accuracy. Disagreements were resolved by discussion. We did not approach any authors for additional data.

Overviews had to meet the following criteria to be eligible for inclusion in the meta-analytical analysis:
  • At least two reviews affected by DCA and two reviews not affected by DCA included (to allow for calculation of a SD)

  • SD greater than 0 (i.e., the quality ratings varied across SRs)

Within each overview, we calculated the difference in mean quality score between SRs with and without DCA. These differences were standardized by the pooled SD. We conducted random effects meta-analyses (MAs) using DerSimonian and Laird’s heterogeneity variance estimator. All analyses were performed with RevMan 5.3. The included SRs served as units of analysis. The standardized mean difference (SMD) was chosen as the principal summary measure in meta-analysis to account for different scales. Mean differences (MD) were calculated when all overviews used the same scale. We used I2 to quantify inconsistency [11].

We expected skewed data due to unbalanced and small frequencies of reviews per group. Therefore, we checked the data by calculating the observed mean minus the lowest possible value (e.g., 1 for AMSTAR) and by dividing this by the SD [11]. A ratio less than 2 suggests skew, while there is strong evidence for a skewed distribution if the ratio is less than 1 [12]. We performed a sensitivity analysis by excluding all overviews where the ratio was less than 2 in any of the two groups, i.e., reviews with or without DCA. A subgroup analysis was also performed for Cochrane overviews and non-Cochrane overviews. Further meta-analyses were conducted for overviews using the original quality assessment instruments without any modifications. We were not able to investigate the impact of independent (i.e., quality assessment is performed by authors without DCA) versus non-independent quality assessment of SRs in overviews with overlapping authors due to (too) few overviews in this subsample.

Results

DCA in non-Cochrane overviews

In total, we included 78 non-Cochrane overviews (see Appendix for list of included and excluded overviews). They included a median of 14 reviews (interquartile range (IQR), 8.25–24). In 40 of 78 non-Cochrane overviews (51%), at least one of the included reviews was affected by DCA, and a median of 1 (IQR, 0–2) reviews per non-Cochrane overview were affected by DCA. In 8 out of these 40 overviews (20%), quality assessment was conducted independently. Two non-Cochrane overviews affected by DCA described this as a limitation, and four as a declaration of interest. Safeguards against potential bias arising from DCA were described in two non-Cochrane overviews. Table 1 illustrates this by contrasting these figures to the results of our former study on Cochrane overviews.
Table 1

Comparison of Cochrane overviews and non-Cochrane overviews

 

Cochrane overviews (n = 20)

Non-Cochrane overviews (n = 78)

Mann-Whitney U test (p value)

Odds ratio (95% CI)

Included SRs (median, IQR)

10 (6; 18.5)

14 (8.25; 24)

0.184

 

SRs affected by DCA per overview (median, IQR)

5 (2.5; 7)

1 (0; 2)

<  0.001

 

Overviews affected by DCA

18 (90%)

40 (51%)*

 

8.6 (1.9–39.4)

Independent assessment

8 (44%)

8 (20%)

 

3.2 (1.0–10.7)

SRs not assessed independently (median, IQR)

0 (0; 4.25)

1 (0; 2)

0.787

 

Reported as limitation

5 (28%)

2 (5%)

 

7.3 (1.3–42.3)

Reported as declaration of interest

11 (61%)

4 (10%)

 

11.0 (2.8–42.8)

Safeguard

7 (39%)

2 (5%)

 

12.1 (2.2–66.8)

CI confidence interval, SRs systematic reviews, IQR interquartile range, DCA dual (co-)authorship

*Two overviews were assessed as partly and are not included in the denominator

Results from meta-analytical comparison

Out of 20 Cochrane overviews and 78 non-Cochrane overviews included in the descriptive analysis, 14 overviews (6 Cochrane overviews and 8 non-Cochrane overviews) were included in the meta-analysis (see Fig. 1). All Cochrane overviews applied AMSTAR to assess the methodological quality of the included SRs. Four of these applied the original instrument [1316], while one of them calculated a percentage score to account for “not applicable” responses [16]. Two Cochrane overviews modified AMSTAR allowing for a maximum score of 10 [17, 18]. Among non-Cochrane overviews, three applied the original AMSTAR version [1921], while five applied the original OQAQ version [2226].
Figure 1
Fig. 1

Flow Chart

In all but three overviews mean quality scores were higher for reviews with DCA. The meta-analysis showed a SMD of 0.58 (95% confidence interval (CI) 0.27 to 0.90) indicating higher quality scores in reviews with overlapping authors (see Fig. 2). There was little inconsistency in the observed SMDs (I2 = 19%, p = 0.24). The test for subgroup differences shows no evidence of a difference between Cochrane (SMD 0.44; 95% CI 0.07 to 0.81) and non-Cochrane overviews (SMD 0.62; 95% CI 0.06 to 1.17). The difference in subgroup estimates was 0.18 (95% CI − 0.48 to 0,84, p value 0.60) as calculated by Z test. There was some evidence of inconsistency in the meta-analysis for non-Cochrane overviews (I2 = 45%, p = 0.08), while no inconsistency was observed for Cochrane overviews (I2 = 0%, p = 0.60).
Figure 2
Fig. 2

Mean quality scores

Six overviews were excluded due to skew data in the sensitivity analysis (see Fig. 3). All six excluded overviews were non-Cochrane overviews. Thus, the sensitivity analysis resembles the subgroup analysis for Cochrane overviews. However, the effect decreased to a SMD of 0.34 (95% CI − 0.00 to 0.69), with no evidence of inconsistency (I2 = 0%, p = 0.77).
Figure 3
Fig. 3

Mean quality scores (sensitivity analysis)

In total, six overviews used the original AMSTAR version without any modifications. The meta-analysis showed that reviews affected by DCA were scored one point higher than reviews not affected by DCA with respect to their methodological quality. The MD was 1.06 (95% CI − 0.31 to 2.44), with strong evidence of inconsistency (I2 = 72%, p = 0.003) (see Fig. 4). The effect was stronger for the OQAQ. The MD was 1.92 (95% CI 1.19 to 2.65), with no evidence of inconsistency (I2 = 0%, p = 0.53) based on five overviews (see Fig. 5).
Figure 4
Fig. 4

Mean quality scores for AMSTAR

Figure 5
Fig. 5

Mean quality scores for OQAQ

Discussion

Within this study, we compared DCA of Cochrane and non-Cochrane overviews and investigated whether there is an association between DCA and quality assessments of SRs in Cochrane and non-Cochrane overviews.

The comparison of Cochrane overviews with non-Cochrane overviews revealed significant differences with respect to the prevalence of DCA. While nearly all Cochrane overview were affected by DCA to some extent, this was the case for only half of non-Cochrane overviews (90 vs 51%). Furthermore, the proportion of reviews affected by DCA was much larger in Cochrane overviews. Since the Cochrane Collaboration is dedicated to evidence syntheses, we would expect clustering of authors in overviews. The higher proportion of overlaps in Cochrane overviews can be explained by the fact that Cochrane overviews usually exclude non-Cochrane reviews [4].

However, authors of Cochrane overviews were also more aware of the problems that might arise from dual (co-)authorship. They more often considered DCA to be a limitation or reported it in the declaration of interest section. Also, quality assessments of included reviews with DCA were more often conducted by authors not involved in the reviews. This may be due to higher awareness of conflict of interests among Cochrane reviewers or Cochrane policies. Both the declarations of interests section as well as Cochrane’s code of conduct in the Cochrane Handbook emphasize independence, transparency and acknowledgement of conflicts of interest [27]. Furthermore, minimizing bias by avoiding conflicts of interest is also stated as a goal in the fourth principle of the Cochrane Collaboration [28]. Specifically, the Cochrane policy stipulates that authors should not extract data from or assess quality of research they were involved in. Such stringent policies do not seem to exist for authors conducting overviews outside of the Cochrane Collaboration or other similarly spirited organizations. A recent survey of SRs showed that statements on conflicts of interest are more often (100 vs. 83%) included in Cochrane reviews than in non-Cochrane reviews [29]. In another survey, 97% of SRs reported conflict of interest disclosures [30]. In this study, which specifically looked at non-financial conflicts of interest, Cochrane authors more frequently reported such conflicts of interest compared to non-Cochrane authors (19 vs. 5%, p = 0.004) [30].

While the majority of medical journals require conflict of interest statements nowadays, only about half require statements on non-financial conflicts of interest and hardly any ask for intellectual conflicts of interest specifically [31], although conflicts of interest definitions often vary [32]. Intellectual conflicts of interest are defined as “academic activities that create the potential for an attachment to a specific point of view that could unduly affect an individual’s judgment about a specific recommendation” [33]. However, there is still a discussion in the scientific community about the presence of intellectual conflicts of interest [3436].

Despite the conflict of interest policies within the Cochrane Collaboration, our study showed that reviews affected by DCA obtained higher quality scores than reviews not affected DCA in Cochrane overviews. This finding also occurred in non-Cochrane overviews. Overviews with DCA scored one and two points higher in overviews applying the original AMSTAR or OQAQ tool, respectively. When interpreting this, it is important to keep in mind that the range of possible scores is 0–11 and 1–7 for AMSTAR and OQAQ, respectively. Thus, the difference of two points for the OQAQ is also more important in relative terms as the scale is shorter than for AMSTAR. A possible explanation for the difference between both tools observed here is the subjectivity of the OQAQ. However, there is no guidance on how to derive at the overall assessment. Counting the number of “yes” items in AMSTAR is therefore to a lesser extent subjective. It is also important to keep in mind that deriving an overall score is inherent in application of the OQAQ. A sum score is not mentioned for AMSTAR in its source publication and has never been validated [8]. It can be questioned whether any of these differences are relevant in terms of interpreting the methodological quality of a SR. Generally, a 1-point difference in AMSTAR should not reflect huge differences of methodological quality between SRs, although this might depend on the item affected by the judgment. For example, application of unjustified statistical methods will usually have a higher impact on the methodological quality of a SR than not providing a list of included and excluded studies. Nevertheless, it should be kept in mind that it is also common to categorize SRs based on their AMSTAR score. For example, the Canadian Agency for Drugs and Technologies in Health (CADTH) determines categories of quality as follows: low (score 0 to 3), medium (score 4 to 7), and high (score 8 to 11) [37]. When a cut-off is used as an inclusion or exclusion criterion for a SR within in overview, a 1-point difference may have an important impact.

This is the first study empirically assessing dual (co-)authorship in overviews. Although our results show that reviews affected by DCA obtain higher scores for methodological quality than reviews not affected by DCA, the difference is not necessarily a result of authors’ biased quality assessments. Our analysis was performed at the overview level, and we did not collect any content-specific characteristics from the included reviews. Several other aspects might also explain the results. It is well established in the literature that Cochrane reviews have a higher methodological quality than non-Cochrane reviews [9, 38, 39]. Unfortunately, we were not able to include this in our analysis due to the low number of reviews included in overviews. In addition, the methodological quality of SRs has risen over time [40, 41]. This might be of interest when comparing several health care interventions in an overview where some interventions are more up-to-date than others. Also, the comparison of health care interventions from different fields (e.g., pharmacology, surgery, complementary alternative medicine) might be important as the quality of SRs is not necessarily equal among disciplines. All these potential explanatory variables might have an impact on the results of our analysis if they are not equally distributed among reviews with and without DCA.

We are not able to draw any definite conclusions based on our findings. For example, we were not able to investigate the impact of independent quality assessment of SRs (i.e., quality assessment is performed by independent authors) in overviews with DCA due to a too low number of overviews. Thus, we were only able to differentiate between reviews with and without DCA. While doing so, we assumed that overview authors that have co-authored an included SR pose a potential conflict of interest to the whole overview, irrespective of their tasks performed. It has been stressed that although quality assessment of a review is not performed by any of its own authors, evaluating a review of one of the other group members might also introduce bias [19]. Another idea would be to ask independent authors to evaluate the quality of SRs in overviews. However, this approach seems not feasible as the actual authors will already have a guess about the methodological quality after having performed study selection and data extraction. Furthermore, authors may find it difficult to draw conclusions when they did not perform assessments themselves. Authors of overviews and SRs may also be very well aware of the advantages and drawbacks of quality assessment tools causing them to report what is necessary in order to receive as much points as possible on a quality assessment scale (e.g., AMSTAR or OQAQ). Therefore, our results could be explained by differences in reporting rather than methodological quality.

We encourage future authors of SRs and overviews to report who was involved in which steps of study selection, data collection, and quality assessment by providing initials of persons performing these steps. This would allow further analyses in the future as this would allow differentiating between overviews where authors affected by DCA were involved in quality assessment, for example. In other words, while the unit of analysis is the SR in the current analysis, it could be shifted to single authors.

Future studies are needed to investigate the influence of reviews affected by DCA and to identify ways of how to best deal with it. Also, it would be prudent to investigate whether overviews not affected by DCA have a lower methodological quality. This could be done by a reassessment and comparison of the methodological quality of included reviews. Assessors should be blinded against the aim of the study, and their assessments would be compared with the original ratings. In the absence of bias (i.e., reviews affected by DCA in fact do have a higher methodological quality), both assessments should be comparable in theory. If bias is present, we would assume that scores of reviews affected by DCA would be lower in the reassessment than their original ratings, while no such effect would be observed in reviews without DCA. Needless to say, the issue of DCA is not overview specific, but also arises with systematic reviews and primary studies.

Limitations

This study has some limitations which must be pointed out. First, our search strategy for the identification of non-Cochrane overviews followed a precision maximizing approach, thus our sample might lack representativeness. Second, we did not perform a sample size calculation before the study as sample sizes calculations are known to be difficult in this context [42]. Third, the analyzed data is skewed and unbalanced. This might question the use of standard methods for meta-analyses of parametric data. However, we have tried to estimate impact of skew data by excluding them in a sensitivity analysis. Fourth, our study is based on quality scores of reviews. We have also calculated scores in cases where the authors refrained from doing this. This is true for AMSTAR where the overall score has never been validated and might be an inappropriate measure of methodological quality. However, there is no alternative approach we could have opt for to investigate differences in quality scores of reviews with and without DCA. Lastly, four of the included non-Cochrane overviews were published by one group of authors [2326]. Thus, the results of these overviews cannot be seen completely independent of each other.

Conclusions

DCA frequently occurs in overviews. Nearly all Cochrane overviews are affected by DCA. Reviews with DCA obtain higher methodological quality scores than reviews without. Potential conflicts of interest are one explanation for this association. The reasons need to be further investigated, however. Authors need guidance what to do if they are going to include their own review.

Abbreviations

AMSTAR: 

The Assessment of Multiple Systematic Reviews

CADTH: 

Canadian Agency for Drugs and Technologies in Health

CI: 

Confidence interval

DCA: 

Dual (co-)authorship

IQR: 

Interquartile ranges

MD: 

Mean difference

OQAQ: 

Overview Quality Assessment Questionnaire

SD: 

Standard deviation

SMD: 

Standardized mean difference

SRs: 

Systematic reviews

Declarations

Acknowledgements

None

Funding

There was no external funding for the research or publication of this article.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

DP substantially contributed to the conception, analysis, and interpretation of the data for the work and to the drafting of the work. JH substantially contributed to the analysis and interpretation of the data for the work. RB substantially contributed to the conception, analysis, and interpretation of the data for the work. AW, JH, and RB revised the drafting of the work critically for important intellectual content. All authors contributed to the final approval of the version to be published and are in agreement to be accountable for all aspects of the work and in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Institute for Research in Operative Medicine, Witten/Herdecke University, Cologne, Germany
(2)
Institute for Quality and Efficiency in Health Care (IQWiG), Cologne, Germany
(3)
Central Research Institute of Ambulatory Health Care in Germany (ZI), Berlin, Germany

References

  1. Becker LA, Oxman AD: Overviews of reviews. In: Cochrane Handbook for Systematic Reviews of Interventions. edn.: Wiley; 2008: 607–631. https://www.handbook.cochrane.org. Accessed 1 Mar 2017.
  2. Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Pieper D, Buechter R, Jerinic P, Eikermann M. Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol. 2012;65(12):1267–73.View ArticlePubMedGoogle Scholar
  4. Buchter RB, Pieper D. Most overviews of Cochrane reviews neglected potential biases from dual authorship. J Clin Epidemiol. 2016;77:91–4.View ArticlePubMedGoogle Scholar
  5. Mandrioli D, Kearns CE, Bero LA. Relationship between research outcomes and risk of Bias, study sponsorship, and author financial conflicts of interest in reviews of the effects of artificially sweetened beverages on weight outcomes: a systematic review of reviews. PLoS One. 2016;11(9):e0162198.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Kliner M, Garner P. When trial authors write Cochrane Reviews: competing interests need to be better managed. Cochrane Database Syst Rev 2014(9):Ed000089.Google Scholar
  7. Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Syst Rev. 2016;5(1):190.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Popovich I, Windsor B, Jordan V, Showell M, Shea B, Farquhar CM. Methodological quality of systematic reviews in subfertility: a comparison of two different approaches. PLoS One. 2012;7(12):e50403.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44(11):1271–8.View ArticlePubMedGoogle Scholar
  11. Deeks JJ, Higgins JPT, Altman DG: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions. edn.: Wiley; 2008: 243–296. https://www.handbook.cochrane.org. Accessed 1 Mar 2017.
  12. Altman DG, Bland JM. Detecting skewness from summary information. BMJ (Clinical research ed). 1996;313(7066):1200.View ArticleGoogle Scholar
  13. Pollock A, Farmer SE, Brady MC, Langhorne P, Mead GE, Mehrholz J, van Wijck F: Interventions for improving upper limb function after stroke. Cochrane Database Syst Rev 2014(11):CD010820.Google Scholar
  14. Wu L, Norman G, Dumville JC, O'Meara S, Bell-Syer SE. Dressings for treating foot ulcers in people with diabetes: an overview of systematic reviews. Cochrane Database Syst Rev. 2015;7:CD010471.Google Scholar
  15. Singh JA, Christensen R, Wells GA, Suarez-Almazor ME, Buchbinder R, Lopez-Olivo MA, Tanjong Ghogomu E, Tugwell P. Biologics for rheumatoid arthritis: an overview of Cochrane reviews. Cochrane Database Syst Rev. 2009;4:CD007848.Google Scholar
  16. Cahill K, Stevens S, Perera R, Lancaster T. Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev. 2013;5:CD009329.Google Scholar
  17. Farquhar C, Rishworth JR, Brown J, Nelen WL, Marjoribanks J. Assisted reproductive technology: an overview of Cochrane reviews. Cochrane Database Syst Rev. 2015;7:CD010537.Google Scholar
  18. Brown J, Farquhar C. Endometriosis: an overview of Cochrane reviews. Cochrane Database Syst Rev. 2014;3:CD009590.Google Scholar
  19. Jacobs WC, Rubinstein SM, Willems PC, Moojen WA, Pellise F, Oner CF, Peul WC, van Tulder MW: The evidence on surgical interventions for low back disorders, an overview of systematic reviews. Eur Spine J 2013, 22(9):1936–1949.Google Scholar
  20. Kitsiou S, Pare G, Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. J Med Internet Res. 2015;17(3):e63.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Lauche R, Cramer H, Hauser W, Dobos G, Langhorst J. A systematic overview of reviews for complementary and alternative therapies in the treatment of the fibromyalgia syndrome. Evid Based Complement Alternat Med. 2015;2015:610615.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Ernst E, Lee MS. Acupuncture for rheumatic conditions: an overview of systematic reviews. Rheumatology (Oxford, England). 2010;49(10):1957–61.View ArticleGoogle Scholar
  23. Lee MS, Ernst E. Systematic reviews of t’ai chi: an overview. Br J Sports Med. 2012;46(10):713–8.View ArticlePubMedGoogle Scholar
  24. Lee MS, Ernst E. Acupuncture for surgical conditions: an overview of systematic reviews. Int J Clin Pract. 2014;68(6):783–9.View ArticlePubMedGoogle Scholar
  25. Lee MS, Kim JI, Ernst E. Is cupping an effective treatment? An overview of systematic reviews. J Acupunct Meridian Stud. 2011;4(1):1–4.View ArticlePubMedGoogle Scholar
  26. Lee MS, Oh B, Ernst E. Qigong for healthcare: an overview of systematic reviews. JRSM Short Reports. 2011;2(2):7.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Green S, Higgins JPT: Preparing a Cochrane Review. In: Cochrane Handbook for Systematic Reviews of Interventions. edn.: Wiley; 2008: 11–30. https://www.handbook.cochrane.org. Accessed 1 Mar 2017.
  28. Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD: Introduction. In: Cochrane Handbook for Systematic Reviews of Interventions. edn.: Wiley; 2008: 1–9. https://www.handbook.cochrane.org. Accessed 1 Mar 2017.
  29. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catala-Lopez F, Li L, Reid EK, Sarkis-Onofre R, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Hakoum MB, Anouti S, Al-Gibbawi M, Abou-Jaoude EA, Hasbani DJ, Lopes LC, Agarwal A, Guyatt G, Akl EA. Reporting of financial and non-financial conflicts of interest by authors of systematic reviews: a methodological survey. BMJ Open. 2016;6(8):e011997.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Shawwa K, Kallas R, Koujanian S, Agarwal A, Neumann I, Alexander P, Tikkinen KAO, Guyatt G, Akl EA. Requirements of clinical journals for authors’ disclosure of financial and non-financial conflicts of interest: a cross sectional study. PLoS One. 2016;11(3):e0152301.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Blum JA, Freeman K, Dart RC, Cooper RJ. Requirements and definitions in conflict of interest policies of medical journals. JAMA. 2009;302(20):2230–4.View ArticlePubMedGoogle Scholar
  33. Guyatt G, Akl EA, Hirsh J, Kearon C, Crowther M, Gutterman D, Lewis SZ, Nathanson I, Jaeschke R, Schunemann H. The vexing problem of guidelines and conflict of interest: a potential solution. Ann Intern Med. 2010;152(11):738–41.View ArticlePubMedGoogle Scholar
  34. Poole D. When does a point of view become an intellectual conflict of interest? Crit Care Med. 2008;36(5):1688. author reply 1688-1689View ArticlePubMedGoogle Scholar
  35. Sniderman AD, Furberg CD. Pluralism of viewpoints as the antidote to intellectual conflict of interest in guidelines. J Clin Epidemiol. 2012;65(7):705–7.View ArticlePubMedGoogle Scholar
  36. Bion J. Financial and intellectual conflicts of interest: confusion and clarity. Curr Opin Crit Care. 2009;15(6):583–90.View ArticlePubMedGoogle Scholar
  37. Methods for Development. https://www.cadth.ca/methods-development. Accessed 1 Mar 2017.
  38. Remschmidt C, Wichmann O, Harder T. Methodological quality of systematic reviews on influenza vaccination. Vaccine. 2014;32(15):1678–84.View ArticlePubMedGoogle Scholar
  39. Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis N. Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm? Eur J Orthod. 2013;35(2):244–8.View ArticlePubMedGoogle Scholar
  40. MacDonald SL, Canfield SE, Fesperman SF, Dahm P. Assessment of the methodological quality of systematic reviews published in the urological literature from 1998 to 2008. J Urol. 2010;184(2):648–53.View ArticlePubMedGoogle Scholar
  41. Samargandi OA, Hasan H, Thoma A. Methodologic quality of systematic reviews published in the plastic and reconstructive surgery literature: a systematic review. Plast Reconstr Surg. 2016;137(1):225e–36e.View ArticlePubMedGoogle Scholar
  42. Giraudeau B, Higgins JP, Tavernier E, Trinquart L. Sample size calculation for meta-epidemiological studies. Stat Med. 2016;35(2):239–50.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s). 2018

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement