Commentary
Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Helping people make well-informed decisions about health care: old and new challenges to achieving the aim of the Cochrane Collaboration

Systematic Reviews20132:77

https://doi.org/10.1186/2046-4053-2-77

©  Oxman; licensee BioMed Central Ltd. 2013

Received: 7 May 2013

Accepted: 22 May 2013

Published: 20 September 2013

Open Peer Review reports

Abstract

The aim of the Cochrane Collaboration is to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions. This aim is as relevant now as it was 20 years ago, when the Cochrane Collaboration was established. Substantial progress has been made toward addressing challenges to achieving the Collaboration’s aim. At the same time, a huge amount of work remains to be done. Current challenges include improving the quality of reviews, methodological challenges, meeting the needs of contributors and users and taking on new challenges while staying focused on the Collaboration’s aim. Radical thinking and substantial change may be needed to identify and implement pragmatic strategies to ensure that reviews are up-to-date and informative. Methodological challenges include the development and application of better methods for addressing explanatory factors, incorporating non-randomized evidence and making comparisons across multiple interventions. Innovations in editorial processes and strategies to meet the needs of low- and middle-income countries and diverse users of Cochrane reviews are needed. Finally, although it is important to consider broadening the aims of the Collaboration to include types of questions other than the effects of interventions and types of products other than the Cochrane Library, we should not lose sight of the aim of the Cochrane Collaboration. Addressing that aim is still a major challenge that requires the collaboration of thousands of people around the world and continuing improvements in the methods used to achieve that aim.

Background

At the turn of the 21st century, I described 10 major challenges to achieving the Cochrane Collaboration’s aim [1, 2]. Herein I consider progress in addressing those challenges and a new set of challenges. The need to address these challenges (by the Cochrane Collaboration or by others) remains the same; that is, that the alternative––poorly informed decisions––is not acceptable.

The Cochrane Collaboration’s aim

The aim of the Cochrane Collaboration is to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions [1].

Ten challenges at the turn of the 21st century

Ethical challenges
  1. 1

    Building on enthusiasm while avoiding duplication

     
  2. 2

    Building on enthusiasm while minimizing bias

     
  3. 3

    Promoting access while ensuring continuity

     
Social challenges
  1. 4

    Ensuring sustainability

     
  2. 5

    Accommodating diversity

     
Logistical challenges
  1. 6

    Identifying trials

     
  2. 7

    Managing criticisms and updating reviews

     
Methodological challenges
  1. 8

    Deciding what types of studies to include in reviews

     
  2. 9

    Summarizing the strength of evidence

     
  3. 10

    Effectively involving consumers

     

Discussion

Recent critiques of the Cochrane Collaboration have questioned the relevance of Cochrane reviews; the sustainability of the approach taken by the Cochrane Collaboration [3]; whether increasing rigor, demands on review authors and the cost of Cochrane reviews are warranted [3, 4]; and whether Cochrane reviews are not rigorous enough [5]. My view is that the aim of the Cochrane Collaboration is as relevant now as it was 20 years ago. We need methods that are both efficient and that produce reviews that are reliable and informative. Although we should continue to encourage and welcome criticism and to question the methods that we use, there is less reason to question the importance of our aim or the principles upon which our work is based [1].

Progress since the turn of the 21st century

We have made substantial progress toward addressing the challenges listed in Box 2. The Cochrane Collaboration continues to generate and build on enthusiasm. The number of contributors has increased from more than 4,000 in 1999 to over 40,000 in 2013, including more than 24,000 review authors. There are now 54 review groups, 9 of which have been registered since 1999. The number of completed Cochrane reviews has increased from less than 700 (and more than 600 protocols) to over 5,000 (and more than 2,000 protocols).

Review groups continue to struggle to find a balance between accommodating the specific interests of review authors while avoiding overlapping reviews and reviews that are too narrowly or too broadly focused. New innovations that help address this challenge include overviews of reviews [6], priority-setting processes [7, 8], collaborating with and responding to the needs of stakeholder organizations [4] and developing and implementing policies that guide decisions about which new titles to register and for managing reviews that are within the scope of more than one review group [9]. New databases such as PDQ-Evidence (http://www.pdq-evidence.org) and Epistemonikos (http://www.epistemonikos.org) make it possible to quickly and easily identify related systematic reviews and see where there are gaps and overlap.

The Cochrane Database of Systematic Reviews (CDSR) has a high impact factor (5.9 in 2011, the 10th-highest impact factor among 153 journals categorized as Medicine, General and Internal, based on 7,721 citations of 1,306 reviews published in 2009 and 2010), and there is increased access to Cochrane reviews. As a consequence, fewer authors request to publish duplicate versions of Cochrane reviews in other journals, and the Collaboration now has a policy that virtually excludes this. Registration of Cochrane review titles has been an essential tool to reduce duplication of Cochrane reviews. More recently, a register of systematic review protocols for non-Cochrane reviews (PROSPERO) has been established to help reduce unplanned duplication of non-Cochrane systematic reviews [10].

Registering protocols can also help to reduce bias in reviews [7]. Other innovations to address the risk of bias include developing and implementing a clear policy regarding declarations of interest and commercial sponsorship [11] and a tool for systematically assessing and reporting risks of bias in included studies [12]. There have been a number of initiatives to help ensure that reviews address questions of global importance and avoid taking a perspective that is biased toward high-income countries [13–17]. These initiatives have also helped to accommodate diversity by enabling participation in the Collaboration by people in low- and middle-income countries (LMICs).

Access to the Cochrane Library has increased substantially. Over half of the world’s population now has one-click access that is free at the point of use, including free access in all low-income countries. Paid national licenses have also contributed to continuity and the sustainability of the Collaboration, among other things, by paying for methods innovations [18] and an Editorial Unit focused on improving the quality of the Cochrane Library [19]. This year, an open access policy was introduced, making all Cochrane reviews open access 12 months after publication with an option to pay a publication fee to make a review open access immediately. This policy and other policies that increase access to Cochrane reviews will likely further contribute to ensuring continuity and sustainability by helping to ensure demand for Cochrane reviews.

Summaries of findings tables and improvements in plain language summaries help to improve assessments of the certainty of the evidence summarized in Cochrane reviews and to communicate the key findings [20–22]. These innovations contribute further to ensuring continuity and sustainability by improving the accessibility and quality of Cochrane reviews, provided they are implemented across reviews. Currently only about 10% of reviews have a summary of findings table.

Current challenges

Although we have achieved a great deal over the past 20 years and substantial progress has been made toward achieving the Collaboration’s aims, a huge amount of work remains to be done. An updated list of challenges includes improving the quality of reviews, methodological challenges, meeting the needs of contributors and users and taking on new challenges while staying focused on the Collaboration’s aim.

Ten challenges (2013) 20 years after the launch of the Cochrane Collaboration

Improving the quality of reviews
  1. 1

    Updating reviews

     
  2. 2

    Ensuring that reviews are informative

     
Methodological challenges
  1. 3

    Addressing explanatory factors*

     
  2. 4

    Incorporating evidence from nonrandomized studies

     
  3. 5

    Comparing multiple interventions

     
Meeting the needs of contributors and users
  1. 6

    Ensuring that editorial processes are effective and efficient

     
  2. 7

    Addressing the needs of low- and middle-income countries

     
  3. 8

    Meeting the needs of diverse users

     
Taking on new challenges while staying focused on the Collaboration’s aim
  1. 9

     Addressing different types of questions

     
  2. 10

     Preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions

     

*Characteristics of people (including their settings or contexts), interventions, the comparison, outcome measures, or study design that could potentially explain differences in results or limit the applicability of findings.

The challenge of keeping reviews up-to-date has increased since 1999. There are now many more reviews and little improvement in the efficiency of updating strategies. We need to further develop, evaluate and implement pragmatic strategies to update reviews effectively and efficiently. These include strategies for prioritizing which reviews need updating and when [23–25], more efficient search strategies, automated processes [26] and ensuring that review groups are adequately resourced and able to provide support to review authors. Radical thinking and substantial change may be needed, such as reducing the number of review groups and revamping editorial processes to ensure that they are efficient, support review authors and minimize the burden placed on authors, referees and editorial teams.

At the same time, there are many ways in which reviews could be improved and made more informative. The Methodological Expectations of Cochrane Intervention Reviews (MECIR) include 80 standards for the conduct of reviews and 108 for the reporting of reviews [27]. Many shortcomings can be identified in Cochrane reviews using the MECIR or other standards. Given that many review authors and editorial teams are already overwhelmed, it is necessary to set priorities regarding which improvements are most important and to take a gradual approach toward improving the reliability, readability and usefulness of Cochrane reviews. An example of a list of initial improvements that could be made in each new and updated review is shown below. Such lists may vary to some extent from review group to review group. They also will change over time as improvements are made and new priorities arise. However, the ultimate aim should remain to make Cochrane reviews as informative and useful as possible.

An initial list of improvements for new and updated Cochrane reviews

Ensuring that reviews are informative
  1. 1.

     Ensure that any important potential adverse effects of the interventions are addressed (whether the included studies report those outcomes or not).

     
  2. 2.

     Identify relevant disadvantaged groups and address differential effects and applicability to those groups in the Results and Discussion sections of the review [28].

     
  3. 3.

     Include summary of findings tables and justifying assessments of the certainty of the evidence [29]. Include full evidence profiles as appendices.

     
  4. 4.

     Ensure that the conclusions in the abstract, discussion and implications for practice are consistent with the summary of findings.

     
  5. 5.

     Interpret statistical significance correctly [30].

     
  6. 6.

     Base conclusions only on findings from the synthesis of included studies, and do not make recommendations.

     
  7. 7.

     Ensuring the methodological quality of reviews

     
  8. 8.

     Explain and justify any changes to the protocol.

     
  9. 9.

     Include risk of bias tables.

     
  10. 10.

     Provide a clear description of factors that affect interpretation and judgment about the reliability of any subgroup estimates [25–27]. Ensure that reviews are readable.

     
  11. 11.

     Ensure that results are reported consistently in the abstract, summary of findings and the Results and Discussion sections.

     
  12. 12.

     Keep the main text as short as possible; for example, document in appendices search strategies, lengthy aspects of the protocol that were not implemented and other details of the review that are not of interest to most readers.

     
  13. 13.

     Make sure the review is understandable to someone who is not familiar with the topic of the review and that it is easy to read.

     

The development and application of better methods for addressing explanatory factors (in subgroup analyses, exploring heterogeneity or considering the applicability of results) [31–33], incorporating non-randomized evidence and making comparisons across multiple interventions [34] are needed. Incorporating nonrandomized evidence in Cochrane reviews may be important for a number of reasons. On the other hand, including nonrandomized evidence requires additional time, increases the difficulty of doing a review and may not be informative [35]. Although several review groups routinely conduct reviews of nonrandomized studies [36–38], much work is needed to develop pragmatic strategies for deciding when and how to incorporate nonrandomized evidence in Cochrane reviews [35, 39–42].

We have taken important steps toward supporting contributors in LMICs and ensuring the relevance of Cochrane reviews to people living in LMICs. It is necessary to continue and expand upon those efforts. This should include strategies for providing ongoing support to keep up with methodological developments, helping people whose first language is not English and securing sustainable sources of funding and protected time to work on reviews. Working with partners such as the World Health Organization is one important way of ensuring reviews meet the needs of people living in LMICs.

Innovations are needed to ensure that the needs of both contributors and users are met. Innovations that take advantage of CDSR being an electronic publication can play an important role in meeting the needs of diverse users. For example, interactive summary of findings tables would enable review authors to tailor their summary of findings to different target audiences (for example, health professionals, patients, policymakers and guideline developers) and speakers of different languages and would allow users to interact with the summaries by expanding or reducing the amount of information that is shown and accessing explanations and alternative presentations using numbers, text or visualizations [43]. Reviews could also incorporate interactive tables that address the relevance of the findings of reviews to different contexts [17, 44]. Comments could be solicited from people from different contexts with different perspectives that address both the applicability of the findings and could help to elucidate other factors (besides the evidence of effects) that need to be considered when making a decision [45].

Conclusion

Although it is important to consider broadening the aims of the Collaboration to include other types of questions (such as reviews of diagnostic test accuracy [46] and qualitative evidence of factors affecting the implementation of health interventions [47]) and other products (such as derivative publications), we should not lose sight of the original aims of the Cochrane Collaboration: preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions. Addressing those aims is still a huge challenge that will continue to require the collaboration of thousands of people around the world and continuing improvements in the methods used to achieve those aims.

Declarations

Acknowledgements

I thank Iain Chalmers, Paul Garner, Claire Glenton, Sophie Hill, Sally Hopewell, Simon Lewin and Jimmy Volmink for their helpful comments on a draft of this paper.

Authors’ Affiliations

(1)
Global Health Unit, Norwegian Knowledge Centre for the Health Services

References

  1. Cochrane Collaboration: Description of the Cochrane Collaboration. Cochrane policy manual. Available at: http://www.cochrane.org/policy-manual/welcome
  2. Oxman AD: The Cochrane Collaboration in the 21st century: ten challenges and one reason why they must be met. Systematic reviews in health care: meta-analysis in context. Edited by: Egger M, Smith GD, Altman DG. 2001, London: BMJ Publishers, 459-473.View ArticleGoogle Scholar
  3. Brassey J: A critique of the Cochrane Collaboration: liberating the literature. (7 April 2013). Available at: http://blog.tripdatabase.com/2013/04/a-critique-of-cochrane-collaboration.html
  4. Bastian H, Glasziou P, Chalmers I: Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med. 2010, 7: e1000326-10.1371/journal.pmed.1000326. doi:10.1371/journal.pmed.1000326View ArticlePubMedPubMed CentralGoogle Scholar
  5. Doshi P, Jones M, Jefferson T: Rethinking credible evidence synthesis. BMJ. 2012, 344: d7898-10.1136/bmj.d7898.View ArticlePubMedGoogle Scholar
  6. Becker LA, Oxman AD: Chapter 22: overviews of reviews. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). Edited by: Higgins JPT, Green S. Available at: http://handbook.cochrane.org/
  7. Bero LA, Binder L: The Cochrane Collaboration review prioritization projects show that a variety of approaches successfully identify high-priority topics. J Clin Epidemiol. 2013, 66: 472-473. 10.1016/j.jclinepi.2012.03.015.View ArticlePubMedGoogle Scholar
  8. Cochrane agenda and priority setting methods group. Available at: http://capsmg.cochrane.org/
  9. Cochrane Collaboration: Guidelines for managing reviews with overlapping scope (section 2.2.15). Cochrane policy manual. Oxford, UK: Author, 196-198. Available at: http://www.cochrane.org/sites/default/files/uploads/Cochrane_Policy_Manual.pdf
  10. Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L: Establishing a minimum dataset for prospective registration of systematic reviews: an international consultation. PLoS One. 2011, 6: e27319-10.1371/journal.pone.0027319. doi:10.1371/journal.pone.0027319View ArticlePubMedPubMed CentralGoogle Scholar
  11. Cochrane Collaboration: Declaration of interest (section 1.5.2) and Commercial sponsorship (section 2.3). Cochrane policy manual. Oxford, UK: Author, 110-111. 198–202. Available at: http://www.cochrane.org/organisational-policy-manual
  12. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA, Cochrane Bias Methods Group; Cochrane Statistical Methods Group: The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011, 343: d5928-10.1136/bmj.d5928. doi:10.1136/bmj.d5928View ArticlePubMedPubMed CentralGoogle Scholar
  13. Tharyan P, Clarke M, Green S: How the Cochrane Collaboration is responding to the Asian tsunami. PLoS Med. 2005, 2: e169-10.1371/journal.pmed.0020169. doi:10.1371/journal.pmed.0020169View ArticlePubMedPubMed CentralGoogle Scholar
  14. Cochrane Collaboration Steering Group: Enhancing global participation: development of the Cochrane Academy. Cochrane Collaboration Steering Group Bulletin No. 14. 2012, Oxford, UK, 3-Available at: http://www.cochrane.org/sites/default/files/uploads/2012%20ccsg%20bulletin%20%2014%20_0.pdf Google Scholar
  15. Effective Health Care Research Consortium: Effective health care. Available at: http://www.evidence4health.org/
  16. EPOC Review Group: Norwegian satellite of the Cochrane Effective Practice and Organisation of Care Group: Available at: http://epocoslo.cochrane.org/
  17. Rosenbaum SE, Glenton C, Wiysonge CS, Abalos E, Mignini L, Young T, Althabe F, Ciapponi A, Marti SG, Meng Q, Wang J, la Hoz Bradford AM, Kiwanuka SN, Rutebemberwa E, Pariyo GW, Flottorp S, Oxman AD: Evidence summaries tailored to health policy-makers in low- and middle-income countries. Bull World Health Organ. 2011, 89: 54-61. 10.2471/BLT.10.075481. doi:10.2471/BLT.10.075481View ArticlePubMedGoogle Scholar
  18. Chandler J: Methods innovation fund. Cochrane Database Syst Rev. 2012, 1: 4-5.Google Scholar
  19. Cochrane Editorial Unit: Available at: http://www.editorial-unit.cochrane.org
  20. Rosenbaum SE, Glenton C, Nylund HK, Oxman AD: User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010, 63: 607-619. 10.1016/j.jclinepi.2009.12.013.View ArticlePubMedGoogle Scholar
  21. Rosenbaum SE, Glenton C, Oxman AD: Summary of Findings tables improved understanding and rapid retrieval of key information in Cochrane reviews. J Clin Epidemiol. 2010, 63: 620-626. 10.1016/j.jclinepi.2009.12.014.View ArticlePubMedGoogle Scholar
  22. Glenton C, Santesso N, Rosenbaum S, Nilsen ES, Rader T, Ciapponi A, Dilkes H: Presenting the results of Cochrane Systematic Reviews to a consumer audience: a qualitative study. Med Decis Making. 2010, 30: 566-577. 10.1177/0272989X10375853. doi:10.1177/0272989X10375853View ArticlePubMedGoogle Scholar
  23. Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, Barrowman N: A systematic review identified few methods and strategies describing when and how to update systematic reviews. J Clin Epidemiol. 2007, 60: 1095-1104.View ArticlePubMedGoogle Scholar
  24. Chung M, Newberry SJ, Ansari MT, Yu WW, Wu H, Lee J, Suttorp M, Gaylor JM, Motala A, Moher D, Balk EM, Shekelle PG: Two methods provide similar signals for the need to update systematic reviews. J Clin Epidemiol. 2012, 65: 660-668. 10.1016/j.jclinepi.2011.12.004. doi:10.1016/j.jclinepi.2011.12.004View ArticlePubMedPubMed CentralGoogle Scholar
  25. Cochrane Editorial Unit: Fit for purpose: centralised updating support for high priority Cochrane Reviews. Available at: http://www.editorial-unit.cochrane.org/fit-purpose-centralised-updating-support-high-priority-cochrane-reviews
  26. Tsafnat G, Dunn A, Glasziou P, Coiera E: The automation of systematic reviews. BMJ. 2013, 346: f139-10.1136/bmj.f139. doi:10.1136/bmj.f139View ArticlePubMedGoogle Scholar
  27. Higgins J, Churchill R, Lasserson T, Chandler J, Tovey D: Update from the Methodological Expectations of Cochrane Intervention Reviews (MECIR) project. Cochrane methods. Edited by: Chandler J, Clarke M, Higgins J. 2012, Chichester, UK: John Wiley & Sons, 2-3. Available at: http://www.thecochranelibrary.com/SpringboardWebApp/userfiles/ccoch/file/Files/coch_Method_2012%5B1%5D.pdf Google Scholar
  28. Munabi-Babigumira SM, Johansen M, Lewin S, Odegaard-Jensen J, Opiyo N, Oxman AD: Equity considerations in EPOC reviews. EPOC-specific resources for review authors. 2013, Oslo: Norwegian Knowledge Centre for the Health Services, Available at: http://epocoslo.cochrane.org/epoc-specific-resources-review-authors Google Scholar
  29. Guyatt GH, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J: GRADE guidelines 1. Introduction - GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011, 64: 383-94. 10.1016/j.jclinepi.2010.04.026.View ArticlePubMedGoogle Scholar
  30. Munabi-Babigumira SM, Johansen M, Lewin S, Odegaard-Jensen J, Opiyo N, Oxman AD: Results should not be reported as statistically significant or statistically non-significant. EPOC-specific resources for review authors. 2013, Oslo: Norwegian Knowledge Centre for the Health Services, Available at: http://epocoslo.cochrane.org/epoc-specific-resources-review-authors Google Scholar
  31. Munabi Babigumira SM, Johansen M, Lewin S, Odegaard-Jensen J, Opiyo N, Oxman AD: What are explanatory factors and why should they be included in protocols?. EPOC-specific resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services, Available at: http://epocoslo.cochrane.org/epoc-specific-resources-review-authors
  32. Sun X, Briel M, Walter SD, Guyatt GH: Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ. 2010, 340: 850-854. 10.1136/bmj.c850.View ArticleGoogle Scholar
  33. Oxman AD: Subgroup analyses: the devil is in the interpretation. BMJ. 2012, 344: e2022-10.1136/bmj.e2022. doi:10.1136/bmj.e2022View ArticlePubMedGoogle Scholar
  34. The Cochrane Comparing Multiple Interventions Methods Group: Available at: http://cmimg.cochrane.org
  35. Glenton C, Lewin S, Mayhew A, Scheel I, Odgaard-Jensen J: Nonrandomized studies are not always found even when selection criteria for health systems intervention reviews include them: a methodological study. J Clin Epidemiol. 2013, 66: 367-370. 10.1016/j.jclinepi.2012.11.009.View ArticlePubMedGoogle Scholar
  36. Munabi Babigumira SM, Johansen M, Lewin S, Odegaard-Jensen J, Opiyo N, Oxman AD: What study designs should be included in an EPOC review and what should they be called?. EPOC-specific resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services, Available at: http://epocoslo.cochrane.org/epoc-specific-resources-review-authors
  37. Armstrong R, Waters E, Jackson N, Oliver S, Popay J, Shepherd J, Petticrew M, Anderson L, Bailie R, Brunton G, Hawe P, Kristjansson E, Naccarella L, Norris S, Pienaar E, Roberts H, Rogers W, Sowden A, Thomas H: Guidelines for systematic reviews of health promotion and public health interventions. Version 2. Australia: Melbourne University, October 2007. Available at: http://ph.cochrane.org/review-authors
  38. Ryan R, Hill S, Broclain D, Horey D, Oliver S, Prictor M: Cochrane Consumers and Communication Review Group. Study design guide. 2009Google Scholar
  39. Rockers PC, Feigl AB, Røttingen JA, Fretheim A, de Ferranti D, Lavis JN, Melberg HO, Bärnighausen T: Study-design selection criteria in systematic reviews of effectiveness of health systems interventions and reforms: a meta-review. Health Policy. 2012, 104: 206-214. 10.1016/j.healthpol.2011.12.007. doi:10.1016/j.healthpol.2011.12.007View ArticlePubMedGoogle Scholar
  40. Golder S, Loke Y, McIntosh HM: Room for improvement?. A survey of the methods used in systematic reviews of adverse effects. BMC Med Res Methodol. 2006, 6: 3-PubMedGoogle Scholar
  41. Loke YK, Price D, Herxheimer A, Cochrane Adverse Effects Methods Group: Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. 2007, 7: 32-10.1186/1471-2288-7-32.View ArticlePubMedPubMed CentralGoogle Scholar
  42. Golder S, Loke YK, Bland M: Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med. 2011, 8: e1001026-10.1371/journal.pmed.1001026. doi:10.1371/journal.pmed.100View ArticlePubMedPubMed CentralGoogle Scholar
  43. Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): Work Package 5: Strategies for communicating evidence to inform decisions about health system and public health interventions: an interactive summary of findings to improve understanding and use of the evidence of the effects of interventions. Dundee, UK: DECIDE Project Office, Available at: http://www.decide-collaboration.eu/WP5/Workplan
  44. Department of Reproductive Health and Research, World Health Organization: The WHO Reproductive Health Library. Available at: http://www.who.int/hrp/rhl/en/
  45. Munabi-Babigumira SM, Johansen M, Lewin S, Odegaard-Jensen J, Opiyo N, Oxman AD: Implications for practice. Norwegian Knowledge Centre for the Health Services. 2013, Available at: http://epocoslo.cochrane.org/epoc-specific-resources-review-authors.Google Scholar
  46. Cochrane Diagnostic Test Accuracy Working Group: Cochrane Diagnostic Test Accuracy Working Group. Available at: http://srdta.cochrane.org/
  47. Cochrane Qualitative and Implementation Methods Group: Cochrane Qualitative and Implementation Methods Group. Available at: http://cqim.cochrane.org

Copyright

© Oxman; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement