Helping people make well-informed decisions about health care: old and new challenges to achieving the aim of the Cochrane Collaboration

The Cochrane Collaboration’s aim

The aim of the Cochrane Collaboration is to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions [1].

Ten challenges at the turn of the 21st century

Progress since the turn of the 21st century

We have made substantial progress toward addressing the challenges listed in Box 2. The Cochrane Collaboration continues to generate and build on enthusiasm. The number of contributors has increased from more than 4,000 in 1999 to over 40,000 in 2013, including more than 24,000 review authors. There are now 54 review groups, 9 of which have been registered since 1999. The number of completed Cochrane reviews has increased from less than 700 (and more than 600 protocols) to over 5,000 (and more than 2,000 protocols).

Review groups continue to struggle to find a balance between accommodating the specific interests of review authors while avoiding overlapping reviews and reviews that are too narrowly or too broadly focused. New innovations that help address this challenge include overviews of reviews [6], priority-setting processes [7, 8], collaborating with and responding to the needs of stakeholder organizations [4] and developing and implementing policies that guide decisions about which new titles to register and for managing reviews that are within the scope of more than one review group [9]. New databases such as PDQ-Evidence (http://www.pdq-evidence.org) and Epistemonikos (http://www.epistemonikos.org) make it possible to quickly and easily identify related systematic reviews and see where there are gaps and overlap.

The Cochrane Database of Systematic Reviews (CDSR) has a high impact factor (5.9 in 2011, the 10th-highest impact factor among 153 journals categorized as Medicine, General and Internal, based on 7,721 citations of 1,306 reviews published in 2009 and 2010), and there is increased access to Cochrane reviews. As a consequence, fewer authors request to publish duplicate versions of Cochrane reviews in other journals, and the Collaboration now has a policy that virtually excludes this. Registration of Cochrane review titles has been an essential tool to reduce duplication of Cochrane reviews. More recently, a register of systematic review protocols for non-Cochrane reviews (PROSPERO) has been established to help reduce unplanned duplication of non-Cochrane systematic reviews [10].

Registering protocols can also help to reduce bias in reviews [7]. Other innovations to address the risk of bias include developing and implementing a clear policy regarding declarations of interest and commercial sponsorship [11] and a tool for systematically assessing and reporting risks of bias in included studies [12]. There have been a number of initiatives to help ensure that reviews address questions of global importance and avoid taking a perspective that is biased toward high-income countries [13–17]. These initiatives have also helped to accommodate diversity by enabling participation in the Collaboration by people in low- and middle-income countries (LMICs).

Access to the Cochrane Library has increased substantially. Over half of the world’s population now has one-click access that is free at the point of use, including free access in all low-income countries. Paid national licenses have also contributed to continuity and the sustainability of the Collaboration, among other things, by paying for methods innovations [18] and an Editorial Unit focused on improving the quality of the Cochrane Library [19]. This year, an open access policy was introduced, making all Cochrane reviews open access 12 months after publication with an option to pay a publication fee to make a review open access immediately. This policy and other policies that increase access to Cochrane reviews will likely further contribute to ensuring continuity and sustainability by helping to ensure demand for Cochrane reviews.

Summaries of findings tables and improvements in plain language summaries help to improve assessments of the certainty of the evidence summarized in Cochrane reviews and to communicate the key findings [20–22]. These innovations contribute further to ensuring continuity and sustainability by improving the accessibility and quality of Cochrane reviews, provided they are implemented across reviews. Currently only about 10% of reviews have a summary of findings table.

Current challenges

Although we have achieved a great deal over the past 20 years and substantial progress has been made toward achieving the Collaboration’s aims, a huge amount of work remains to be done. An updated list of challenges includes improving the quality of reviews, methodological challenges, meeting the needs of contributors and users and taking on new challenges while staying focused on the Collaboration’s aim.

Ten challenges (2013) 20 years after the launch of the Cochrane Collaboration

^*Characteristics of people (including their settings or contexts), interventions, the comparison, outcome measures, or study design that could potentially explain differences in results or limit the applicability of findings.

The challenge of keeping reviews up-to-date has increased since 1999. There are now many more reviews and little improvement in the efficiency of updating strategies. We need to further develop, evaluate and implement pragmatic strategies to update reviews effectively and efficiently. These include strategies for prioritizing which reviews need updating and when [23–25], more efficient search strategies, automated processes [26] and ensuring that review groups are adequately resourced and able to provide support to review authors. Radical thinking and substantial change may be needed, such as reducing the number of review groups and revamping editorial processes to ensure that they are efficient, support review authors and minimize the burden placed on authors, referees and editorial teams.

At the same time, there are many ways in which reviews could be improved and made more informative. The Methodological Expectations of Cochrane Intervention Reviews (MECIR) include 80 standards for the conduct of reviews and 108 for the reporting of reviews [27]. Many shortcomings can be identified in Cochrane reviews using the MECIR or other standards. Given that many review authors and editorial teams are already overwhelmed, it is necessary to set priorities regarding which improvements are most important and to take a gradual approach toward improving the reliability, readability and usefulness of Cochrane reviews. An example of a list of initial improvements that could be made in each new and updated review is shown below. Such lists may vary to some extent from review group to review group. They also will change over time as improvements are made and new priorities arise. However, the ultimate aim should remain to make Cochrane reviews as informative and useful as possible.

An initial list of improvements for new and updated Cochrane reviews

The development and application of better methods for addressing explanatory factors (in subgroup analyses, exploring heterogeneity or considering the applicability of results) [31–33], incorporating non-randomized evidence and making comparisons across multiple interventions [34] are needed. Incorporating nonrandomized evidence in Cochrane reviews may be important for a number of reasons. On the other hand, including nonrandomized evidence requires additional time, increases the difficulty of doing a review and may not be informative [35]. Although several review groups routinely conduct reviews of nonrandomized studies [36–38], much work is needed to develop pragmatic strategies for deciding when and how to incorporate nonrandomized evidence in Cochrane reviews [35, 39–42].

We have taken important steps toward supporting contributors in LMICs and ensuring the relevance of Cochrane reviews to people living in LMICs. It is necessary to continue and expand upon those efforts. This should include strategies for providing ongoing support to keep up with methodological developments, helping people whose first language is not English and securing sustainable sources of funding and protected time to work on reviews. Working with partners such as the World Health Organization is one important way of ensuring reviews meet the needs of people living in LMICs.

Innovations are needed to ensure that the needs of both contributors and users are met. Innovations that take advantage of CDSR being an electronic publication can play an important role in meeting the needs of diverse users. For example, interactive summary of findings tables would enable review authors to tailor their summary of findings to different target audiences (for example, health professionals, patients, policymakers and guideline developers) and speakers of different languages and would allow users to interact with the summaries by expanding or reducing the amount of information that is shown and accessing explanations and alternative presentations using numbers, text or visualizations [43]. Reviews could also incorporate interactive tables that address the relevance of the findings of reviews to different contexts [17, 44]. Comments could be solicited from people from different contexts with different perspectives that address both the applicability of the findings and could help to elucidate other factors (besides the evidence of effects) that need to be considered when making a decision [45].