Skip to main content


Publication of reviews synthesizing child health evidence (PORSCHE): a survey of authors to identify factors associated with publication in Cochrane and non-Cochrane sources



Cochrane Child Health maintains a register of child-relevant Cochrane systematic reviews (SRs) to provide a comprehensive source of high-quality evidence. However, a large number of SRs are published outside of The Cochrane Collaboration (Cochrane), impacting the comprehensiveness of the Cochrane Database of Systematic Reviews (CDSR). We surveyed authors who published child-relevant SRs with Cochrane and elsewhere in the medical literature to (1) understand their experiences in preparing and publishing SRs and (2) identify factors influencing choice of publication venue.


We identified SRs published in the CDSR for the most recent complete year prior to our study (2013; n = 145). We searched the medical literature and randomly selected the same number of SRs published the same year. We developed an internet-based survey and contacted the corresponding author of each review via email. Data were analyzed descriptively. Qualitative analysis elicited common themes from open-ended questions.


Seventy-six (26 %) responded: 41 % Cochrane, 42 % non-Cochrane, and 17 % published in both venues. Among respondents who published their SR in both venues (n = 13), 46 % found it easier to publish in a non-Cochrane journal, 15 % easier with Cochrane, and 31 % similar. Main reasons for conducting SRs with Cochrane (n = 44) were Cochrane’s positive reputation (82 %) and good impact factor (66 %). Among respondents who published their SR in a non-Cochrane journal (n = 32), most frequent reasons for not conducting their SR with Cochrane were time required to follow Cochrane processes (25 %), lack of knowledge about how to conduct an SR with Cochrane (19 %), administrative processes (16 %), and perception that non-Cochrane journals yielded more interest (16 %). Among respondents who published their SR in a non-Cochrane journal (n = 32), 78 % did not register their review and 22 % did not prepare a protocol.


Key reasons for publishing in Cochrane are its positive reputation and impact factor. Reasons for publishing in non-Cochrane sources include lack of familiarity or challenges with the Cochrane processes and desire to publish in a source more directly relevant to the topic of interest. End users looking for evidence in the form of SRs need to be aware that there is a vast number of SRs published across the medical literature. Efforts to optimize the identification of SRs in non-Cochrane sources (e.g., through effective labeling or protocol/review registration) and their content will help end users find the necessary synthesized evidence to support clinical practice.


The Cochrane Collaboration (Cochrane) [] was a pioneer in the area of evidence-based medicine. Established in 1993, it has been an advocate for the conduct and dissemination of systematic reviews of evidence on, initially, healthcare interventions and, more recently, in the area of diagnostic test accuracy. Cochrane’s systematic reviews are published in the online Cochrane Database of Systematic Reviews (CDSR), one of the databases included in The Cochrane Library [].

Individuals interested in conducting a systematic review through Cochrane are invited to register a title with one of 52 review groups which each focus on a specific area of health (e.g., acute respiratory infections, back and neck, breast cancer, etc.) [1]. Once the title is approved, review teams comprised of at least two authors prepare a protocol based on Cochrane guidance [2]. The protocol is submitted to the relevant review group which manages peer-review and publication of the protocol in the CDSR. After approval of the protocol, authors complete the systematic review; this is also submitted to the review group which manages peer-review and eventual publication in the CDSR. Additional details on the process for proposing and registering new reviews are available online [3]. The Cochrane Handbook provides detailed guidance for conducting systematic reviews; [2] further, Cochrane’s Central Editorial Unit [4] has established standards for protocols, reviews, and updates of reviews which are intended to guide the conduct and peer-review of Cochrane products [5].

Many systematic reviews are also conducted outside of Cochrane and published as technical reports by sponsoring agencies or in peer-reviewed medical journals. Many initiatives have emerged to increase the rigor of systematic reviews and transparency of reporting. One key initiative is the preferred reporting items for systematic reviews and meta-analyses (PRISMA) [6]. PRISMA provides a checklist to help improve reporting by systematic review authors and is endorsed by a number of editorial organizations and hundreds of journals [7].

In addition to review groups, Cochrane has networks pertaining to healthcare domains or populations of special interest [8]. These groups do not oversee the production of systematic reviews; rather, their functions are to facilitate linkages between Cochrane and external stakeholders, disseminate results of Cochrane reviews to the relevant stakeholders, and identify and tag titles, protocols, and reviews relevant to their scope [9]. They may also be involved in promoting the production of reviews, training, and conducting methodological research related to the production or dissemination of systematic reviews within their “field” or population of interest.

Cochrane Child Health is one such entity which was established to ensure that the unique health needs of children are reflected within The Cochrane Collaboration [], []. The Cochrane Library is a key output of Cochrane and an important source of synthesized evidence to inform healthcare delivery; however, to be most useful, it needs to include evidence that is up-to-date, comprehensive, of high methodological quality, and relevant to specific populations such as children. One crucial way of achieving this goal is to encourage authors of systematic reviews to publish their work in the CDSR. If authors find it easier and more appealing to publish their work in non-Cochrane arenas, this will dilute the comprehensiveness and therefore usefulness and applicability of the CDSR. Furthermore, it will be more difficult for clinicians and other decision-makers to quickly find relevant summarized data.

End users looking for evidence in the form of systematic reviews need to be aware that there are a vast number of reviews published outside of Cochrane. Previous research has shown that Cochrane reviews are of higher methodological quality; [10] it is also important to identify whether there are systematic differences in where authors choose to publish and the rationale for these choices. As the methods for systematic reviews of healthcare interventions have become more mainstream, and increasing numbers of clinicians and researchers have been trained in these methods and have experience completing systematic reviews, the facilitation provided by Cochrane may be less sought after and instead the processes that occur during review development may be perceived as a hindrance to timely publication.

Our goal was to understand the motivations of systematic reviewers for conducting and publishing their systematic reviews either with Cochrane (i.e., in the CDSR) or in other peer-reviewed journals. To meet our goal, we conducted a survey of authors who have completed and published child-relevant systematic reviews with the Cochrane Collaboration and elsewhere in the medical literature to (1) understand their experiences in preparing and publishing reviews and (2) identify factors that influenced choice of publication venue.


We created a sample of recently published child-relevant systematic reviews. We identified all completed systematic reviews of healthcare interventions published in the CDSR for the most recent complete year prior to our study (2013; n = 145). We felt that capturing reviews published during a complete year would provide a broad representation to identify variables of interest. We then searched the medical literature (Additional file 1: Appendix A) and randomly selected the same number of child-relevant systematic reviews focusing on healthcare interventions published in the same year. We first identified 4855 records of systematic reviews from the same year. We defined a systematic review as all of the following (Additional file 1: Appendix B): (a) included a literature search that contained more than one named database, or one named database plus another sources (e.g., checking references, handsearching, contact with researchers to identify published studies, citation searching, internet searching, or other systematic attempts to identify potential studies); (b) reported inclusion and/or exclusion criteria; (c) described methods for study selection and data extraction; (d) assessed methodological quality of included studies; and (e) described or summarized results of the included/relevant studies. Non English-language reviews were excluded.

The list of systematic review records was randomly ordered, and we screened citations until we found an equivalent number of systematic reviews (n = 145) meeting our eligibility criteria, i.e., therapeutic intervention, child-relevant (Additional file 1: Appendix B). We defined a child-relevant therapeutic intervention as either of the following: (a) an intervention intended to improve the health and well-being of children; and (b) studies on breastfeeding or nutritional supplements for fetal and/or newborn health. Studies without pediatric outcomes were excluded, as well as primarily descriptive reports of studies in a given field with no synthesis of results (e.g., a descriptive analysis characterizing RCTs conducted in child health). We defined child-relevant as (a) intended to include children 0 to 18 years; (b) contained adults and children; or (c) studied an intervention/condition intended to improve health and well-being of children. Systematic reviews that only examined or reported maternal outcomes were excluded. We then identified the corresponding author of each review. We ensured that none of the authors surveyed were duplicates within and between lists (Cochrane vs. non-Cochrane). Since our non-Cochrane list was generated by screening records until we identified our target number, some of the Cochrane reviews may have also been published in a non-Cochrane source.

We developed an internet-based survey using Research Electronic Data Capture (REDCap) software [11] (Additional file 1: Appendix C). Study data were collected and managed using REDCap electronic data capture tools hosted at the University of Alberta [11]. REDCap is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources [12].

Survey questions asked about: (1) why authors chose to publish their systematic reviews within or outside of the Cochrane Collaboration; (2) authors’ experiences preparing their systematic review; (3) whether they would undertake another review either within or outside of Cochrane; and (4) the reasoning and experiences of authors who chose to publish the same review in the CDSR and also in another journal. For most questions, we had pre-defined options that respondents could select as well as an “other” option with an opportunity for the respondent to elaborate with text descriptions. Prior to implementation, the survey was pilot tested among a convenience sample of five review authors (one was a co-author, RF, and four were independent of the study team) to ensure that the questions were worded appropriately and information was presented in a way that would elicit appropriate responses. The survey was revised based on their responses and feedback. Among the five individuals who pilot-tested the survey, three authors had published in both Cochrane and non-Cochrane sources, while two authors had published only in non-Cochrane sources.

We contacted the corresponding author of each review via email using the author’s published contact information. Participants were asked to respond to questions with respect to the specific publications (SR) we had identified. We identified 290 authors: 145 Cochrane authors and 145 non-Cochrane authors. We sent 290 emails with links to the survey directly to corresponding authors of the identified reviews. The survey was administered for 6 weeks, from April to May 2015. We sent two reminders following the initial email, each at 2-week intervals.

The study was approved by the University of Alberta’s research ethics board prior to implementation. All survey responses were anonymized. At the end of the survey, participants had the opportunity to enter their name in a draw for one of six US$50 Amazon gift cards (the names were entered on a separate web page and were not linked to the survey). We indicated that they would have about a 1 in 50 chance of winning based on our complete study sample; based on actual participation, the chance of winning was 1 in 13. Only one survey question was mandatory (i.e., respondents were required to complete the question before continuing with the survey); this was the first question asking authors to indicate if they had published with Cochrane, non-Cochrane or both. The remaining questions were based on response to this first question with slightly different questions asked of each group (Additional file 1: Appendix C). No other questions were mandatory in order to submit the survey. For incomplete questionnaires, we used the data for the questions that were answered; incomplete questions were marked as missing data. Respondents were able to amend answers at any time prior to submission of the survey (i.e., they could go backwards and forwards through the survey as they liked until they submitted). The survey invitation was linked to each participant’s email address. Participants were able to open the survey using the link as many times as they liked until they submitted the survey, at which time the link no longer worked. This ensured that we did not obtain repeat or duplicate responses.

Data were analyzed descriptively with univariate statistics using SPSS version 22.0. We divided respondents into three categories: (1) those who had only published their systematic review in the CDSR; (2) those who had published their review in both the CDSR and a non-Cochrane source (based on information provided by respondents); and (3) those who had only published their review in a non-Cochrane source. Qualitative analysis of the responses to open-ended survey questions was conducted to elicit common themes.


Seventy-six (26 %) individuals responded. Table 1 presents characteristics of the respondents; 41 % (n = 31) had published their review with Cochrane, 42 % (n = 32) in a non-Cochrane source, and 17 % (n = 13) in both. The majority of respondents were researchers (45 %; n = 34), clinician-scientists (32 %; n = 24), or clinicians (13 %; n = 10). A majority of participants had completed multiple systematic reviews (2–5, 41 % (n = 31); 6–10, 13 % (n = 10); 11–20, 5 % (n = 4); >20, 13 % (n = 10)), with 24 % (n = 18) of respondents having completed only one systematic review. A minority of the sample had no current involvement with Cochrane (38 %; n = 29); the proportion was higher for the authors who had only published their systematic review in a non-Cochrane source (75 %; n = 24).

Table 1 Characteristics of respondents and their intentions to participate in another systematic review

Among respondents who published their systematic review in both Cochrane and non-Cochrane journals (n = 13), the majority published in Cochrane first (77 %; n = 10). Nearly half reported it was easier to publish their systematic review in a non-Cochrane journal (46 %; n = 6); only 15 % (n = 2) found it easier with Cochrane, and 31 % (n = 4) rated ease of publishing equal between Cochrane and non-Cochrane journals. Respondents were divided in their rating of the timeliness of publishing with 31 % (n = 4) indicating it was more timely with Cochrane, 31 % (n = 4) more timely with a non-Cochrane journal, and 39 % (n = 5) indicating the same. A minority (31 %; n = 4) paid fees to publish their article in a peer-reviewed journal, and a similar proportion (31 %; n = 4) published in an open access journal.

Among respondents who published their systematic review in Cochrane (n = 44), the main reasons for conducting their systematic reviews with Cochrane were Cochrane’s positive reputation (82 %; n = 36) and good impact factor (66 %; n = 29) (respondents could select multiple response options). The majority who published their review with Cochrane were satisfied (48 %; n = 21) or very satisfied (32 %; n = 14). Respondents also rated the specific support they received (Fig. 1).

Fig. 1

Respondents’ rating of support they received through the Cochrane Collaboration

Table 2 summarizes responses of authors who published their systematic review in a non-Cochrane journal (n = 32). Nineteen percent (n = 6) had considered conducting their SR with Cochrane, while 75 % (n = 24) had not. The most frequent reasons for not conducting their systematic review with Cochrane were time required to follow Cochrane processes (25 %; n = 8), lack of knowledge about how to conduct a systematic review with Cochrane (19 %; n = 6), administrative processes (16 %; n = 5), and a perception that peer-reviewed journal publication yielded more interest (16 %; n = 5). Seventy-eight percent (n = 25) did not register their review and 22 % (n = 7) did not prepare a protocol before starting the systematic review. The most common reasons for not registering their systematic review were lack of knowledge about systematic review registers (31 %; n = 10), they did not think of it (19 %; n = 6), and not interested due to time required (13 %; n = 4). Respondents who published their systematic review in a non-Cochrane journal used the following Cochrane resources: Cochrane Library (50 %; n = 16), Cochrane Handbook for Systematic Reviewers of Interventions (50 %; n = 16), and a trials register (13 %; n = 4).

Table 2 Responses of authors who published in non-Cochrane journal (n = 32)

Overall, the majority (92 %; n = 70) of respondents indicated they would participate as an author in another systematic review. Overall, the majority (72 %; n = 52) indicated they would participate with Cochrane, with 20 % (n = 15) indicating outside of Cochrane. Among those not interested in participating with Cochrane (n = 15), the majority indicated that they were not interested due to the administrative processes (53 %; n = 8) or the time required to follow Cochrane processes (47 %; n = 7).

Themes that emerged from the open-ended responses, with supporting statements, are provided in Table 3. The following were some of the positive themes identified regarding The Cochrane Collaboration: Cochrane is recognized as producing high-quality, methodologically rigorous systematic reviews; Cochrane is considered an excellent organization with which to work; the support in producing systematic reviews offered through Cochrane is highly valued; and the resources offered through Cochrane are highly valued (e.g., RevMan, Handbook). The following were some criticisms of Cochrane processes for review production: application of systematic review standards varies by Cochrane Review Group; producing a Cochrane review is lengthy; some of the Cochrane requirements (e.g., search completed within 12 months) conflict with process (e.g., time for editorial review); time requirements often prohibit trainees (e.g., graduate students) from working with Cochrane; the increasing methodological requirements (e.g., GRADE) are adding complexity for authors, require additional training, and increase production time. Related to the last theme was an impression that the increasing methodological requirements may have an adverse impact on the readability and utility of the systematic reviews. Finally, the following themes help explain authors’ choice to seek publication outside of Cochrane: publication in non-Cochrane sources may reach a wider audience, or specific audience of interest (e.g., clinical specialty), and the scope of Cochrane is restrictive (e.g., only RCTs, clinical topics of interest, quantitative focus).

Table 3 Themes and supporting statements from open-ended questions

Suggestions offered by respondents to increase involvement and production of systematic reviews with Cochrane are summarized in Table 4. These include: streamlining processes; continuing to offer training and support; emphasizing the high-quality systematic reviews offered through Cochrane; increasing awareness about the high-impact factor of the CDSR; and addressing concerns of Cochrane being overly restrictive and quantitatively focused.

Table 4 Suggestions offered by respondents to increase involvement and production of systematic reviews with the Cochrane Collaboration


Our results provide some insights into authors’ experiences preparing and publishing systematic reviews, as well as factors that influence choice of publication arena, specifically Cochrane and non-Cochrane sources. Key reasons for publishing in Cochrane are its positive reputation and impact factor. Reasons for publishing in non-Cochrane sources include lack of familiarity or challenges with the Cochrane processes, and desire to publish in a source more directly relevant to the topic of interest. End users looking for evidence in the form of systematic reviews need to be aware that there are a vast number of systematic reviews published across the medical literature. The value of publishing systematic reviews in a single source (e.g., Cochrane) is that they are readily identifiable by end users. Efforts to optimize the identification of systematic reviews in non-Cochrane sources (e.g., through effective labeling or protocol/review registration) and their content will help end users find the necessary synthesized evidence to support clinical practice.

Cochrane was established over 20 years ago and its impact on the production of systematic reviews and development of systematic review methods is highly evident in the results of this survey. Even authors who did not conduct and publish their systematic reviews with Cochrane used Cochrane resources. The Cochrane Handbook for Systematic Reviews of Interventions is published and available online and widely used [2]. Our survey found that authors value the support offered through Cochrane in producing reviews, and the technical resources and guidance available.

Previous research comparing Cochrane and non-Cochrane reviews has found that Cochrane reviews are of superior methodological quality [10]. Cochrane processes (e.g., title registration, peer-review, and publication of protocols) were intended to increase transparency, reduce duplication, and optimize methodological rigor; however, there is little empiric evidence of the impact of the contributions of specific processes on the quality and uptake of Cochrane reviews. Moreover, the process of peer-review can be lengthy, resource intensive, and there is little evidence of its effectiveness in ensuring the quality of published research [13].

While considered more methodologically rigorous, it is unknown whether Cochrane reviews are perceived to be more accessible and usable, and whether there is a difference in terms of uptake and use of Cochrane vs. non-Cochrane reviews in clinical practice. Furthermore, respondents indicated that they published in non-Cochrane sources to reach a different target audience, in particular, members of their clinical specialty. Further research about publication venue for systematic reviews and where end users look for reviews may help ensure the Cochrane Library (and CDSR) is recognized as a “go-to” source for the most relevant and high-quality evidence.

Almost a quarter of non-Cochrane authors indicated that they did not prepare a protocol prior to conducting their review; this is consistent with earlier reports [10]. To our knowledge, there is no evidence that has linked lower methodological quality of systematic reviews without an a priori protocol (i.e., detailed methods beyond simply registering in a systematic reviews register such as PROSPERO). While some non-Cochrane respondents identified the administrative processes as a deterrent to publishing in the CDSR, there is currently a lack of evidence demonstrating that Cochrane reviews actually take longer to produce.

Our findings suggest a tension resulting from the motivation to produce high-quality systematic reviews and the administrative processes, perceived time, and methodological standards required to achieve this goal. Respondents provided some suggestions to address this challenge. Current investigations and interest in technologies and methods to increase efficiencies may help address some of the identified challenges. There is a growing interest as well as work being done within Cochrane to investigate technological approaches to streamlining review production, such as text mining and machine learning [1417]. Covidence, a new software package for review production that links with RevMan and GRADEPro, has also been developed to manage all stages of review production and is revised regularly based on end user input [18]. The EMBASE project has investigated the use of crowdsourcing to increase efficiencies in screening records for CENTRAL [1922]; similar approaches could be investigated within the production of reviews.

The timing of this study coincides with a widespread internal review of the structure and function of Cochrane’s component groups, including the review groups, methods groups, fields, and the consumer network. One of the key motivations for this internal review is to ensure that Cochrane continues to meet its mandate of producing high-quality, relevant, accessible evidence to inform healthcare decision-making. Our results may provide useful information as Cochrane continues to achieve its mandate [23].

This study is limited by the response rate which may affect generalizability; further, the invitation sent to participants had the name and title of one of the Directors of Cochrane Child Health which could have influenced response rate, who responded, and the nature of responses. A further concern is that some of the questions may not have been phrased in a balanced way which could have affected the responses we received, i.e., they may be perceived to be stated from a Cochrane perspective. For example, Cochrane authors were asked why they published with Cochrane, while non-Cochrane authors were not asked the equivalent question, i.e., they were asked why they did not publish in Cochrane, not why they published in a given journal. However, there was a similar number of respondents who had published in Cochrane and non-Cochrane sources, and thematic analyses indicated varied views with both positive and negative opinions. It is important to interpret the results in light of our desire to have a single “go-to” source for high-quality evidence in child health, or at a minimum that reviews of relevance are readily identifiable.

A further limitation is that the sample was derived from child-relevant systematic reviews published in a single year (2013) which may also limit generalizability. Opinions may vary for authors who were updating their reviews compared with authors who were conducting new reviews; however, we did not have adequate information to assess or comment on these differences. Finally, opinions and motives may vary by clinical topic (even within child health) and our selection of non-Cochrane reviews may not have matched the same topics covered in our sample of Cochrane reviews.


We identified reasons that authors publish systematic reviews in Cochrane and non-Cochrane sources. These include the reputation of Cochrane, familiarity or challenges with Cochrane processes, and relevance of the publication venue to the topic of interest. The identification of Cochrane systematic reviews is easier because they reside in the same place (i.e., CDSR housed in The Cochrane Library). Efforts to optimize the identification of systematic reviews in non-Cochrane sources (e.g., through effective labeling or protocol/review registration) and their content will help end users find the necessary synthesized evidence to support clinical practice.


CDSR Cochrane Database of Systematic Reviews, REDCap research electronic data capture, SRs systematic reviews


  1. 1.

    Cochrane Community. []. Accessed 7 Jun 2016.

  2. 2.

    Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. London: The Cochrane Collaboration; 2011.

  3. 3.

    Proposing and registering new reviews []. Accessed 7 Jun 2016.

  4. 4.

    Cochrane Editorial Unit. []. Accessed 7 Jun 2016.

  5. 5.

    Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane Intervention Reviews. London: Cochrane; 2013.

  6. 6.

    PRISMA: Transparent reporting of systematic reviews and meta-analyses. []. Accessed 7 Jun 2016.

  7. 7.

    PRISMA. PRISMA Endorsers.

  8. 8.

    Cochrane. Cochrane Community (beta). 3.4 Fields.

  9. 9.

    Cochrane Community (beta): Core functions of fields []. Accessed 7 Jun 2016.

  10. 10.

    Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78.

  11. 11.

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

  12. 12.

    Citing REDCap []. Accessed 7 Jun 2016.

  13. 13.

    Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev. 2007(2):Mr000016.

  14. 14.

    Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev. 2014;3:74.

  15. 15.

    Elliott J, Sim I, Thomas J, Owens N, Dooley G, Riis J, Wallace B, Thomas J, Noel-Storr A, Rada G, et al. #CochraneTech: technology and the future of systematic reviews. Cochrane Database Syst Rev. 2014;9:Ed000091.

  16. 16.

    Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc. 2016 23(1):193-201.

  17. 17.

    Felizardo KR, Barbosa EF, Martins RM, Valle PHD, Maldonado JC. Visual text mining: ensuring the presence of relevant studies in systematic literature reviews. Int J Softw Eng Knowl Eng. 2015;25(5):909–28.

  18. 18.

    Elliott J, Arno A. Covidence: Cochrane’s new online tool for review production. In: Cochrane Colloquium. Vienna, Austria; 2015.

  19. 19.

    The EMBASE screening project: six months old and going strong []. Accessed 7 Jun 2016.

  20. 20.

    Noel-Storr A, Dooley G, Glanville J. The EMBASE project: an analysis of highlighted words and phrases. In: 22nd Cochrane Colloquium. Hyderabad, India; 2014.

  21. 21.

    Noel-Storr A, Dooley G, Glanville J, Foxlee R. The EMBASE project 2: crowdsourcing citation screening. In: 23rd Cochrane Colloquium. Vienna; 2015. p. 117.

  22. 22.

    Noel-Storr A, Dooley G, Glanville J, Foxlee R. The EMBASE project 3: the 48-hour citation screening challenge. In: 23rd Cochrane Colloquium. Vienna; 2015. p. 66.

  23. 23.

    Cochrane’s structure and function reviews—a status overview. []. Accessed 7 Jun 2016.

Download references


We thank Andrea Milne for conducting the literature search, Megan Nuspl for assistance in exporting the Cochrane reviews, Catherene Joseph and Tariq AlShawabkeh for assistance with screening, and Robin Featherstone for article retrieval. We thank the following individuals who assisted with pilot testing the survey instrument: Michele Dyson, Joan Robinson, Jennifer Pillay, and Kelly Russell.


This project was funded through the Canadian Institutes of Health Research (CIHR): Knowledge Synthesis and Translation by Cochrane Canada, CON-105529. The first author holds a New Investigator Salary Award from CIHR.

Authors’ contributions

LH, DT, RF, and KW designed the study; reviewed the study materials (e.g., survey instrument); contributed to the analysis and interpretation of the results; and contributed to and critically reviewed the manuscript. LH oversaw all aspects of the study’s implementation. KS prepared the ethics submission; assisted with the survey development and implementation; conducted the descriptive analyses; and assisted with manuscript writing, revisions, and preparation for submission. AW assisted with primary and secondary screening to identify reviews; assisted with administration of the survey; and reviewed the manuscript. All authors read and approved the final manuscript.

Competing interests

LH, DT, RF, and KW have leadership roles in The Cochrane Collaboration. AW is a staff member of Cochrane Child Health.

Ethics approval and consent to participate

This study was approved by the University of Alberta Research Ethics Board 2. The Study ID is Pro00053378. Consent to participate in the study was implied by the overt action of completing the survey.

Author information

Correspondence to Lisa Hartling.

Additional files

Additional file 1:

Appendix A: Search Strategy. Appendix B: Screening criteria for selection of non-Cochrane reviews. Appendix C: Survey Questions. (DOCX 51 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark


  • Systematic reviews
  • Cochrane Collaboration
  • Publication
  • Survey


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.