Skip to main content
Download PDF

The score after 10 years of registration of systematic review protocols

Systematic Reviews volume 11, Article number: 191 (2022) Cite this article

Abstract

Background

With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched 10 years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status.

Methods

PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random-stratified sampling and eligibility screening, data extraction on publication and journal characteristics, and protocol registration status, was performed. Both descriptive and multivariable comparative statistical analyses were performed.

Results

A total of 357 SRs (2020: n = 163; 2021: n = 194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported using PRISMA as a reporting guideline had higher odds of having a protocol than publications that did not report PRISMA (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor had higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe). Of the 33 SRs published in journals that endorse PROSPERO, 45% did not have a protocol. Most SR protocols were registered in PROSPERO (n = 129; 96%).

Conclusions

We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. To guide strategies to increase the uptake of SR protocol registration, further research is needed to gain understanding of the benefits and informativeness of SRs protocols among different stakeholders.

Systematic review registration

osf.io/9kj7r/

Peer Review reports

Introduction

Events following the coronavirus disease 2019 (COVID-19) pandemic have underlined the realization that societies form interconnected networks that are part of a global community. This is a realization that the academic community has embraced for several decades [1]. Collecting and reporting data in a transparent manner, as well as sharing data through collaborative networks, belong to the fundamental principles of open science [2]. Closely related is the open access movement, which promotes free availability of research output. Both open science and open access aim to improve research quality, research data integrity, research data usage efficiency, and societal impact and implementation of academic research [3, 4].

Prospective protocol registration in publicly accessible repositories increases transparency and research quality. A priori protocols help researchers to prepare the research process, to choose appropriate methods, and to specify the research questions and outcomes beforehand [5]. Registration of protocols facilitates thorough checking of review methods and reduces unnecessary duplication of research by independent researchers [6, 7]. Furthermore, risk of bias is minimized, and potential academic misconduct might be prevented [4]. Besides these benefits, prospective protocol registration also serves clinicians, patients, and policymakers. Clinical trial registration (e.g., ClinicalTrials.gov) informs patients and clinicians about novel clinical interventions in the investigation phase [8]. Also, trial registration may help to prevent cover up of non-favorable clinical outcomes, thereby reducing publication bias and increasing patient safety [4, 6].

In the era of evidence-based medicine, a number of organizations, including Cochrane, the Campbell Collaboration, and the Joanna Briggs Institute have served the research and clinical community with high-quality evidence syntheses. These organizations were among the first to publish protocols for systematic reviews (SRs). Although Cochrane reviews constitute a small fraction (7%) of all published SRs [9], reviews from this organization are considered the “gold standard” [10]. A recent study reported that on average 80 systematic reviews were published per day in 2019 [9]. Other research has found that 67% of meta-analyses are overlapping, within one of the topics, a total of 13 overlapping meta-analyses [11]. This underscores the necessity of protocol registration for non-Cochrane reviews, which could potentially reduce research overlap and duplication and thereby research waste and waste of public finance [7, 12]. For this, the Prospective Register of Systematic Reviews (PROSPERO) was launched in 2011, allowing researchers from various disciplines to register protocols for SRs related to health care [10, 13]. Since then, other registries such as the Open Science Framework have become available. Besides registration, SR protocols can be published in peer-reviewed journals. Main advantage of protocol registration in online registries is easy accessibility, while protocol publication could benefit from improvements in methodological quality suggested by reviewers during the protocol peer-review process.

Since the launch of PROSPERO, the number of protocol registrations for systematic reviews is increasing [10, 13,14,15]. Several studies have assessed SR protocol registration [10, 13,14,15,16,17,18,19,20,21]. Some studies assessed protocol records [10, 16] or assessed whether a protocol record resulted in a published SR [13]. Other studies assessed the status of SR protocol registration from the onset of PROSPERO in various research areas (e.g., dentistry, environmental science) or type of review (e.g., diagnostic, intervention, prognostic) [14, 15, 17,18,19,20,21]. Three studies showed an increase in the proportion of SR protocol registration in time [14, 15, 20]. Noteworthy is that in one study, which included both Cochrane and non-Cochrane reviews, most SRs with registered protocols were Cochrane reviews [17]. Here, we investigated the proportion of SRs of intervention studies that have a registered protocol 10 years since the launch of PROSPERO.

Objectives

This study aims to assess the proportion SRs of intervention studies which have a registered or published protocol 10 years since the launch of PROSPERO. In addition, to gain more insight into factors associated with having a SR protocol, we explored the association of SR characteristics with protocol registration or publication status.

Methods

The protocol of this study was preregistered on the Open Science Framework (OSF) on September 17, 2021: osf.io/9kj7r/ [22]. At the time of preregistration, we had carried out our literature search and piloted our selection process, but had not commenced data extraction and/or analysis. Reporting of this manuscript was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) [23, 24].

Information sources

PubMed was searched for systematic reviews of interventions using the search filter by health-evidence Canada [25]. Details on the search strategy are provided in Supplementary file 1, Table 1. The search was last carried out on July 2, 2021. From the records identified by the search strategy, two random samples of 10% were drawn using simple random-stratified sampling in EPPI-Reviewer [26].

Table 1 Characteristics of included articles
Full size table

Eligibility criteria

Systematic reviews of human intervention studies were eligible for inclusion in this study if “systematic review” or “meta-analysis” was stated in the title or abstract or if it was clear that it was a systematic review, such as a Cochrane review. The global COVID-19 pandemic increased time pressure on researchers to publish COVID-related research as quickly as possible. As a consequence, researchers could have omitted protocol registration in order to decrease SR processing time. To avoid selection bias of protocol registration due to the global COVID-19 pandemic, we included records published in January 2020 or January 2021. To allow for correction of bias, we have also extracted whether included SRs were COVID related. Either e-pub or journal publication date was used for selection, which did not match the indexed publication date in PubMed for some of the studies [27]. We had no restriction on language. Non-interventional reviews, literature reviews, updates of reviews, and umbrella reviews were excluded.

Study selection and data collection

The selection process and data collection were performed consecutively by a single author (either KvdB, MG, or CO). First, title and abstract were screened. When potentially eligible, full text was obtained and further screened for inclusion. When all eligibility criteria were met, data extraction was performed by the same author. Difficulties in study selection or data extraction were discussed by the authors until agreement was reached. Study selection was randomly (approximately 10% of studies) checked by KvdB. Extracted data was checked by an independent second researcher (KvdB, CO, and DI).

Data were recorded in a piloted data extraction sheet in Microsoft Excel (version 16.54, 2021). We manually extracted data on the characteristics of the SR publications (publication year, number of authors, number of affiliations, funding, conflicts of interest statement, self-reported use of PRISMA guidelines, and COVID relatedness). Additionally, the following details regarding protocol registration or publication were extracted: database of protocol registration and wherein the article the protocol was mentioned (abstract, methods, or other location). We assessed whether an identifier or registration number of the protocol was reported and whether this number was correct, as this is crucial for protocol retrieval. The registration number was considered correct if it led to a corresponding protocol. We identified journals that endorsed PROSPERO (i.e., journals that specifically stated their support on the website of PROSPERO) [28] and assessed the number of SRs that were published in these journals. Individual journal author guidelines were not checked for PROSPERO endorsement.

From Web of Science and the Journal Citation Reports, data were automatically retrieved for the following: topic of the publication (Web of Science categories), journal, journal impact factor, and journal of published protocol (if applicable). As journals without indexation in the Science Citation Index Expanded (SCIE) or the Social Science Citation Index (SSCI) have no impact factor, it was considered to be equal to zero.

Data synthesis and analysis

The primary outcome for this study was the proportion of SR publications with a protocol registration or publication. Characteristics of the reviews were quantified using descriptive statistics. Normality of the data was checked through visual inspection by two authors (KvdB, PH). We used frequencies and percentages for categorical variables. Mean and standard deviation were used for normally distributed continuous variables and median and interquartile range if data were skewed.

For our secondary objective, variables included in the multivariable logistic regression to explore the association with protocol registration or publication in non-Cochrane SRs were informed by previous research within our team [29, 30]. We assumed that protocol registration or publication was more likely in SRs with a higher number of authors and affiliations, a higher journal impact factor, a COI statement reported, and reported to using the PRISMA guidelines. We also assumed that there would be differences in protocol registration according to different types of funding, compared to no funding and different continents, for which we choose the Europe as a reference category. We included year of publication (reference category = 2020) in the multivariable analysis to control for potential differences between these years. All statistical analyses were conducted in RStudio [31], and figures were created using the ggplot2 package [32].

Results

A total of 6669 records were identified in our literature search (Fig. 1). A random sample of 1267 records was screened and assessed for eligibility. Reasons for exclusion were publication date (i.e., in months other than January), non-interventional studies (diagnostic or prognostic studies), or review types other than systematic review (e.g., updated review, scoping review) or nonhuman studies.

Fig. 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analysis flow chart. *Indexed publication date in January 2020 or January 2021. ^Extracted publication date (either e-pub or journal publication date) in January 2020 or January 2021

Full size image

In total, 357 publications were included for analysis (n = 163 for January 2020 and n = 194 for January 2021). A reference list of included studies is provided in the Supplementary file 1. Characteristics of the included SRs are portrayed in Table 1. Overall, 96% of SRs reported a COI statement, 76% reported to use PRISMA guidelines, and most SRs were either not funded (n = 134, 38%) or had nonprofit funding (n = 116, 32%). A quarter of SRs did not report any information on funding (n = 92, 26%). The origin of the corresponding authors varied over 44 different countries and were mostly from Asia (n = 127, 36%), Europe (n = 99, 28%), and North America (n = 89, 25%) (Fig. 2 and Supplementary file 2, Fig. 1). The included SR publications also showed great variety in topics (n = 63) and journals in which the article was published (n = 280) (Fig. 3 and Supplementary file 2, Table 2). We identified 33 (9% of total sample) journals in our sample that explicitly endorses PROSPERO. For 17 publications, no topic category could be assigned. The three most frequent topics were surgery (n = 39), pharmacology and pharmacy and medicine (n = 31), and general and internal (n = 28).

Fig. 2
figure 2

Continents of origin of the corresponding authors of all included SRs. The figure represents the total sample of SRs (N = 357)

Full size image
Fig. 3
figure 3

Topics of all included SRs. Topics are classified according to Web of Science (WoS) categories. No category was assigned to 17 included SRs that were not indexed to a WoS category. The figure represents the total sample of SRs (N = 357). As each SR could have been indexed to multiple topics, the total number exceeds the number of SRs

Full size image
Table 2 Characteristics of SRs with protocol registration
Full size table

Proportion of SRs with a protocol and differences with SRs without a protocol

Our sample included 6 (1%) Cochrane reviews (Table 1). Of the non-Cochrane reviews, 135 (38%) SRs had a protocol either registered, published, or both. The number of authors, affiliations, source of funding, and COVID-19 relatedness were similar between the SRs with and without a protocol. The percentage of SRs with a protocol was 42% in 2021 compared to 33% in 2020. Between the continents, percentages of SRs with a protocol varied from 25% (Asia) up to 61% (Australia). Several topic categories had ≥50% SR publications with a protocol, such as the following: clinical neurology, psychiatry, nutrition and dietetics, rehabilitation, respiratory system, and public, environmental, and occupational health. As there is a great variety in topics, only those with more than five publications are mentioned.

In SRs with a registered protocol, 86% reported to have applied the PRISMA guidelines. In contrast, in SR publications without protocol registration, the percentage that reported having used the PRISMA guidelines was 72%. The median journal impact factor was 3.8 (Q1–Q3, 2.6–5.4) for SRs with a protocol and 3.2 (Q1–Q3, 2.3–4.3) for those without a protocol. A total of 29 reviews were published in journals that were not indexed in the SCIE or SSCI. Of those reviews, 7 had a protocol, and 22 did not have a protocol. Of the SRs published in PROSPERO-endorsed journals, 45% (15 of 33) did not have a registered or published protocol.

The multivariable logistic regression (Supplementary file 2, Table 3) showed that the following factors were related to protocol registration or publication: self-reported use of PRISMA (reported vs. not reported), journal impact factor, and Asia as country of origin (versus Europe) (McFadden R2 = 0.096, p < 0.001). SR publications that reported using PRISMA guidelines have higher odds of having a protocol than publications that did not report using the PRISMA guidelines (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor have a higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe).

Protocol registration or publication

In Cochrane reviews, protocol registration is reported in a specific section of the publication, in which deviations from the protocol are clarified. In line with Cochrane instructions, in the six Cochrane reviews from our study sample, protocol registration was not reported in the “Abstract” or “Methods” section of the publication, for which reason they are not presented in Table 2.

From the non-Cochrane review publications with a protocol (n = 135), 96% (n = 129) had protocol registration alone, and 4% (n = 5) had both registered and published their protocol. One SR had a published protocol and was excluded from Table 2, as it had no registration characteristics. Protocols were published in four different journals, and one protocol was posted on the National Institute for Health and Care Excellence (NICE) website (Supplementary file 2, Table 4).

Most SR publications reported protocol registration either in the “Methods” section (74%) or both in the abstract and methods (22%) (Table 2). Two SR publications reported information on protocol registration solely in the abstract (2%). Other sections in which protocol information was reported were as follows: at the end of the article (n = 4), on the title page (n = 4), the introduction (n = 1), or in the supplementary information (n = 1).

In our study, PROSPERO was predominantly used for protocol registration (n = 129; 96%). Other databases that were used are OSF (n = 3; 2%), the International Platform of Registered Systematic Review and Meta-Analysis Protocols (INPLASY) (n = 1; 1%), and the Research Registry (n = 1; 1%). As a correct identifier or registration number is key for protocol retrieval, we assessed the number of publications with a correct registration number. Almost all publications reported a registration number (n = 128; 96%), and most of those registration numbers were correct (n = 120; 94%). Incorrect registration numbers were either incomplete or the corresponding protocol could not be retrieved with the registration number reported. Of all registered SRs, 10% (n = 14) provided a direct URL to the protocol.

Discussion

We found that 38% of recently published SRs of interventions reported a registered or published protocol, and 1% of our sample was Cochrane reviews. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR, a more frequent reporting of use of PRISMA guidelines and continent of origin.

Other studies have shown proportions of SRs with protocols ranging from 15 to 33% [14, 15, 17,18,19,20,21]. The proportion of 38% of SRs with a protocol found in our study aligns with the increasing trend over time [14, 15, 20]. Still, there seems to be a lot of variation in protocol registration proportion between studies. A study of environmental reviews observed 17% registered protocols in 2021 [18], while in the field of dentistry and orthodontics, 33–37% registered protocols were observed in 2017 [14, 19]. The differences in observed proportions of protocol registration between studies might not only be explained by the earlier publication date of study but also by the type of reviews (e.g., prognostic, diagnostic, intervention), the research topic, choice of examined registry, or a combination hereof.

Protocol registration seems to be positively linked with a higher methodological quality and improved reporting quality in systematic reviews [13, 17, 19]. In our study, protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines, which can be an indication of higher research quality. In journals with a higher impact factor (> 10), the risk of bias was found to be consistently lower with higher levels of registration of randomized controlled trials and the use of the CONSORT statement [30]. In contrast with Ge et al. (2018), we did not find an association between financial sponsorship and protocol registration. In line with Rombey et al. (2020), we observed that SRs from USA and China had low proportions, and that SRs from Canada and Australia had high proportions of SRs with registered protocols [15]. It is striking that, in our sample, Asia had the highest number of SRs published and the lowest proportion of registered protocols (25%).

In contrast to Allers et al. (2018), we observed that the number of SRs that reported the use of PRISMA was high for both SRs with a protocol (87%) and without a protocol (71%) [13]. A possible explanation is that the self-reported use of PRISMA reporting guidelines requires a smaller effort from review authors than protocol registration as reporting can be easily adjusted in hindsight, unlike protocol registration. Therefore, self-reported PRISMA use might have increased sharply in the last few years, also among authors that did not register a protocol. Hence, reporting guidelines can help to raise awareness for protocol registration among review authors, but reporting guidelines are insufficient to effectuate a priori protocol registration. This is supported by a recent survey study that showed that almost 45% of researchers, who have written a SR between 2010 and 2016, have never registered a SR protocol [33]. Most common reasons for not registering were as follows: lack of knowledge on benefits or process of protocol registration, lack of time, and nonmandatory requirement [33].

In 2005, the International Committee of Medical Journal Editors (ICMJE) has introduced mandatory trial registration guidelines for clinical trials on human subjects [34]. No such requirement exists for SRs. Even though some journals and organizations have endorsed protocol registration in PROSPERO, our results show that one-third of SRs published in these journals did not have a protocol. Hence, recommendations in author guidelines are insufficient to effectively implement prospective SR protocol registration. High-quality journals have the responsibility to uphold and continuously increase their quality standards to ensure high publication quality. Only after mandating registration of trials by the ICMJE, other journals followed, and the uptake of trial registration increased. Journal and publishers could fulfill a pioneer role in raising quality standards by making protocol registration a mandatory requirement for publication. As such, they could include assessment of protocol registration during the submission process or introduce a formal check as part of the peer-review process. A select number of journals already require protocol registration for SRs [35, 36]. Protocol registration awareness could be further enhanced by increasing or mandating adherence to the PRISMA for abstract reporting guidelines (reporting protocol registry name and number in the abstract) [37].

Some journals have adopted registered reports, which is similar to the Cochrane registration process, but is not exclusively aimed at SRs. During this process, research articles undergo peer review at the study design or protocol stage [38]. For Cochrane reviews, the scope of the proposed review and skills and experience of the proposed author team have typically been evaluated beforehand [39]. This provides an opportunity for feedback and support in the designing phase of the review to increase SR quality.

In the context of the open science movement, the scientific community and journals need to navigate between recommendation versus obligation of protocol registration to strike a balance between transparency and quality on the one hand and researchers’ time investment on the other. From the perspective of patients and guideline developers, high-quality, non-biased SRs are required for optimal clinical care and patient treatment for the reason that SRs and meta-analyses, more often than single clinical trials, directly inform medical guidelines and thereby medical treatment decisions.

Together, our results stress the need for further efforts to communicate the value of SR protocol registration more effectively, not only among researchers but also among journal editors and peer reviewers. To guide further efforts to highlight the value of SR protocol registration, research is needed to gain understanding of the benefits and informativeness of SRs protocol registration among different stakeholders (researchers, journal editors, peer reviewers, policy makers, the public, clinicians, patients).

Strengths and limitations

A strength of our study is that we have assessed protocol registration from a sample of recently published SRs. However, a potential limitation is that some publications might not have reported protocol registration in their manuscript. Allers et al. found that 12.5% of SRs with a protocol did not report protocol registration or publication. We have not assessed whether this is the case in our sample and whether the proportion of interventional SRs with protocol registration we observed is an underestimation. When SR publications do not refer to the protocol, some of the benefits of protocol registration may be lost.

Another strength of our study is that it is unlikely that our results were influenced by COVID-related reviews. Not only have we selected our search periods to reflect both non-COVID and COVID pandemic times, but eventually, our sample had only a small proportion (2%) of COVID-related SRs.

Lastly, the generalizability of our study is limited, as we only included SRs of healthcare interventions. Protocol registration status might be different for other types of reviews (e.g., diagnostic, prognostic) and remains to be assessed in future research.

Conclusions

We found that 135 out of 357 (38%) recently published interventional SRs had a protocol either registered or published. Our results did show that a higher journal impact factor and more frequent self-reported use of PRISMA guidelines was positively associated with protocol registration or publication. Noteworthy is that the continent with the highest number of published SRs (Asia) has the lowest proportion (25%) of SRs with protocol registration.

Availability of data and materials

The data supporting the conclusions of this article is available in the Open Science Framework (OSF) repository: https://doi.org/10.17605/OSF.IO/TSUF9.

Abbreviations

COI:

Conflict of interest

INPLASY:

International Platform of Registered Systematic Review and Meta-Analysis Protocols

JIF:

Journal impact factor

NICE:

National Institute for Health and Care Excellence

OSF:

Open Science Framework

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analysis

PROSPERO:

Prospective Register of Systematic Reviews

SCIE:

Science Citation Index Expanded

SR:

Systematic review

SSCI:

Social Science Citation Index

References

  1. Jamali D, Barkemeyer R, Leigh J, Samara G. Open access, open science, and coronavirus: mega trends with historical proportions. Bus Ethics. 2020.

  2. Vicente-Saez R, Martinez-Fuentes C. Open science now: a systematic literature review for an integrated definition. J of Bus Res. 2018;88:428–36.

    Article  Google Scholar 

  3. Tennant JP, Waldner F, Jacques DC, Masuzzo P, Collister LB, Hartgerink CH. The academic, economic and societal impacts of open access: an evidence-based review. F1000Res. 2016;5:632.

  4. Munafo MR, Hollands GJ, Marteau TM. Open science prevents mindless science. BMJ. 2018;363:k4309.

    Article  Google Scholar 

  5. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

    Article  Google Scholar 

  6. Krleza-Jeric K, Chan AW, Dickersin K, Sim I, Grimshaw J, Gluud C. Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):956–8.

    Article  Google Scholar 

  7. Straus S, Moher D. Registering systematic reviews. CMAJ : Can Med Asso j = j de l'Asso med can. 2010;182(1):13–4.

    Article  Google Scholar 

  8. Zarin DA, Keselman A. Registering a clinical trial in ClinicalTrials.gov. Chest. 2007;131(3):909–12.

    Article  Google Scholar 

  9. Hoffmann F, Allers K, Rombey T, Helbach J, Hoffmann A, Mathes T, et al. Nearly 80 systematic reviews were published each day: observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin Epidemiol. 2021;138:1–11.

    Article  Google Scholar 

  10. Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev. 2018;7(1):32.

    Article  Google Scholar 

  11. Siontis KC, Hernandez-Boussard T, Ioannidis JPA. Overlapping meta-analyses on the same topic: survey of published studies. BMJ. Br Med J. 2013;347.

  12. Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:7.

    Article  Google Scholar 

  13. Allers K, Hoffmann F, Mathes T, Pieper D. Systematic reviews with published protocols compared to those without: more effort, older search. J Clin Epidemiol. 2018;95:102–10.

    Article  Google Scholar 

  14. Sideri S, Papageorgiou SN, Eliades T. Are orthodontic systematic reviews registered a priori in PROSPERO? J Orthod. 2017;44(4):249–55.

    Article  Google Scholar 

  15. Rombey T, Doni K, Hoffmann F, Pieper D, Allers K. More systematic reviews were registered in PROSPERO each year, but few records’ status was up-to-date. J Clin Epidemiol. 2020;117:60–7.

    Article  Google Scholar 

  16. Booth A, Mitchell AS, Mott A, James S, Cockayne S, Gascoyne S, et al. An assessment of the extent to which the contents of PROSPERO records meet the systematic review protocol reporting items in PRISMA-P. F1000Res. 2020;9:773.

  17. Ge L, Tian JH, Li YN, Pan JX, Li G, Wei D, et al. Association between prospective registration and overall reporting and methodological quality of systematic reviews: a meta-epidemiological study. J Clin Epidemiol. 2018;93:45–55.

    Article  Google Scholar 

  18. The prevalence of recommended research practices in contemporary environmental health systematic reviews [Internet]. 2022 [cited 01-02-2020]. Available from: https://osf.io/j2d5v/. https://doi.org/10.17605/OSF.IO/J2D5V.

  19. Dos Santos MBF, Agostini BA, Bassani R, Pereira GKR, Sarkis-Onofre R. Protocol registration improves reporting quality of systematic reviews in dentistry. BMC Med Res Methodol. 2020;20(1):57.

    Article  Google Scholar 

  20. Tsujimoto Y, Tsujimoto H, Kataoka Y, Kimachi M, Shimizu S, Ikenoue T, et al. Majority of systematic reviews published in high-impact journals neglected to register the protocols: a meta-epidemiological study. J Clin Epidemiol. 2017;84:54–60.

    Article  Google Scholar 

  21. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.

    Article  Google Scholar 

  22. Assessing the proportion of prospectively available systematic review protocols and the databases or repositories used by authors for registering or posting their protocol [Internet]. 2021. Available from: https://osf.io/9kj7r/. https://doi.org/10.17605/OSF.IO/TSUF9.

  23. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. J Clin Epidemiol. 2021;134:178–89.

    Article  Google Scholar 

  24. Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139–42.

    Article  Google Scholar 

  25. Lee E, Dobbins M, DeCorby K, McRae L, Tirilis D, Husson H. An optimal search filter for retrieving systematic reviews and meta-analyses. BMC Med Res Methodol. 2012;12.

  26. Thomas J, Graziosi S, Brunton J, Ghouze Z, O'Driscoll P, Bond M. EPPI-Reviewer: advanced software for systematic reviews, maps and evidence synthesis. London: UCL Soc Res Ins: EPPI-Centre Software; 2020.

    Google Scholar 

  27. Stefanie Haustein TDB, Rodrigo Costas. When is an article actually published? An analysis of online availability, publication, and indexation dates. 2015.

  28. Support for PROSPERO [Available from: https://www.crd.york.ac.uk/PROSPERO/#aboutpage.

  29. Predicting questionable research practices in randomized clinical trials [Internet]. 2018. Available from: osf.io/27f53.

  30. Vinkers CH, Lamberink HJ, Tijdink JK, Heus P, Bouter L, Glasziou P, et al. The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement. PLoS Biol. 2021;19(4):e3001162.

    Article  CAS  Google Scholar 

  31. R Core Team. R. A language and environment for statistical computing. Vienna: R Found for Sta Comp; 2021.

    Google Scholar 

  32. Wickham H. ggplot2: elegant graphics for data analysis. New York: Springer-Verlag; 2016.

    Book  Google Scholar 

  33. Tawfik GM, Giang HTN, Ghozy S, Altibi AM, Kandil H, Le HH, et al. Protocol registration issues of systematic review and meta-analysis studies: a survey of global researchers. BMC Med Res Methodol. 2020;20(1):213.

    Article  Google Scholar 

  34. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Croat Med J. 2004;45(5):531–2.

    PubMed  Google Scholar 

  35. Goldberg J, Boyce LM, Soudant C, Godwin K. Assessing journal author guidelines for systematic reviews and meta-analyses: findings from an institutional sample. J Med Libr Assoc. 2022;110(1):63–71.

    Article  Google Scholar 

  36. Nagendrababu V, Duncan HF, Dummer PMH. International Endodontic Journal policy on mandatory prospective (a priori) protocol registration for clinical trials and systematic reviews. Int Endod J. 2021;54(10):1685–6.

    Article  Google Scholar 

  37. Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, et al. PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10(4):e1001419.

    Article  Google Scholar 

  38. Nosek BA, Beck ED, Campbell L, Flake JK, Hardwicke TE, Mellor DT, et al. Preregistration is hard, and worthwhile. Trends Cogn Sci. 2019;23(10):815–8.

    Article  Google Scholar 

  39. CJ CM. Chapter II: Planning a Cochrane Review. In: JPT H, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane; 2021.

    Google Scholar 

Download references

Acknowledgements

We would like to thank Demy Idema for her contribution to this research by checking part of the data extraction.

Funding

No funding was received for this study.

Author information

Authors and Affiliations

  1. Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands

    Kim van der Braak, Mona Ghannad, Claudia Orelio, Pauline Heus, Johanna A. A. Damen, René Spijker & Lotty Hooft

  2. Research Support, Diakonessenhuis Utrecht, Bosboomstraat 1, 3582 KE, Utrecht, The Netherlands

    Claudia Orelio

  3. Medical Library, Amsterdam Public Health, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands

    René Spijker

  4. Division of General Internal Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA

    Karen Robinson

  5. Section for Evidence-Based Practice, Department of Health and Functioning, Western Norway University of Applied Sciences, Bergen, Norway

    Hans Lund

Authors
  1. Kim van der Braak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mona Ghannad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claudia Orelio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pauline Heus
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Johanna A. A. Damen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. René Spijker
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Karen Robinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hans Lund
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Lotty Hooft
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Planning and designing, protocol development, and registration, KB, MG, PH, JD, RS, KR, HL, and LH. Feedback and polishing manuscript, KB, CO, PH, JD, RS, KR, HL, and LH. Screening and data extraction, KB, MG, CO, PH, JD, and HL. Statistical analysis, KB and JD. Writing manuscript and revisions, KB and CO. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Kim van der Braak.

Ethics declarations

Ethics approval and consent to participate

Not applicable as this study only includes already published systematic reviews.

Consent for publication

All authors were able to provide feedback on the manuscript and consent to publication of this version of the manuscript.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary Table 1.

Search string

Additional file 2: Supplementary Figure 1.

Countries of origen of the corresponding author. Supplementary Table 2. Frequency of included publications per journal title. Supplementary Table 3. outcomes of logistic regression. Supplementary Table 4. Frequency of SR protocols published in journals/ databases other than protocol registries.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van der Braak, K., Ghannad, M., Orelio, C. et al. The score after 10 years of registration of systematic review protocols. Syst Rev 11, 191 (2022). https://doi.org/10.1186/s13643-022-02053-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-022-02053-9

Keywords

Systematic Reviews

ISSN: 2046-4053