Does haste make waste? Prevalence and types of errors reported after publication of studies of COVID-19 therapeutics
Systematic Reviews volume 12, Article number: 216 (2023)
The COVID-19 pandemic spurred publication of a rapid proliferation of studies on potential therapeutic agents. While important for the advancement of clinical care, pressure to collect, analyze, and report data in an expedited manner could potentially increase the rate of important errors, some of which would be captured in published errata. We hypothesized that COVID-19 therapeutic studies published in the early years of the pandemic would be associated with a high rate of published errata and that, within these errata, there would be a high prevalence of serious errors.
We performed a review of published errata associated with empirical studies of COVID-19 treatments. Errata were identified via a MEDLINE and Embase search spanning January 2020 through September 2022. Errors located within each published erratum were characterized by location within publication, error type, and error seriousness.
Of 47 studies on COVID-19 treatments with published errata, 18 met inclusion criteria. Median time from publication of the original article to publication of the associated erratum was 76 days (range, 12–511 days). A majority of errata addressed issues with author attribution or conflict of interest disclosures (39.5%) or numerical results (25.6%). Only one erratum contained a serious error: a typographical error which could have misled readers into believing that the treatment in question had serious adverse effects when in fact it did not.
Despite accelerated publication times, we found among studies of COVID-19 treatments the majority of errata (17/18) reported minor errors that did not lead to misinterpretation of the study results. Retractions, an indicator of scientific misdirection even more concerning than errata, were beyond the scope of this review.
Publication of biomedical research in scientific journals is the primary means of disseminating scientific findings to the research community and the public . The goal is to accurately describe and interpret research findings, but mistakes are inevitable. Errors range from simple typographical glitches to more serious coding, arithmetic, or statistical errors that distort the study findings and lead to erroneous conclusions. Before publication, biomedical research reporting errors can be intercepted by authors, reviewers, or editors. Afterwards, the onus is on authors, journal editorial staff, and astute readers; noteworthy errors detected in this way are reported in the form of published errata, corrections, or corrigendum. While few studies have evaluated the incidence of such correctives, one report limited to general internal medicine and cardiology indicates they may be as frequent as 4.2 per 100 published articles .
In considering factors that may contribute to errors in the scientific record, one culprit may be pressure to communicate information rapidly, as might be seen when two competing research groups are rushing to claim a scientific discovery  or when external circumstances demand rapid reporting to support critical public health needs. The first 2 years of the COVID-19 pandemic created significant demand for research on therapies meant to inhibit the virus or mitigate the severity of clinical illness. Given the desperate need to understand the pathogenesis of COVID-19 and identify effective treatments, publications underwent rapid editorial review, with one international study reporting a nearly tenfold reduction in time from submission to acceptance for COVID-related compared with non-COVID related articles (11.3 versus 106 days) . This accelerated schedule may have decreased the rigor of peer review.
Although several studies have evaluated retractions of COVID-19 publications [5,6,7,8], few have examined errors identified in published errata. We sought to characterize published errata related to COVID-19 treatment research and determine if compressed review times reported during the pandemic were associated with a higher prevalence and severity of error. We expected to find both a high rate of published errata and a high proportion of serious errors within published errata.
We conducted a systematic review of errata published between January 1, 2020, and September 1, 2022, that were associated with articles reporting empirical studies of COVID-19 treatments. Errata published in English and related to COVID-19 therapeutics were identified on MEDLINE and Embase using the search terms COVID-19, erratum, corrigendum, and correction (the complete list of search terms, prepared with the assistance of a reference librarian, is provided in Additional file 1). Article-erratum pairs were included if the article’s topic was an empiric evaluation of COVID-19 treatment including head-to-head comparisons of treatment modalities, treatment versus placebo, or treatment versus standard of care or supportive care. Study types included clinical trials, observational studies, and systematic reviews. Articles were excluded if they investigated the treatment of another disease process during the COVID-19 pandemic, COVID-19 diagnostics, epidemiologic studies of the prevalence of treatment, or were editorial or opinion pieces on COVID-19 that did not present an empirical evaluation of a treatment.
Errata were categorized by location of the error in the original publication (abstract, body of the article, tables or figures, references, or supplemental material) and error type (author disclosure or conflict of interest, author attribution, numerical or statistical error, textual error, or citation error). Errors were then categorized as serious or minor. Errors were defined as serious if they (1) affected the study results in a way that could lead to erroneous conclusions by the author(s) or readers or (2) were found in the abstract or title, increasing the risk of error propagation. Two independent reviewers screened articles for inclusion and categorized errors identified in the errata. When discrepancies between reviewers arose (n = 3), the articles were jointly re-reviewed; all three were excluded by consensus. Data was managed in a REDCap  database and descriptive statistics were used to analyze data.
The initial search produced 171 results; however, on review, 124 were removed because they (1) lacked an associated erratum (n = 112), (2) represented duplicate entries (n = 8), or (3) consisted of abstracts without full articles (n = 4) (Fig. 1). Of the remaining 47 article-erratum pairs, 18 articles (19 associated errata) met inclusion criteria (1 article had 2 separate published errata). Fifteen of the 18 studies were randomized clinical trials [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24], two were systematic reviews [25, 26], and one was an observational study .
Among the 19 errata, 55% addressed 1 error within the published article, 28% 2 to 4 errors, and 17% ≥ 5 errors. Higher impact journals appeared to be somewhat over-represented (Table 1). Two thirds had first authors based outside of Europe or North America. Errors were most often found in the abstract, title, or byline and frequently involved author misattribution (16.3%) or failure to disclose potential conflicts of interest (23.3%) (Table 2). Median time from publication of the original article to publication of the associated erratum was 76 days (range 12 to 511 days). Among the errors identified, only one was characterized as serious (Table 3). This serious error was a typographical error in the Results section of the abstract. The sentence read, “There was severe adverse event recorded in the study group”; however, the correct statement was “There was no severe adverse event recorded in the study group.” This error would have led readers to question the safety of the study treatment, if the reader did not further investigate the body of the article which contained the correct results for the treatment group.
In this study of published errata associated with articles on COVID-19 therapeutics, most reported errors were minor. Consistent with other studies, about half of detected errors were corrected in the original article’s online publication . Many of these errors were related to author misattribution or failure to disclose conflict of interest. Others involved numerical errors that were primarily located in tables and figures. One serious error among 19 errata yields a serious error rate of 5%, which may lend cause for concern. This error, located in the abstract, misrepresented that the treatment group had serious adverse events, when in fact there were none. While we concede that therapeutic decisions are infrequently made on the basis of a single scientific study, a clinician quickly reviewing the abstract could have been dissuaded from pursuing what was found to be an effective COVID-19 therapeutic due to concerns of misreported adverse events with the treatment.
There have been several prior publications exploring reported error rates and seriousness of those errors. These studies have largely examined specific journals and included articles of all topics/types (Table 4). We observed a lower rate of serious errors (5%) than a study that reviewed randomized control trials from four high-impact journals (10%) . When reviews expanded to include all study types, serious errors ranged from 14 to 25% of published errata [2, 29, 30]. One review of five radiology imaging journals found a very low overall errata rate (< 2%), with 6% of those errors found to be serious . Another review dedicated to errors in authorship points out that each author is expected to proofread the manuscript prior to publication and thus errors related to name misspelling should be easily identified and corrected prior to publication . That we found several errata related to misattribution of authorship—which ought to have been uncovered and corrected during the pre-publication process—raises the question as to whether other, less obvious errors were not identified. Additionally, this finding raises questions about the diligence of co-authors during the final pre-submission period.
The process of scientific publication is supposed to be self-correcting. Our findings suggest that despite the additional pressure to rapidly disseminate research on COVID-19 treatment, there is not a high rate of errors that would change interpretation of study results or conclusions. However, we are only able to capture errors that are published in errata; other errors may remain. Thus, our results might underestimate the true number of errors in publications related to COVID-19 therapeutics. In addition, article retractions, representing another major category of error (or malfeasance), were beyond the scope of this review. As reported by Peterson et al., COVID-19 articles have disproportionately retracted over the time span of the pandemic .
The COVID-19 pandemic was a sudden, disruptive force to the scientific publication process. We hypothesized that the unprecedented volume of research on a novel disease, the urgent need to disseminate findings, and necessity for accurate and high-quality data would have stress-tested the editorial and review processes leading to a noticeable increase in serious errors. Reassuringly, our findings align with error rates reported among studies examining targeted journals which included all study topics. This finding suggests that journals have been able to maintain publication standards for COVID-19 therapeutics during the pandemic. Increased transparency of changes to a journal’s editorial evaluation of COVID-19 research would be helpful in understanding the resources needed and potential for burnout among editors and reviewers. In addition, future research is warranted to evaluate the sustainability of the recent changes to the editorial and review processes and determine whether new approaches to the publication process could translate to non-COVID-19 research.
Wilkes MS. Medical researchers and the media: attitudes toward public dissemination of research. JAMA. 1992;268(8):999. https://doi.org/10.1001/jama.1992.03490080073027.
Hauptman PJ, Armbrecht ES, Chibnall JT, Guild C, Timm JP, Rich MW. Errata in medical publications. Am J Med. 2014;127(8):779-785.e1. https://doi.org/10.1016/j.amjmed.2014.03.012.
Isaacson W. The Code Breaker: Jennifer Doudna, Gene Editing, and the Future of the Human Race. Simon & Schuster. 2021.
Putman MS, Ruderman EM, Niforatos JD. Publication rate and journal review time of COVID-19–related research. Mayo Clin Proc. 2020;95(10). https://doi.org/10.1016/j.mayocp.2020.08.017.
Boschiero MN, Carvalho TA, Marson FA de L. Retraction in the era of COVID-19 and its influence on evidence-based medicine: is science in jeopardy? Pulmonology. 2021;27(2):97–106. https://doi.org/10.1016/j.pulmoe.2020.10.011.
Frampton G, Woods L, Scott DA. Inconsistent and incomplete retraction of published research: a cross-sectional study on COVID-19 retractions and recommendations to mitigate risks for research, policy and practice. Ochodo E, ed. PLoS One. 2021;16(10):e0258935. https://doi.org/10.1371/journal.pone.0258935.
Soltani P, Patini R. Retracted COVID-19 articles: a side-effect of the hot race to publication. Scientometrics. 2020;125(1):819–22. https://doi.org/10.1007/s11192-020-03661-9.
Shi X, Abritis A, Patel RP, et al. Characteristics of retracted research articles about COVID-19 vs other topics. JAMA Netw Open. 2022;5(10):e2234585. https://doi.org/10.1001/jamanetworkopen.2022.34585.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2). https://doi.org/10.1016/j.jbi.2008.08.010.
Ravichandran R, Mohan SK, Sukumaran SK, et al. An open label randomized clinical trial of Indomethacin for mild and moderate hospitalised COVID-19 patients. Sci Rep. 2022;12(1):6413. https://doi.org/10.1038/s41598-022-10370-1.
Blum VF, Cimerman S, Hunter JR, et al. Nitazoxanide superiority to placebo to treat moderate COVID-19 – a pilot prove of concept randomized double-blind clinical trial. eClinicalMedicine. 2021;37:100981. https://doi.org/10.1016/j.eclinm.2021.100981.
Liu JY, Hua MX, Du CJ, et al. The dual role of anti-viral therapy in the treatment of coronavirus disease 2019. Eur Rev Med Pharmacol Sci. 2020;24(22). https://doi.org/10.26355/eurrev_202011_23854.
Gharebaghi N, Nejadrahim R, Mousavi SJ, Sadat-Ebrahimi SR, Hajizadeh R. The use of intravenous immunoglobulin gamma for the treatment of severe coronavirus disease 2019: a randomized placebo-controlled double-blind clinical trial. BMC Infect Dis. 2020;20(1). https://doi.org/10.1186/s12879-020-05507-4.
Agarwal A, Mukherjee A, Kumar G, Chatterjee P, Bhatnagar T, Malhotra P. Convalescent plasma in the management of moderate COVID-19 in adults in India: open label phase II multicentre randomised controlled trial (PLACID Trial). BMJ. 2020;371. https://doi.org/10.1136/bmj.m3939.
Hermine O, Mariette X, Tharaux PL, Resche-Rigon M, Porcher R, Ravaud P. Effect of tocilizumab vs usual care in adults hospitalized with COVID-19 and moderate or severe pneumonia: a randomized clinical trial. JAMA Intern Med. 2021;181(1). https://doi.org/10.1001/jamainternmed.2020.6820.
Guan WJ, Wei CH, Chen AL, et al. Hydrogen/oxygen mixed gas inhalation improves disease severity and dyspnea in patients with coronavirus disease 2019 in a recent multicenter, open-label clinical trial. J Thorac Dis. 2020;12(6). https://doi.org/10.21037/jtd-2020-057.
Seo H, Kim H, Bae S, et al. Fluvoxamine treatment of patients with symptomatic COVID-19 in a community treatment center: a preliminary result of randomized controlled trial. Infect Chemother. 2022;54(1):102. https://doi.org/10.3947/ic.2021.0142.
Bégin P, Callum J, Jamula E, et al. Convalescent plasma for hospitalized patients with COVID-19: an open-label, randomized controlled trial. Nat Med. 2021;27(11):2012–24. https://doi.org/10.1038/s41591-021-01488-2.
Hu K, Guan W, Bi Y, et al. Efficacy and safety of Lianhuaqingwen capsules, a repurposed Chinese herb, in patients with coronavirus disease 2019: a multicenter, prospective, randomized controlled trial. Phytomedicine. 2021;85:153242. https://doi.org/10.1016/j.phymed.2020.153242.
Kyriazopoulou E, Poulakou G, Milionis H, et al. Early treatment of COVID-19 with anakinra guided by soluble urokinase plasminogen receptor plasma levels: a double-blind, randomized controlled phase 3 trial. Nat Med. 2021;27(10):1752–60. https://doi.org/10.1038/s41591-021-01499-z.
Krolewiecki A, Lifschitz A, Moragas M, et al. Antiviral effect of high-dose ivermectin in adults with COVID-19: a proof-of-concept randomized trial. eClinicalMedicine. 2021;37:100959. https://doi.org/10.1016/j.eclinm.2021.100959.
Sharma A, Shah M, Lakshmi S, et al. A pilot study for treatment of COVID-19 patients in moderate stage using intravenous administration of ozonized saline as an adjuvant treatment-registered clinical trial. Int Immunopharmacol. 2021;96:107743. https://doi.org/10.1016/j.intimp.2021.107743.
Pandit A, Bhalani N, Bhushan BLS, et al. Efficacy and safety of pegylated interferon alfa-2b in moderate COVID-19: a phase II, randomized, controlled, open-label study. Int J Infect Dis. 2021;105:516–21. https://doi.org/10.1016/j.ijid.2021.03.015.
Liesenborghs L, Spriet I, Jochmans D, et al. Itraconazole for COVID-19: preclinical studies and a proof-of-concept randomized clinical trial. EBioMedicine. 2021;66:103288. https://doi.org/10.1016/j.ebiom.2021.103288.
Ribeiro TB, Mazotti TA, Silva NA de O, Stein AT, Diaz-Quijano FA, de Melo DO. Evaluation of the initial response in clinical trial efforts for COVID-19 in brazil. Rev Bras Epidemiol. 2020;23:1–13. https://doi.org/10.1590/1980-549720200104.
Axfors C, Schmitt AM, Janiaud P, et al. Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nat Commun. 2021;12(1). https://doi.org/10.1038/s41467-021-22446-z.
Rojas-Marte G, Khalid M, Mukhtar O, et al. Outcomes in patients with severe COVID-19 disease treated with tocilizumab: a case-controlled study. QJM. 2020;113(8). https://doi.org/10.1093/qjmed/hcaa206.
Royle P, Waugh N. Should systematic reviews include searches for published errata? Heal Inf Libr J. 2004;21(1):14–20. https://doi.org/10.1111/j.1471-1842.2004.00459.x.
Farrah K, Rabb D. Errata for trial publications are not uncommon, are frequently not trivial, and can be challenging to access: a retrospective review. J Med Libr Assoc. 2019;107(2). https://doi.org/10.5195/jmla.2019.629.
Molckovsky A, Vickers MM, Tang P. Characterization of published errors in high-impact oncology journals. Curr Oncol. 2011;18(1):26–32. https://doi.org/10.3747/co.v18i1.707.
Castillo M, Northam M, Halm KE. Postpublication errors in imaging-related journals. Am J Neuroradiol. 2012;33(8):1447–8. https://doi.org/10.3174/ajnr.A3026.
Trikalinos TA. Does it mean anything if your own name is wrong in your published paper? FASEB J. 2009;23(8):2345–8. https://doi.org/10.1096/fj.09-130922.
Bhatt VR, Aryal MR, Panta S, Mosalpuria K, Armitage JO. A retrospective analysis of reported errata in five leading medical journals in 2012. J Community Hosp Intern Med Perspect. 2014;4(5):25738. https://doi.org/10.3402/jchimp.v4.25738.
Peterson CJ, Alexander R, Nugent K. COVID-19 article retractions in journals indexed in PubMed. Am J Med Sci. 2022;364(1). https://doi.org/10.1016/j.amjms.2022.01.014.
The project was supported by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH), through grant number UL1 TR001860 and linked award KL2 TR001859 to Dr Ascher. The authors wish to acknowledge the assistance of Bruce Abbott, MLS Health Sciences Librarian UC Davis, for his assistance in creating the search terms for this review.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Chatterton, B., Ascher, S.B., Duan, N. et al. Does haste make waste? Prevalence and types of errors reported after publication of studies of COVID-19 therapeutics. Syst Rev 12, 216 (2023). https://doi.org/10.1186/s13643-023-02381-4