Skip to main content

Advertisement

Steps toward more complete reporting of systematic reviews of diagnostic test accuracy: Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA)

Abstract

Reporting standards in biomedical research have been shown to be suboptimal. The publication of the PRISMA statement has improved the completeness of reporting of systematic reviews, but several issues specific to diagnostic test accuracy are not included in the PRISMA statement. Therefore, a diagnostic test accuracy extension of the PRISMA statement, PRISMA-DTA, was created. This commentary addresses completeness of reporting in systematic reviews, the PRISMA-DTA statement, and strategies for optimal uptake of reporting guidelines.

Background

Inadequate reporting in biomedical research is a long-recognized issue, with the concern being that it does not allow for assessment of methodological quality and applicability of results, and consequently leads to wasted research that may be prone to bias [1, 2]. Poorly reported randomized controlled trials are associated with exaggerated treatment effects [3]. The EQUATOR Network was launched in 2008 to facilitate the development and dissemination of reporting guidelines to improve the quality of reporting in biomedical literature [4]. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was published in 2009 with the goal to improve reporting of systematic reviews and meta-analysis and reducing research waste [2, 5, 6]. Since the publication of the PRISMA statement, reporting of systematic reviews has improved in several areas [7, 8].

While the focus of PRISMA was to help improve the completeness of reporting systematic reviews examining the effects of health interventions, several key issues in systematic reviews of diagnostic test accuracy (DTA) were not adequately addressed [9,10,11,12]. Several important differences exist between the conduct of a systematic review of interventions compared to a review of DTA. These differences also need to be transparently reported. Systematic reviews of interventions, and the PRISMA statement, reference the clinical question in terms of Participants, Intervention, Comparison, and Outcomes (PICO) whereas in DTA systematic reviews, Participants, Index test(s), and Target condition (PIT) is more applicable [5, 9]. Data handling also differs between DTA systematic reviews versus systematic reviews of interventions, with issues such as multiple thresholds for test positivity, multiple readers of the index test, and indeterminate test results [9]. Additionally, since sensitivity and specificity are correlated and vary with respect to the diagnostic threshold, hierarchical methods are generally recommended for meta-analysis of DTA [12, 13]. Using hierarchical pooling methods has been shown to give significantly different results than non-hierarchical methods and thus reporting of the methods used for meta-analysis is critically important in DTA systematic reviews. To address these unique issues, and others, an extension to PRISMA for authors reporting reviews of DTA called PRISMA-DTA was developed. The onus to improve the reporting of DTA systematic reviews falls on everybody involved in the publication of a manuscript, from the study authors, to journal peer reviewers and journal editors; and thus, awareness of PRISMA-DTA is of importance to anyone involved in the DTA systematic review research domain.

PRISMA-DTA

A systematic review of the literature regarding reporting of DTA systematic reviews was performed followed by a three-round Delphi process and an in-person consensus meeting to generate the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA) extension of the original PRISMA statement [9, 10]. PRISMA-DTA for abstracts was also developed. The PRISMA-DTA and PRISMA-DTA for abstracts publication is available (open-access) in the Journal of the American Medical Association (JAMA) [9]. PRISMA-DTA and PRISMA-DTA for abstracts user-friendly checklists for authors and reviewers are available in both Word and PDF formats from both The EQUATOR Network and the PRISMA websites [5, 14]. This extension of the PRISMA statement should be used for all DTA systematic reviews performed moving forward. Additionally, submissions of DTA systematic reviews to conferences should be adherent to PRISMA-DTA for abstracts as conference abstracts are often published and, like published manuscripts, should meet minimum reporting standards.

Reporting guideline uptake

Optimal uptake of reporting guidelines has three main constituents: endorsement, implementation, and adherence. Endorsement is a policy decision by a journal to recommend that authors submit the appropriate reporting guideline for their research type, as is the case with the journal Systematic Reviews, along with other leading journals in fields commonly producing DTA systematic reviews, such as Clinical Chemistry and Radiology. Implementation is the action(s) taken by journals to ensure that authors adhere to an endorsed reporting guideline. This may include requiring a checklist of the relevant reporting guideline be submitted with the manuscript prior to consideration of the manuscript by the journal editorial team. Adherence refers to action(s) taken by authors to ensure that a research report is compliant with the items recommended by the appropriate/relevant reporting guideline. Adherence is the responsibility of the manuscript author to create well-reported and reproducible research. As the editorial burden of manually assessing and enforcing adherence of all submitted manuscripts would be enormous, enforcement of adherence currently largely relies on peer reviewers and editors. From the perspective of the author submitting a review, using PRISMA-DTA will ideally improve the completeness and clarity of reporting of the submitted work, perhaps aiding peer reviewers and editors to more quickly reach a decision of the merits of the review for publication.

Solutions

Reporting of systematic reviews has been shown to be improved since the publication of the PRISMA statement [8]. Studies have shown that journal endorsement of reporting guidelines does not necessarily improve the completeness of reporting of systematic reviews [15, 16]. However, in a recent assessment of the adherence of DTA systematic reviews to PRISMA-DTA, it was found that reviews citing PRISMA or an extension of PRISMA had significantly more complete reporting than reviews that did not [16]. The lack of improvement in reporting seen in journals endorsing reporting guidelines versus journals that did not is potentially confounded by authors using reporting guidelines despite lack of journal endorsement. In the 100 reviews in the PRISMA-DTA reporting assessment, two thirds cited PRISMA or a PRISMA extension, whereas only one third were submitted to journals endorsing reporting guidelines for systematic reviews [16]. Journal endorsement of reporting guidelines, although not shown in isolation to improve DTA systematic review reporting, can help drive the overall use of reporting guidelines, which was shown to improve reporting of DTA systematic reviews [16]. Thus, we encourage all journals to endorse reporting guidelines.

While fixed abstract word count limitations by journals were not associated with completeness of abstract reporting, it was shown that abstracts with higher word counts were reported more completely [16]. Thus, we encourage authors, if faced with a fixed word count limit for the abstract, to be as complete as possible and use the word count allotment fully. Additionally, studies using supplemental material were reported more completely [16]. If authors are faced with full-text word count limits, restrictions on figures and tables, or other constraints, we encourage the use of online supplemental material which can be mentioned in the full text of the article to steer readers towards complete information for their review. As motivation for review authors, if required, it was shown that systematic reviews published in high impact radiology journals were cited more often when reporting was more complete [17].

Several DTA-specific issues were found to be reported in less than one third of reviews in the PRISMA-DTA adherence assessment [16]. These include the clinical setting, which has been shown to influence test accuracy, as well as definitions of the target condition, index test, and reference standard [16]. Systematic reviews of DTA pose unique methodological and thus reporting issues not seen in systematic reviews of interventions. The originally published PRISMA statement was not designed to address these issues, and thus, a specialized DTA reporting guideline should be used to ensure adequate DTA-specific reporting for all DTA systematic reviews moving forward.

Interestingly, a recently published study found that study authors found reporting guidelines to be most useful when used early in the research process, rather than after the manuscript was complete or requested by a journal editor [18]. This shows that the perceived value of reporting guidelines is highest during the planning and protocol development phase. Knowing this, it seems an emphasis with respect to reporting guidelines should be made when educating biomedical researchers about different study designs. If researchers are aware of reporting guidelines and can use them early in the research process, when they have shown to be perceived as the most helpful, then hopefully usage rates of reporting guidelines on the whole will improve.

Conclusion

With the publication of the PRISMA-DTA and PRISMA-DTA for abstract checklists and the continued efforts of authors, peer reviewers, and journal editors, hopefully we can raise the quality of reporting in DTA systematic reviews.

Availability of data and materials

Not applicable

Abbreviations

DTA:

Diagnostic test accuracy

JAMA:

Journal of the American Medical Association

PICO:

Participants, Intervention, Comparison, and Outcomes

PIT:

Participants, Index test(s), and Target condition

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PRISMA-DTA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Diagnostic test accuracy

References

  1. 1.

    Schulz KF. Randomised trials, human nature, and reporting guidelines. Lancet. 1996;348:596–8.

  2. 2.

    Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267–76.

  3. 3.

    Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–12.

  4. 4.

    Altman DG, Simera I, Hoey J, Moher D, Schulz K. EQUATOR: reporting guidelines for health research. Lancet. 2008;371:1149–50.

  5. 5.

    Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

  6. 6.

    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34.

  7. 7.

    Tunis AS, McInnes MD, Hanna R, Esmail K. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology. 2013;269:413–26.

  8. 8.

    Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13:e1002028.

  9. 9.

    McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, Clifford T, Cohen JF, Deeks JJ, Gatsonis C, Hooft L, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA Statement. JAMA. 2018;319:388–96.

  10. 10.

    McGrath TA, Alabousi M, Skidmore B, Korevaar DA, Bossuyt PMM, Moher D, Thombs B, McInnes MDF. Recommendations for reporting of systematic reviews and meta-analyses of diagnostic test accuracy: a systematic review. Syst Rev. 2017;6:194.

  11. 11.

    McGrath T, McInnes M, Langer F, Hong J, Korevaar D, Bossuyt P. Treatment of multiple test readers in diagnostic accuracy systematic reviews of imaging studies. Eur J Radiol. 2017;93:59–64.

  12. 12.

    McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM. Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology. 2016;281:78–85.

  13. 13.

    Deeks J, Bossuyt P, Gatsonis C: Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. 1.0.0 edn: The Cochrane Collaboration; 2013.

  14. 14.

    Enhancing the quality and transparency of health research [https://www.equator-network.org]. Accessed 28 Nov 2018.

  15. 15.

    Stevens A, Shamseer L, Weinstein E, Yazdi F, Turner L, Thielman J, Altman DG, Hirst A, Hoey J, Palepu A, et al. Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804.

  16. 16.

    Salameh JP, McInnes MDF, Moher D, Thombs BD, McGrath TA, Frank R, Dehmoobad Sharifabadi A, Kraaijpoel N, Levis B, Bossuyt PM. Completeness of reporting of systematic reviews of diagnostic test accuracy based on the PRISMA-DTA Reporting Guideline. Clin Chem. 2019;65:291–301.

  17. 17.

    van der Pol CB, McInnes MD, Petrcich W, Tunis AS, Hanna R. Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates? PLoS One. 2015;10:e0119892.

  18. 18.

    Dewey M, Levine D, Bossuyt PM, Kressel HY. Impact and perceived value of journal reporting guidelines among radiology authors and reviewers. Eur Radiol 2019. https://www.ncbi.nlm.nih.gov/pubmed/30694365

Download references

Acknowledgements

We would like to thank the PRISMA-DTA Group for all their hard work bringing this project to completion.

Funding

Funding Sources:

1. Canadian Institute for Health Research (Grant Number 375751)

2. Canadian Agency for Drugs and Technologies in Health (CADTH)

3. STAndards for Reporting of Diagnostic Accuracy Studies Group (STARD)

4. Research Stipend Program, Department of Radiology, University of Ottawa

Author information

TM drafted the initial version. DM and MM made substantial additions to the first and subsequent drafts. All authors read and approved the final manuscript.

Correspondence to Matthew D. F. McInnes.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

David Moher is a co-editor-in-chief of the journal. All other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Systematic reviews
  • Diagnostic test accuracy
  • PRISMA-DTA
  • Completeness of reporting

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.