Skip to main content
Download PDF

Recommendations for reporting of systematic reviews and meta-analyses of diagnostic test accuracy: a systematic review

Systematic Reviews volume 6, Article number: 194 (2017) Cite this article

Abstract

Background

This study is to perform a systematic review of existing guidance on quality of reporting and methodology for systematic reviews of diagnostic test accuracy (DTA) in order to compile a list of potential items that might be included in a reporting guideline for such reviews: Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA).

Methods

Study protocol published on EQUATOR website. Articles in full text or abstract form that reported on any aspect of reporting systematic reviews of diagnostic test accuracy were eligible for inclusion. We used the Ovid platform to search Ovid MEDLINE®, Ovid MEDLINE® In-Process & Other Non-Indexed Citations and Embase Classic+Embase through May 5, 2016. The Cochrane Methodology Register in the Cochrane Library (Wiley version) was also searched. Title and abstract screening followed by full-text screening of all search results was performed independently by two investigators. Guideline organization websites, published guidance statements, and the Cochrane Handbook for Diagnostic Test Accuracy were also searched. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Standards for Reporting Diagnostic Accuracy (STARD) were assessed independently by two investigators for relevant items.

Results

The literature searched yielded 6967 results; 386 were included after title and abstract screening and 203 after full-text screening. After reviewing the existing literature and guidance documents, a preliminary list of 64 items was compiled into the following categories: title (three items); introduction (two items); methods (35 items); results (13 items); discussion (nine items), and disclosure (two items).

Conclusion

Items on the methods and reporting of DTA systematic reviews in the present systematic review will provide a basis for generating a PRISMA extension for DTA systematic reviews.

Peer Review reports

Background

In their 2015 report titled “Improving Diagnosis in Healthcare”, the National Academy of Medicine identified a better understanding of the performance of diagnostic tests as an imminent priority for patient safety [1]. Systematic reviews, which incorporate findings from multiple primary studies, can increase confidence in our understanding of the accuracy of diagnostic tests in detecting medical conditions or diseases [2]. Systematic reviews and meta-analyses are cited more than any other study design and are prioritized in clinical practice guidelines [3,4,5]. Consistent with this, the number of systematic reviews, including those on diagnostic test accuracy (DTA), has grown extremely rapidly over the past decade [6, 7].

When systematic reviews and meta-analyses are poorly reported, readers are not able to assess the quality of the review and its underlying primary studies or to weigh the applicability of its conclusions. Thus, incomplete or inaccurate reports that do not transparently and completely convey review methods and results may mislead readers, rather than clarify the true value of a test. This contributes to waste of scarce medical research resources [8, 9] and hinders efforts to ensure the reproducibility of research. Previous studies have shown that many published DTA systematic reviews are not adequately reported [10, 11].

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement is a 27-item checklist and flow diagram that aims to provide guidance on complete and transparent reporting of systematic reviews [12]. Use of reporting guidelines, such as PRISMA, is associated with more informative reporting of medical research [10]. PRISMA was developed primarily for systematic reviews of medical interventions. While DTA systematic reviews share some common elements with intervention reviews, there are important differences. Thus, some items in the original PRISMA checklist may not apply to DTA reviews, and some essential items necessary for reporting DTA systematic reviews may be lacking [2, 6, 13, 14]. Existing guidance for reporting of DTA systematic reviews is limited to non-systematic “expert opinion” [2, 15, 16], guidance on specific methodologic items [6, 17], or work that is not yet complete [18].

The PRISMA-DTA group is developing an extension for DTA systematic reviews and meta-analyses. As the initial step, we performed a systematic review of existing guidance on reporting of DTA systematic reviews in order to compile a list of potential items that might be included in a reporting guideline for such reviews, the PRISMA extension for DTA (PRISMA-DTA).

Methods

The protocol for this review is available on the EQUATOR network’s website (http://www.equator-network.org/) in “guidelines under development” [19].

Database search

To identify published articles pertaining to reporting of DTA systematic reviews, an experienced medical information specialist (BS) developed a search strategy through an iterative process in consultation with the review team. The strategy was peer-reviewed prior to execution by another senior information specialist using the PRESS checklist [20]. Using the Ovid platform, we searched Ovid MEDLINE® and Ovid MEDLINE® In-Process & Other Non-Indexed Citations and Embase Classic+Embase on May 5, 2016. We also searched the Cochrane Methodology Register in the Cochrane Library, which contains records published July 2012 and earlier, (Wiley version) on the same date. Strategies used a combination of controlled vocabulary (e.g., “Diagnostic Tests, Routine,” “Review Literature as Topic,” “Publication Bias”) and keywords (e.g., “DTA,” “systematic review,” “reporting”). Vocabulary and syntax were adjusted across databases. There were no date or language restrictions on any of the searches. Specific details regarding search strategies appear in Appendix 1.

Inclusion/exclusion criteria, study selection, and data extraction

We included articles in full-text or abstract form that reported on any aspect of reporting DTA systematic reviews. Specifically, we included studies that evaluated the quality of reporting of any aspect of DTA systematic reviews and studies that provided guidance or suggestions as to how a DTA systematic review should be performed.

Titles and abstracts of all search results were screened independently for potential relevance by two investigators (MA, MDFM). For any citation deemed potentially relevant, full texts were retrieved and independently assessed in duplicate for inclusion with disagreements being resolved by consensus (TAM, MDFM). To facilitate the extraction process, studies were divided into several categories pertaining to the specific reporting topics: assessment of quality of reporting, general guidance on performing or reporting DTA systematic reviews, guidance on search methods for primary DTA studies, assessment of heterogeneity, pooling and meta-analysis methods, assessment of publication bias, risk of bias, and “other.” Reference list of included sources is provided in Appendix 2.

In addition to sources related to DTA systematic reviews, the following sources were reviewed: reporting guideline organizations’ websites (Enhancing the QUAlity and Transparency of Health Research (EQUATOR) [21]), guidance for reporting systematic reviews and meta-analyses of other types of research (Meta-analysis of Observational Studies in Epidemiology (MOOSE) [22], PRISMA [12], PRISMA extensions [23,24,25,26,27]), guidance for reporting diagnostic test accuracy studies (STARD 2015 [28], STARD for abstracts), guidance for, or tools for assessing the methodologic quality of systematic reviews and meta-analyses (A Measurement Tool to Assess Systematic reviews (AMSTAR) [29], risk of bias in systematic reviews (ROBIS) [30], Methodological Expectations of Cochrane Intervention Reviews (MECIR) [31]), and The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (completed chapters) [18]. Post hoc assessment of the following items not included in the initial search was done: the Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Comparative Effectiveness Research, the Institute of Medicine’s 2011 Standards for Systematic Reviews and the Centre for Reviews and Dissemination guidance [32,33,34]. No additional items were generated from these sources.

The PRISMA and STARD 2015 checklists were initially assessed independently and in duplicate in order to compile a list of potentially relevant items for the PRISMA-DTA statement. Any item that was deemed possibly relevant to DTA systematic reviews by either investigator was included. Next, all other guidance documents (reporting checklists, The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy, etc.) and full texts of potentially relevant records were similarly assessed in duplicate for additional potentially relevant items (TAM, MDFM). Again, any item that was deemed possibly relevant to DTA systematic reviews by either investigator was included. Items deemed relevant may have had wording changed from the original source to make them more applicable to systematic reviews of diagnostic test accuracy and/or broken into multiple sub-items to facilitate the Delphi process for PRISMA-DTA. All included items were used to generate a comprehensive summary of existing guidance on reporting of DTA systematic reviews.

Results

Database search

The database search yielded 6967 results. After title and abstract screening, 386 results remained. This was further reduced to 203 results after full-text screening (Fig. 1 ).

Fig. 1
figure 1

Study flow diagram

Full size image

Identification of potentially relevant items

After searching the existing literature and guidance documents, a preliminary list of 64 unique items was compiled and divided into the following categories mirroring the PRISMA statement: title (three items); introduction (two items); methods (35 items); results (13 items); discussion (nine items), and disclosure (two items). The methods section was further divided into eligibility criteria and search strategy (10 items), study selection and data extraction (seven items), primary study data items that should be provided (one item containing 10 sub-items.), risk of bias and heterogeneity (six items), and summary measures and statistics (11 items). The identified items along with citations for the sources from which they were taken are presented in Table 1; shaded items on the table indicate items specific to diagnostic accuracy systematic reviews, while unshaded items represent more general guidance for systematic reviews.

Table 1 Potential relevant items for PRISMA-DTA checklist. Items deemed by the authors to apply specifically to DTA reviews are in Bold
Full size table

Items were taken from 19 unique sources with publication dates between 2007 and 2016, a combination of guidance documents and some of the 203 search results. The 19 sources included the PRISMA statement [12], the PRISMA Explanation and Elaboration document [35], STARD 2015 [28], MECIR [31], AMSTAR [36], QUADAS-2 [14], eight research articles [6, 17, 37,38,39,40,41,42], two reviews [2, 43], two DTA statistical methodology overviews [44, 45], and one conference abstract [46]. Many of the 203 included results contained redundant information; one source was cited per item.

Summary of rationale for relevant items

This section will highlight some of the items that are proposed that have particular relevance to DTA systematic reviews.

Title: The potential items listed in this section aim to clearly identify “big picture” components of study design; this not only allows immediate reader comprehension, but enhances indexing and searchability. Items 1 and 2 are drawn from PRISMA and STARD 2015 and require that the title indicate that the study is a systematic review (item 1) and is a study of diagnostic accuracy (item 2). Item 3 required reporting on whether the study design is comparative (one test vs. another) or non-comparative; comparative design is increasingly important, common, and associated with methodologic challenges [37].

Introduction: Item 4 requires framing the role of the index test in the existing clinical pathway; understanding the clinical role of a test is essential to generalizability of findings. For example, if a test evaluation focuses on a “triage” test (e.g., d-dimer for determination of pre-test probability prior to CT pulmonary angiogram), it may not be appropriate to generalize its use as a “replacement” test (e.g., d-dimer as a replacement for CT). The performance of diagnostic tests is variable depending on the specific clinical scenario [28, 47].

Methods—protocol, eligibility, and search: All items in this section are generalizable to all systematic reviews; none were deemed to be specific to DTA systematic reviews.

Methods—study selection and data collection: Multiple items in this section focus on specific details of the search strategy and are aimed at enhancing reproducibility. None of these is of particular specific relevance to DTA reviews; however, detail additional to that recommended by PRISMA has been listed since subsequent systematic review methodologic recommendations have suggested their inclusion [31].

Methods—primary study data items: Item 25 focuses on which characteristics from primary studies included in a review should be reported. Several aspects of this item are unique to DTA systematic reviews, such as index test, reference standard, target condition definition, test positivity thresholds, and clinical setting. All this information is vital for readers to make an appropriate assessment of the review.

Methods—risk of bias and heterogeneity: Assessment of study quality and heterogeneity are not unique to DTA reviews. However, study quality assessment for diagnostic accuracy studies includes assessment of risk of bias and concerns regarding applicability, thus the quality assessment tool used in DTA reviews should capture and report these issues (item 24) [14]. Additionally, since sensitivity and specificity are correlated, univariate measures of heterogeneity, such as I 2, are typically not appropriate to report heterogeneity in diagnostic test accuracy reviews. Thus, heterogeneity may be reported either qualitatively or using measures that account for the correlation between sensitivity and specificity (item 28) [2].

Methods—summary statistics: Multiple readers may interpret an index test. How this is accounted for statistically may affect the results and, therefore, should be reported (item 33) [17]. An important difference in DTA meta-analysis from interventions is the correlation between sensitivity and specificity. Thus, it is very important to report the statistical model used for meta-analysis so readers can determine the impact of these methods on the results (item 34) [6].

Results: In order to facilitate reproduction of analyses and to make it clear to the readers which data was meta-analyzed, 2 × 2 data for each study included in meta-analyses should be made available (item 46) [43, 45].

Discussion and disclosure: All items in this section are generalizable to all systematic reviews; none was deemed to be specific to DTA systematic reviews.

Discussion

We consulted existing guidance on the reporting of systematic reviews and the published literature related to the conduct and reporting of DTA systematic reviews to identify 64 potential items for reporting DTA systematic reviews. The systematic, comprehensive search categorized by manuscript section builds on prior work, which has been based on non-systematic searches and expert opinion. The items identified will form the basis of a Delphi process that will be conducted to generate the PRISMA-DTA checklist. Items have been broken down into single concepts or descriptors for the Delphi process. During the Delphi process, suggestions from the PRISMA-DTA group will be incorporated. Thus, some items may not appear on the final PRISMA-DTA checklist. Additionally, PRISMA-DTA group members may propose additional items during the Delphi process. Wording of items as presented here may also be adjusted at the PRISMA-DTA consensus meeting. Therefore, it is advised to consult the final checklist after it has been published for use in guiding reporting systematic reviews of diagnostic test accuracy.

This evaluation improves on prior work, which has largely been based on non-systematic reviews, and expert opinion. The work is a small but essential step towards a clear reporting guideline for DTA systematic reviews. Future work should not only include creating the PRISMA-DTA checklist, but evaluating for “baseline” adherence to PRISMA-DTA in order to guide knowledge translation interventions aimed at targeted improvements for reporting of DTA systematic reviews.

Strengths and limitations

This systematic review benefits from a comprehensive, expert, peer-reviewed search, duplicate extraction, and categorization of potentially relevant items by manuscript section which mirrors the format of the PRISMA checklist. Limitations of our systematic review are that we did not formally assess the quality of sources for included items, we provide only a qualitative summary, and we may not have identified potentially relevant items from work yet to be published. We believe that many of these shortcomings will be addressed in the process for generation of the PRISMA-DTA checklist as outlined in our complete study protocol [48].

Conclusions

The reporting of DTA systematic reviews is often incomplete [10, 11, 49]. Incomplete reporting has been identified as a preventable source of waste in biomedical research [43]. Therefore, a reporting guideline specific to DTA systematic reviews is needed to reduce waste, increase utility, and facilitate reproducibility of these reviews. This systematic review is the first step towards gathering all relevant evidence pertinent to reporting of DTA systematic reviews. This step is critical in the EQUATOR network’s established guidance for reporting guidelines development [50]. This information will serve as the substrate for a PRISMA-DTA extension to guide reporting of DTA systematic reviews and will complement the more than 300 reporting guidelines indexed by the EQUATOR Network [21].

Abbreviations

AHRQ:

Agency for Healthcare Research and Quality

AMSTAR:

A Measurement Tool to Assess Systematic reviews

DTA:

Diagnostic test accuracy

EQUATOR:

Enhancing the QUAlity and Transparency Of health Research

MECIR:

Methodological Expectations of Cochrane Intervention Reviews

MOOSE:

Meta-analysis of Observational Studies in Epidemiology

PRESS:

Peer Review of Electronic Search Strategies

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PRISMA-DTA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy

QUADAS:

QUality Assessment of Diagnostic Accuracy Studies

ROBIS:

Risk of Bias in Systematic Reviews

STARD:

Standards for Reporting Diagnostic Accuracy

References

  1. Singh H, Graber ML. Improving diagnosis in health care—the next imperative for patient safety. N Engl J Med. 2015;373:2493–5.

    Article  PubMed  Google Scholar 

  2. McInnes MD, Bossuyt PM. Pitfalls of systematic reviews and meta-analyses in imaging research. Radiology. 2015;277:13–21.

    Article  PubMed  Google Scholar 

  3. Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;293:2362–6.

    Article  CAS  PubMed  Google Scholar 

  4. Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ. 2001;323:334–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical Practice Guidelines We Can Trust. National Academies Press (US); 2011. https://www.ncbi.nlm.nih.gov/books/NBK209539/.

  6. McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM. Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology. 2016;281:78–85.

    Article  PubMed  Google Scholar 

  7. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7:e1000326.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Obstet Gynecol. 2009;114:1341–5.

    Article  PubMed  Google Scholar 

  9. Moher D, Glasziou P, Chalmers I, Nasser M, Bossuyt PM, Korevaar DA, Graham ID, Ravaud P, Boutron I. Increasing value and reducing waste in biomedical research: who’s listening? Lancet. 2016;387:1573–86.

    Article  PubMed  Google Scholar 

  10. Tunis AS, McInnes MD, Hanna R, Esmail K. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology. 2013;269:413–26.

    Article  PubMed  Google Scholar 

  11. Willis BH, Quigley M. The assessment of the quality of reporting of meta-analyses in diagnostic research: a systematic review. BMC Med Res Methodol. 2011;11:163.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Macaskill P GC, Deeks JJ, Harbord RM, Takwoingi Y.: Chapter 10: analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C (editors), Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. In: The Cochrane Collaboration; 2010. http://methods.cochrane.org/sdt/handbook-dta-reviews.

  14. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM, Group Q. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–36.

    Article  PubMed  Google Scholar 

  15. Sotiriadis A, Papatheodorou SI, Martins WP. Synthesizing Evidence from Diagnostic Accuracy TEsts: the SEDATE guideline. Ultrasound Obstet Gynecol. 2016;47:386–95.

    Article  CAS  PubMed  Google Scholar 

  16. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM, Group CDTAW. Systematic reviews of diagnostic test accuracy. Ann Intern Med. 2008;149:889–97.

    Article  PubMed  PubMed Central  Google Scholar 

  17. McGrath T, McInnes M, Langer F, Hong J, Korevaar D, Bossuyt P. Treatment of multiple test readers in diagnostic accuracy systematic reviews of imaging studies. Eur J Radiol. 2017;93:59–64.

    Article  PubMed  Google Scholar 

  18. Deeks J, Bossuyt P, Gatsonis C. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. In: 1.0.0 edn: The Cochrane Collaboration; 2013.

    Google Scholar 

  19. Reporting Guidelines under development. [http://www.equator-network.org/library/reporting-guidelines-under-development/ - 99]. Last access: 14 Sept 2017.

  20. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

    Article  PubMed  Google Scholar 

  21. Enhancing the QUAlity and Transparency Of health Research [https://www.equator-network.org]. Last access: 14 Sept 2017.

  22. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008–12.

    Article  CAS  PubMed  Google Scholar 

  23. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, Ioannidis JP, Straus S, Thorlund K, Jansen JP, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777–84.

    Article  PubMed  Google Scholar 

  24. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, Group P-P: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015, 349:g7647.

  25. Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF, Group P-ID. Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. JAMA. 2015;313:1657–65.

    Article  PubMed  Google Scholar 

  26. Welch V, Petticrew M, Tugwell P, Moher D, O'Neill J, Waters E, White H. Group P-EB: PRISMA-Equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9:e1001333.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, Gøtzsche PC, Lasserson T, Tovey D, Group PfA. PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HC, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Radiology. 2015;277:826–32.

    Article  PubMed  Google Scholar 

  29. Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z, Ramsay T, Bai A, Shukla VK, Grimshaw JM. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One. 2007;2:e1350.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R, group R. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D: Methodlogical standards for the conduct of new Cochrane Intervention Reviews (MECIR). The Cochrane Collaboration; 2013. http://methods.cochrane.org/mecir.

  32. Quality AfHRa: Methods Guide for Effectiveness and Comparative Effectiveness Reviews. 2008.

    Google Scholar 

  33. Research IoMUCoSfSRoCE: Finding What Works in Health Care: standards for systematic reviews. National Academies Press; 2011. https://www.nap.edu/read/13059/chapter/1.

  34. Dissemination CfRa: Systematic Reviews: CRD’s guidance for undertaking reviews in health care. York University; 2009. https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf.

  35. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34.

    Article  PubMed  Google Scholar 

  36. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Takwoingi Y, Leeflang MM, Deeks JJ. Empirical evidence of the importance of comparative studies of diagnostic test accuracy. Ann Intern Med. 2013;158:544–54.

    Article  PubMed  Google Scholar 

  38. Menten J, Lesaffre E. A general framework for comparative Bayesian meta-analysis of diagnostic studies. BMC Med Res Methodol. 2015;15:70.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Leeflang MM, Rutjes AW, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ. 2013;185:E537–44.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Staub LP, Dyer S, Lord SJ, Simes RJ. Linking the evidence: intermediate outcomes in medical test assessments. Int J Technol Assess Health Care. 2012;28:52–8.

    Article  PubMed  Google Scholar 

  41. Naaktgeboren CA, van Enst WA, Ochodo EA, de Groot JA, Hooft L, Leeflang MM, Bossuyt PM, Moons KG, Reitsma JB. Systematic overview finds variation in approaches to investigating and reporting on sources of heterogeneity in systematic reviews of diagnostic studies. J Clin Epidemiol. 2014;67:1200–9.

    Article  PubMed  Google Scholar 

  42. McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM. Overinterpretation of research findings: evidence of “spin” in systematic reviews of diagnostic accuracy studies. Clin Chem. 2017;

  43. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267–76.

    Article  PubMed  Google Scholar 

  44. Takwoingi Y, Riley RD, Deeks JJ. Meta-analysis of diagnostic accuracy studies in mental health. Evid Based Ment Health. 2015;18:103–9.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Riley RD, Ahmed I, Debray TP, Willis BH, Noordzij JP, Higgins JP, Deeks JJ. Summarising and validating test accuracy results across multiple studies for use in clinical practice. Stat Med. 2015;34:2081–103.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Westwood M, Whiting P: Should systematic reviews of diagnostic tests go beyond test accuracy? In 16th Cochrane Colloquium: evidence in the era of globalisation; Freiburg, Germany. 2008. https://www.york.ac.uk/inst//crd/Posters/Should%20systematic%20reviews%20of%20diagnostic%20tests%20go%20beyond%20test%20accuracy.pdf.

  47. Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, Irwig L, Levine D, Reitsma JB, de Vet HC, Bossuyt PM. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6:e012799.

    Article  PubMed  PubMed Central  Google Scholar 

  48. McInnes M, Moher D, Bossuyt P: Development and implementation of a reporting guideline for systematic reviews and meta-analyses of diagnostic accuracy studies: The PRISMA-DTA initiative. 2016.

    Google Scholar 

  49. Willis BH, Quigley M. Uptake of newer methodological developments and the deployment of meta-analysis in diagnostic test research: a systematic review. BMC Med Res Methodol. 2011;11:27.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7:e1000217.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not Applicable.

Availability of data and materials

Data not provided in this manuscript or related appendices is available from the authors on request.

Funding

Canadian Institute for Health Research (Grant Number 375751).

Canadian Agency for Drugs and Technologies in Health (CADTH).

STAndards for Reporting of Diagnostic accuracy studies group (STARD).

University of Ottawa Department of Radiology Research Stipend Program.

Author information

Authors and Affiliations

  1. Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada

    Trevor A. McGrath

  2. Department of Radiology, McMaster University, Hamilton, ON, Canada

    Mostafa Alabousi

  3. Ottawa Hospital Research Institute, Ottawa, ON, Canada

    Becky Skidmore

  4. Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands

    Daniël A. Korevaar & Patrick M. M. Bossuyt

  5. Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada

    David Moher

  6. Lady Davis Institute of the Jewish General Hospital and Department of Psychiatry, McGill University, Montreal, Quebec, Canada

    Brett Thombs

  7. University of Ottawa Department of Radiology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Room c159 Ottawa Hospital Civic Campus, 1053 Carling Ave, Ottawa, ON, K1Y 4E9, Canada

    Matthew D. F. McInnes

Authors
  1. Trevor A. McGrath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mostafa Alabousi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Becky Skidmore
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniël A. Korevaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Patrick M. M. Bossuyt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David Moher
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Brett Thombs
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Matthew D. F. McInnes
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TAM, MA, BS, and MDFM contributed to the data collection. TAM and MDFM contributed to the data analysis. MDFM is the guarantor of the entire study. All authors have substantial contributions to study protocol design and approval, manuscript revision, and approval of final version of the manuscript.

Corresponding author

Correspondence to Matthew D. F. McInnes.

Ethics declarations

Ethics approval and consent to participate

Ethical approval is not required for this type of study at the authors’ institutions.

Consent for publication

All authors provide consent for publication.

Competing interests

David Moher is Editor-in-Chief of Systematic Reviews. No other relevant competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Search Strategy 2016 May 5 Ovid Multifile Database: Embase Classic+Embase <1947 to 2016 May 04>, Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R) <1946 to Present>

Search Strategy:

---------------------------

1 “Diagnostic Techniques and Procedures”/ (80885)

2 exp Diagnostic Imaging/ (2015520)

3 “Diagnostic Tests, Routine”/ (73316)

4 (diagnos* adj3 test*).tw,kw. (153714)

5 (diagnos* adj3 accura*).tw,kw. (155998)

6 (test* adj3 accura*).tw,kw. (29472)

7 (diagnos* adj3 compar*).tw,kw. (53752)

8 diagnostic stud$3.tw,kw. (12147)

9 DTA.tw,kw. (5191)

10 (DTAR or DTARs).tw,kw. (34)

11 or/1-10 (2464773)

12 meta analysis.pt. (65207)

13 meta-analysis as topic/ (26801)

14 (meta-analy* or metanaly* or metaanaly* or met analy* or integrative research or integrative review* or integrative overview* or research integration or research overview* or collaborative review*).tw. (214034)

15 (systematic review* or systematic overview* or evidence-based review* or evidence-based overview* or (evidence adj3 (review* or overview*)) or meta-review* or meta-overview* or meta-synthes* or “review of reviews” or technology assessment* or HTA or HTAs).tw. (251988)

16 exp Technology assessment, biomedical/ (21442)

17 “Review Literature as Topic”/ (56641)

18 or/12-17 (492218)

19 11 and 18 (22011)

20 exp Quality Control/ (351525)

21 Publishing/ (61930)

22 Publication Bias/ (37246)

23 Research Report/ (33327)

24 Periodicals as Topic/ (158426)

25 Checklist/ (15858)

26 Research Design/ (1691506)

27 exp Reproducibility of Results/ (483550)

28 ((report* or method* or publicat*) adj3 (assess or apprais* or bias* or characteristic* or criteri* or critiqu* or evaluat* or guidance* or guideline* or quality or checklist* or check list* or recommend* or score$1 or scoring or standard*)).tw,kw. (665469)

29 reporting.tw,kw. (275122)

30 methodolog*.ti,kw. (76262)

31 PRISMA.tw,kw. (5494)

32 or/20-31 (3475491)

33 19 and 32 (6906)

34 33 use prmz (3684)

35 exp. diagnostic test/ (840676)

36 (diagnos* adj3 test*).tw,kw. (153714)

37 diagnostic accuracy/ (200586)

38 (diagnos* adj3 accura*).tw,kw. (155998)

39 (diagnos* adj3 compar*).tw,kw. (53752)

40 diagnostic test accuracy study/ (44252)

41 (test* adj3 accura*).tw,kw. (29472)

42 DTA.tw,kw. (5191)

43 (DTAR or DTARs).tw,kw. (34)

44 diagnostic stud$3.tw,kw. (12147)

45 or/35-44 (1319550)

46 meta-analysis/ (173526)

47 “systematic review”/ (105948)

48 “meta analysis (topic)”/ (26341)

49 (meta-analy* or metanaly* or metaanaly* or met analy* or integrative research or integrative review* or integrative overview* or research integration or research overview* or collaborative review*).tw. (214034)

50 (systematic review* or systematic overview* or evidence-based review* or evidence-based overview* or (evidence adj3 (review* or overview*)) or meta-review* or meta-overview* or meta-synthes* or “review of reviews” or technology assessment* or HTA or HTAs).tw. (251988)

51 or/46-50 (475344)

52 45 and 51 (20896)

53 medical literature/ (137121)

54 quality control/ (184009)

55 publishing/ (61930)

56 publication/ (145797)

57 checklist/ (15858)

58 reproducibility/ (170497)

59 ((report* or method* or publicat*) adj3 (assess or apprais* or bias* or characteristic* or criteri* or critiqu* or evaluat* or guidance* or guideline* or quality or recommend* or checklist* or check list* or score$1 or scoring or standard*)).tw,kw. (665469)

60 reporting.tw,kw. (275122)

61 methodology/ (1646967)

62 methodolog*.ti,kw. (76262)

63 PRISMA.tw,kw. (5494)

64 or/53-63 (3073039)

65 52 and 64 (6530)

66 65 use emczd (5036)

67 34 or 66 (8720)

68 limit 67 to yr=“2011 -Current” (5297)

69 remove duplicates from 68 (4253)

70 67 not 68 (3423)

71 remove duplicates from 70 (2867)

72 69 or 71 (7120) [TOTAL UNIQUE RECORDS]

73 72 use prmz (3588) [UNIQUE MEDLINE RECORDS]

74 72 use emczd (3532) [UNIQUE EMBASE RECORDS]

Cochrane Library

Search Name: PRISMA - DTA - Reviews/Meta-Analyses - Methodology

Date Run: 05/05/16 14:41:54.305

Description: 2016 May 5 (OHRI)

ID Search Hits

#1 [mh ^“Diagnostic Techniques and Procedures”] 116

#2 [mh “Diagnostic Imaging”] 35671

#3 [mh “Diagnostic Tests, Routine”] 328

#4 (diagnos* near/3 test*):ti,ab,kw 6315

#5 (diagnos* near/3 accura*):ti,ab,kw 5863

#6 (test* near/3 accura*):ti,ab,kw 3703

#7 (diagnos* near/3 compar*):ti,ab,kw 1936

#8 (diagnostic next (study or studies)):ti,ab,kw 189

#9 DTA:ti,ab,kw 22

#10 (DTAR or DTARs):ti,ab,kw 2

#11 in Methods Studies 521

Methods – 521

Appendix 2

Table 2 List of 203 included studies
Full size table

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McGrath, T.A., Alabousi, M., Skidmore, B. et al. Recommendations for reporting of systematic reviews and meta-analyses of diagnostic test accuracy: a systematic review. Syst Rev 6, 194 (2017). https://doi.org/10.1186/s13643-017-0590-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-017-0590-8

Keywords

Systematic Reviews

ISSN: 2046-4053