- Open Access
- Open Peer Review
Sensitivity and specificity of using trial-of-antibiotics versus sputum mycobacteriology for diagnosis of tuberculosis: protocol for a systematic literature review
© The Author(s). 2018
- Received: 21 February 2018
- Accepted: 2 September 2018
- Published: 15 September 2018
Suboptimal diagnostics for pulmonary tuberculosis (PTB) drives use of ‘trial-of-antibiotics (non-tuberculosis)’ in an attempt to distinguish PTB patients from those with bacterial lower respiratory tract infection (LRTI). The underlying assumption—that patients with LRTI will report ‘response’ to broad-spectrum antibiotics, while those with PTB will not—has minimal evidence base for such a widely used intervention. Numerous potential causes of misclassification include bacterial super-infection of active PTB, placebo effect, and antimicrobial resistance (AMR). The main aim of this systematic review is to collate available evidence on the performance of trial-of-antibiotics as a diagnostic test and to explore the timing, interpretation, and decision-making process.
We will search MEDLINE, Embase, and Global Health using the Ovid platform for published studies that recruited adults being investigated for PTB, performed trial-of-antibiotics accompanied by mycobacteriological investigations, and reported both diagnostic test outcomes at the individual level. Following article selection, two authors will independently review titles and abstracts against eligibility criteria then perform full-text screening and extraction into a spreadsheet. We will conduct a risk of bias assessment at the level of the study using QUADAS-2 (University of Bristol) tool that assesses diagnostic evaluation work in four domains: (1) patient selection, (2) the index test, (3) the reference standard, and (4) patient flow and timing of tests. We will perform a narrative synthesis and, where possible, meta-analyses addressing our primary outcome. Our protocol adheres to the standards recommended by the PRISMA-P.
Pooling all available evidence on the accuracy, approach, and interpretation of results of trial-of-antibiotics in the context of PTB diagnosis will meet an urgent need, considering the widespread utilisation and potential for antimicrobial resistance. We therefore believe that our findings will have impact on policy and that they will inform the design of future detailed investigations into this important diagnostic approach.
Systematic review registration
Limitations of current diagnostics remain a challenge in the fight against tuberculosis (TB), a leading cause of infectious disease mortality with 10.4 million new cases and 1.8 million deaths annually . To complement the suboptimal diagnostics, standard diagnostic algorithms in resource-limited settings include a ‘trial-of-antibiotics.’ This is a course of broad-spectrum antibiotics, with negligible Mycobacterium tuberculosis activity, given to patients with symptoms such as cough in order to ‘rule-out’ or ‘rule in’ TB [2–4]. Patients with negative sputum mycobacteriology who respond to the antibiotic treatment are considered TB negative, while those who remain symptomatic are deemed likely to have TB and undergo further evaluations leading on to receiving TB treatment.
Approximately 26.5 million antibiotics courses are prescribed in the course of diagnosis of the 5.3 million smear negative TB registrations per annum . This estimate is based on assuming an average of 5 antibiotic courses per sputum-negative treatment initiation, with 2 courses given to the patients before TB treatment  and the other 3 courses accounting for patients whose symptoms resolved and TB was ruled out . Despite this widespread use, there has been no previous systematic review of the diagnostic performance of trial-of-antibiotics. The objective of this review is to assess existing evidence for the diagnostic sensitivity and specificity of using trial-of-antibiotics compared to sputum culture for TB diagnosis.
Other important evidence gaps on this subject include the choice of non-TB antibiotics (except for avoidance of those with known anti-TB activity), timing of the treatment, number of trials, the definition of treatment response, and the exact management after knowing the treatment outcome. Lack of consolidated evidence in these may be the source of the variations of implementation of trial-of-antibiotics across national programs. We will in this review consolidate existing evidence related to these gaps as our secondary objectives.
Our study will address the following Population, Index test, Reference test, Outcome (PIRO) question.
Our primary objective is to determine the sensitivity and specificity of using a trial-of-antibiotics compared to sputum mycobacteriology for diagnosis of pulmonary TB (PTB).
To describe the timing of prescription of the trial-of-antibiotics in TB diagnostic algorithms as reported in included articles
To describe the type, duration, and number of prescriptions of routine oral antibiotics
To establish how response to trial-of-antibiotics is interpreted and the decision-making process following positive or negative results
Adult patients with respiratory symptoms
Trial-of-antibiotics (any course of broad-spectrum antimicrobial given with the goal of ruling out TB in a symptomatic adult)
Any mycobacteriology test (we expect smear microscopy, smear microscopy using a fluorescent microscope, Cepheid GeneXpert, and mycobacterial culture)
Proportion of mycobacteriology-positive or mycobacteriology-negative participants correctly identified by trial-of-antibiotics (sensitivity and specificity)
Cross-sectional, cohort, and randomised controlled studies
Information sources and search strategy
Search strategy for MEDLINE using Ovid platform
Search in Ovid MEDLINE
Defining study population
(suspect* adj3 (TB or Tuberculosis)).mp.
(presumpt* adj3 (TB or Tuberculosis)).mp.
(probabl* adj3 (TB or Tuberculosis)).mp.
(suspect* adj3 (TB or Tuberculosis)).mp.
Defining study intervention
(Antibiotic* adj3 trial).mp.
(oral* adj3 antibiotic*).mp.
(amox?cillin or erythromycin or azithromycin or doxycyclin* or Vibramycin or clavulanic acid or co-amoxiclav).mp.
Defining study outcome
exp “Sensitivity and Specificity”/
exp “Predictive Value of Tests”/
((positive or negative) adj2 predictive value).mp.
(ppv or npv).mp.
9 and 15 (population and intervention)
23 and 24 (Population and intervention and outcome)
Applying pre-defined limits
limit 25 to yr=“1993 -Current”
In Table 2 below, we have presented our search strategy for MEDLINE, which has also been adapted for Embase and Global Health (see Additional file 1). This search strategy was reviewed by an information retrieval expert from the LSHTM library (Table 2). After completing the search in these databases, we will export results to Endnote X8 and remove all duplicates. We will also include all relevant articles identified from citations and reference lists of all included articles.
Study selection and data extraction
Investigator TD will implement the search strategy, and then, investigators TD and MN will independently sift through titles and abstracts of the resulting papers against the eligibility criteria. TD and MN will independently assess full texts of the included papers for eligibility using the above criteria. The main reason for non-inclusion at the full-text stage will be documented. Investigator KF will resolve any disagreements in eligibility. Investigators TD and MN will then extract data from all the eligible papers into an excel spreadsheet. Should we identify multiple publications from the same study, we will report data from one.
For studies with missing or incomplete information for meta-analysis, we will contact the authors by using the contact information provided in the publications. When attempts to contact the authors have not been successful, such studies will be excluded from the meta-analysis.
We will conduct a risk of bias assessment at the level of the study using the QUADAS-2 (University of Bristol), the recommended tool for evaluating primary studies for the inclusion in systematic reviews for diagnostic accuracy. The tool, provided in Additional file 2, has four domains evaluating (1) patient selection, (2) the index test, (3) the reference standard, and (4) patient flow and timing of tests. Assessment is done with respect to risk of bias and applicability of results.
We will provide a narrative synthesis of our results summarising the key findings, reporting on their consistency and quality, and identifying evidence gaps or limitations. We will perform a meta-analysis for sensitivity and specificity of trial-of-antibiotics against mycobacteriology tests for all studies providing true positives, false positives, true negatives, and false negatives. Our sensitivity-specificity joint modelling will require each study to provide data for both sensitivity and specificity.
We will utilise the MIDAS module  in Stata statistical software (version 15.0; Stata Corporation, College Station, TX, USA), to carry out the meta-analysis. We will also report point estimates and 95% confidence intervals, for sensitivity and specificity of trial-of-antibiotics versus mycobacteriology for each study and for pooled data, using bivariate random effects meta-analysis. We will report these results using a forest plot and plot a summary receiver operating characteristics (SROC) curve. We will examine clinical utility of trial-of-antibiotics using a Fagan plot.
Geographical location. While sensitivity and specificity cannot be influenced by disease prevalence, in the case of trial-of-antibiotics, causes of symptoms and antibiotic susceptibility may vary from place to place. We will assess performance of trial-of-antibiotics versus mycobacteriology in the following regions: (1) sub-Saharan Africa, (2) Asia, and (3) South America.
Type of reference test. The goal of a reference standard test is to provide error-free classification of the disease outcome presence or absence. Since for TB, there is no test that truly meets this definition, the performance of trial-of-antibiotics may vary depending on the inherent properties of each reference standard. We will assess the performance of trial-of-antibiotics versus mycobacteriology in the following regions: (1) studies using microscopy-based approaches, (2) studies using MTB culture, and (3) studies using Cepheid GeneXpert.
Assessment for heterogeneity and publication bias
We will assess the extent of heterogeneity of diagnostic specificity and sensitivity using Cochran Q2 and I2 tests. Diagnostic specificity and sensitivity forest plots and a bivariate boxplot will provide visual representation of the extent of heterogeneity.
There is limited consensus on the most appropriate approach for identifying evidence of publication bias in studies of diagnostic performance. We have decided to use the Deeks funnel plot , where the inverse of the square root of the effective sample size is plotted against the diagnostic odds ratio, and publication bias is deemed absent if the plot achieves a funnel shape.
For individual studies, we will present data as follows: author, year, country, whether a country has a low or high TB burden, population, sample size, design, TB reference standard, and results (sensitivity and specificity). We will present the results of our study selection using the approach prescribed by the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) .
We will prepare a manuscript, which we will submit for publication in a peer-reviewed journal. This work will also form part of a PhD thesis for TD, which he will submit to the London School of Hygiene & Tropical Medicine (LSHTM).
Protocol and registration
We registered this systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO), registration number CRD42017083915.
Our systematic review will be, to our knowledge, the first to pool evidence on the approach, implementation, and accuracy of using a trial-of-antibiotics for the diagnosis of tuberculosis. Trial-of-antibiotics is an integral component of diagnostic algorithms in low- and middle-income countries which, despite leading to 30 million empirical antibiotic prescriptions per annum, remains without strong evidence basis. Our findings therefore have high potential to prompt policy review as well as potentially stimulating funders and researchers to consider future studies into this component of the diagnostic algorithm.
We would like to thank the staff of the library at London School of Hygiene & Tropical Medicine for their support especially during the development of the search strategy.
TD is funded by the Commonwealth Scholarship Commission and the Helse Nord RHF. This review is part of his PhD work at London School of Hygiene & Tropical Medicine. The funding agencies had no role in the preparation of this manuscript nor will they take part in the systematic review process or its publication.
All authors substantially contributed to the conception and design of the study and reviewed all documents and materials. THD developed the first draft of the protocol and manuscript, developed the search strategy and data extraction forms, and performed the final review of the manuscript. MN critically reviewed the protocol and the manuscript and contributed towards the development of the search strategy and data extraction forms. DJS contributed to the study concept and reviewed the manuscript. AGW contributed to the study design and development of the search strategy and reviewed the final manuscript. ELC contributed to the conception, study design, and search strategy and critically reviewed the final protocol and manuscript. KF contributed to the conception, study design, search strategy, and methodological design and critically reviewed the final protocol and manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
This work will not involve direct contact with human subjects or participant identifiable data. Ethical approval is therefore not required for this study.
Consent for publication
All authors have given consent and approval for the manuscript to be submitted for publication.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- World Health Organization. Global tuberculosis report 2016. 2016.Google Scholar
- Ghana Health Service. Guidelines for the clinical management of TB and HIV co-infection in Ghana. Accra: Department DCaP; 2007.Google Scholar
- Ministry of Health. Malawi national TB programme manual. 8 ed. Lilongwe; 2016.Google Scholar
- National Department of Health. South Africa National Tuberculosis Management Guidelines 2014. Pretoria: Coordination TDS; 2014.Google Scholar
- Getahun H, Harrington M, O'Brien R, Nunn P. Diagnosis of smear-negative pulmonary tuberculosis in people with HIV infection or AIDS in resource-constrained settings: informing urgent policy changes. Lancet. 2007;369(9578):2042–9.View ArticleGoogle Scholar
- Wilkinson D, Newman W, Reid A, Squire S, Sturm A, Gilks C. Trial-of-antibiotic algorithm for the diagnosis of tuberculosis in a district hospital in a developing country with high HIV prevalence. Int J Tuberc Lung Dis. 2000;4(6):513–8.PubMedGoogle Scholar
- Dwamena BA. Midas: a program for meta-analytical integration of diagnostic accuracy studies in Stata. Ann Arbor: Division of Nuclear Medicine, Department of Radiology, University of Michigan Medical School; 2007.Google Scholar
- Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882–93.View ArticleGoogle Scholar
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ (Clinical research ed). 2009;339:b2535.View ArticleGoogle Scholar