Diagnostic accuracy of molecular methods for detecting markers of antimalarial drug resistance in clinical samples of Plasmodium falciparum: protocol for an update to a systematic review and meta-analysis
Systematic Reviews volume 7, Article number: 221 (2018)
Each year, infection with Plasmodium causes millions of clinical cases of malaria and hundreds of thousands of deaths. Resistance to different antimalarial medications continues to develop and spread, threatening effective prophylaxis and treatment. Surveillance of resistance is required to inform health policy and preserve effective antimalarial drugs; molecular methods can be used to surveil likely parasite resistances. However, there is no consensus on the most accurate molecular methods, and large variation exists in practice. The objective of this update to this systematic review is to improve and update identification of the sensitivity and specificity of each molecular method for detecting selected antimalarial drug resistance markers.
We will include diagnostic accuracy studies that compare at least two of any molecular methods to examine blood samples from patients diagnosed with, or suspected of having malaria, to detect at least one selected marker of antimalarial drug resistance. We will search PubMed, EMBASE, BIOSIS, and Web of Science from 2000 to present. Two reviewers will independently screen all results, extract data, consider applicability, and evaluate the methodological quality of included studies using QUADAS-2. We will carry out a meta-analysis and use statistical methods to compare results from homogenous studies. We will use narrative to synthesise and compare results of heterogeneous studies.
This review will help to identify sub-optimal molecular methods for antimalarial marker detection which may be discontinued and identify more sensitive and specific methods which may be adopted. More sensitive and specific detection of drug resistance can be used to improve the breadth and accuracy of surveillance. This would enable the identification of previously undiscovered areas of antimalarial resistances and susceptibilities, improve the precision of estimates of the prevalence of resistances, and improve our ability to detect smaller changes in these patterns. Higher-quality evidence generated by more accurate and detailed surveillance can be used to inform guidelines on the use of antimalarial drugs, leading to better outcomes for more patients.
Systematic review registration
This systematic review protocol was registered with PROSPERO on 22 November 2017 (registration number CRD42017082101).
Infection with Plasmodium caused approximately 216 million clinical cases of malaria and 445,000 deaths in 2016 . However, malaria can be prevented and treated effectively [2, 3]. However, some Plasmodium falciparum (P. falciparum) parasites are resistant, or less susceptible, to some antimalarial chemotherapies [4,5,6,7,8,9,10,11,12,13]. The emergence and spread of drug-resistant parasites are a great impediment to achieving control of malaria and eventual elimination.
While worldwide surveillance of artemisinin resistance has been prioritised, surveillance of resistances to other prophylactic and curative antimalarials is also essential [14,15,16,17]. Continuous global surveillance of therapeutic efficacy and drug resistance is enabling the detection of epidemiological patterns of parasite resistances to antimalarial drugs in order to inform malaria prevention and treatment policies [18, 19].
Historically, parasite resistances have been determined using clinical trials or observational studies. In addition, for a limited number of clinical samples, therapeutic efficacy can be tested through ex vivo and in vitro culture methods. Molecular methods may offer many advantages over both of these: reduced patient burden, reduced number of staff, faster results, and the opportunity to test possible resistances to many antimalarial drugs [14, 20,21,22]. However, the correlation between the genotypes of markers and the phenotypes of resistances is mixed in strength for different molecular markers; the clinical relevance of these molecular markers varies [23, 24].
There is a lack of a standardised protocol for detecting resistance via molecular markers, with variation in sample type, collection, storage, DNA extraction, marker detection, and analysis of results, all of which may also affect the sensitivity and specificity of methods [14, 25].
It seems likely that there is a variation in the accuracy of different molecular methods for detecting antimalarial drug resistance. For example, for similar patients, Sarmah et al.  determined pfdhfr N51I, C59R, and S108N for 75/75 samples, while Mishra et al.  were successful for 232/311 samples.
An updated systematic review will identify the molecular methods in use and the sensitivity and specificity of each method for detecting each drug resistance marker. Any methods found to be sub-optimal in a particular setting could be discontinued from use and replaced by the more sensitive and specific methods. Use of more sensitive and specific methods could require fewer resources to generate an equal amount of data. It could result in fewer gaps in surveillance and increased accuracy of surveillance. Improved surveillance could identify previously undiscovered pockets of antimalarial resistances and susceptibilities, improve the accuracy and precision of estimates of known resistance, and improve our ability to detect smaller changes in these patterns.
Improved guidelines on the use of antimalarial drugs, based on this more complete, accurate, and precise information, would likely lead to better outcomes for more patients.
One review on the subject has been carried out; the sensitivities and specificities of many methods were determined in different settings. However, the confidence intervals were large, and the risks of bias were unclear for many studies. In addition, this review was carried out by a single reviewer; an update and extension to the first review is required to improve the methodological quality of the review and to include additional studies .
The objectives of this review are to:
Obtain per study estimates of the sensitivity and specificity of each molecular method for determining the genotype of each drug resistance marker.
Obtain summary estimates of the sensitivity and specificity of each molecular method for determining the genotype of each drug resistance marker.
Use statistical methods to compare estimates of sensitivity and specificity of different methods to identify the most sensitive and specific methods.
Narratively synthesise and compare estimates of sensitivity and specificity to identify the most sensitive and specific methods when statistical synthesis and comparison are not appropriate.
We will select studies according to the criteria below:
We will include only diagnostic accuracy studies. We will include case-control studies in the review but may exclude them from the meta-analysis.
We will include studies examining any number of blood samples, taken from patients or asymptomatic patients when there is a suspicion that they may be carrying P. falciparum parasites. We will exclude data generated using artificial infections, lab-adapted Plasmodium strains, and ‘spiked’ samples.
We will include as index and reference tests any molecular methods for determining the presence or the genotype of at least one molecular marker associated with antimalarial drug resistance. We will include all studies comparing at least two methods. There is no gold standard test and no exhaustive list of methods. By including all methods, we aim to include evidence applicable to all surveillance settings. This was a successful technique for the first iteration of this review.
The target condition is the genotype, haplotype, or copy number of a molecular marker of antimalarial drug resistance; the genes containing these markers are listed in Table 1. We will include studies that examine at least one of these genes.
We will include studies for which sufficient data to construct a 2 × 2 table of test accuracy is published or can be obtained from authors. When results to construct a 2 × 2 table are not recovered, studies will be listed but will not be included in the quantitative or narrative syntheses.
We will include studies from any setting, regardless of the prevalence of Plasmodium or presence of different Plasmodium species.
We will not restrict studies by language of publication.
We will include only studies for which there is published report.
We will search PubMed, EMBASE (Ovid), BIOSIS (Web of Knowledge), and Web of Science Core Collection (Web of Knowledge) from 2000 to present. We will not search for studies published before 2000 as none were found in the comprehensive search from 1983 undertaken in the first iteration of this review. We will check reference lists of eligible studies identified through the search. We will circulate a list of the eligible reports to authors of included reports and to experts in malaria diagnostics and drug resistance that we have identified. We will make an initial search and a second search later in the review to check for new studies. We will utilise literature search strategies using medical subject headings (MeSH) and terms related to malaria, drug resistance, genes of interest, and molecular methods. No filters will be used. Piloted search terms for all databases are included in Additional file 1.
We will upload literature search results to EndNote 8.2 software and Covidence.
We will assess titles or titles and abstracts of search results using a piloted screening checklist based on our inclusion and exclusion criteria; the screening checklist is available in Additional file 2. We will obtain the full text of reports for any potentially eligible studies or when there is any uncertainty. We will use the same screening form to assess full-text reports.
We will use report and study details to identify multiple reports of the same study or samples; we will compare any potential duplicate reports in full. At any stage of the review, if we find multiple reports of the same study, all but one report will be excluded. A second reviewer will independently screen the search results. A third reviewer will resolve disagreements. We will record at least one reason that reports or studies are excluded. Review authors will not be blind to the journal titles, study authors, or institution.
Data collection process
We will extract data from all eligible studies using piloted standardised data extraction forms; the data extraction forms are available in Additional file 3. Two reviewers will independently extract all data. Disagreements will be resolved by a third reviewer. When necessary, we will seek from authors, using the results data extraction form in Additional file 4, unpublished data to construct 2 × 2 tables of test accuracy.
We will extract report and publication details. We will extract study identifiers and characteristics including design, location and time, patient and sample characteristics, molecular methods used and markers examined, flow and timing of patients, and samples through testing. We will extract details on potential sources of bias for assessment using the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) checklist and other items recommended by STARD (Standards for Reporting of Diagnostic Accuracy) [36, 37]. We will extract all results relating to test accuracy. A full list of data items is available in Additional file 3.
Sensitivity and specificity will be calculated from any test accuracy outcomes provided. If data cannot be obtained in a numeric format, it will be translated from graphically presented data using im2graph 1.21.
Outcomes and prioritisation
The primary outcome will be:
The sensitivity and specificity from each study, of each molecular method for determining the genotype of any drug resistance markers in any gene listed in Table 1.
Risk of bias in individual studies
We will assess the risk of bias using the piloted adapted QUADAS-2 tool in Additional file 4. For each domain, two reviewers will independently make an assessment of the risk of bias from the extracted information and rate the risk of bias as “low”, “high”, or “unclear”. If insufficient data is available in reports or obtained as planned, the risk of bias will be rated as “unclear”. Disagreements will be resolved by a third reviewer. In addition, we will consider other sources of bias. We will create a graphical representation of risk of bias within included studies using Covidence. We will describe how the results of the assessment contribute to the overall findings of the review.
We will perform analyses using Stata 15 and display key characteristics in tables. For each study, we will determine sensitivity and specificity with 95% confidence intervals (CIs). Where haplotypes are reported, they will be synthesised independently to genotypes. It is anticipated that thresholds of test positivity and details of multiple test readers will not be available in published reports, as found in the initial iteration of this review. Indeterminate test results will be used to construct 3 × 3 tables of test accuracy and included in calculations of sensitivity and specificity. Index and reference tests will be grouped by exact methods.
We will follow the methods recommended by the Cochrane Collaboration Handbook for Diagnostic Test Accuracy Reviews . Where we judge that studies are sufficiently similar in design, setting, patient and sample characteristics, and test conduct, we will plot their sensitivity and specificity estimates in ROC space and conduct meta-analyses using a bivariate random-effects model to generate and compare summary estimates of sensitivity and specificity.
If we carry out a meta-analysis, we will use a meta-regression to examine heterogeneity in relation to study setting, patient and sample characteristics, and test conduct. We will carry out a sensitivity analysis to examine any assumptions that combining studies with different study settings, patient and sample characteristics, and test conduct was justified. We will use hierarchical logistic regression to compare estimates of sensitivity and specificity for different tests. We will calculate correlation coefficients to test relatedness of sensitivity and specificity.
Where studies are heterogeneous in design, setting, patient and sample characteristics, or test conduct, we will not carry out a meta-analysis and will synthesise the data through narrative, following guidance from Popay et al. .
We will not carry out a formal assessment of publication bias of studies included in this review; techniques such as funnel plots and regression tests have not been found to be useful for identifying publication bias of diagnostic test accuracy studies .
Confidence in cumulative estimate
We will assess the quality of evidence using Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology . We will consider additional domains where appropriate. We will designate quality as high (further research is very unlikely to change our confidence in the estimate of effect), moderate (further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate), low (further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate), or very low (very uncertain about the estimate of effect).
This updated review will help to identify sub-optimal molecular methods for antimalarial marker detection which may be discontinued and identify more sensitive and specific methods which may be adopted. More sensitive and specific detection of drug resistance can be used to improve the breadth and accuracy of surveillance. This would enable the identification of previously undiscovered areas of antimalarial resistances and susceptibilities, improve the precision of estimates of the prevalence of resistances, and improve our ability to detect smaller changes in these patterns. Higher-quality evidence generated by more accurate and detailed surveillance can be used to inform guidelines on the use of antimalarial drugs, leading to better outcomes for more patients.
Plasmodium falciparum chloroquine resistance transporter
Plasmodium falciparum cytochrome b
Plasmodium falciparum dihydrofolate reductase
Plasmodium falciparum dihydropteroate synthase
Plasmodium falciparum kelch 13
Plasmodium falciparum multi-drug resistance transporter 1
Plasmodium falciparum plasmepsin 2
Plasmodium falciparum plasmepsin 3
WHO: World Malaria Report 2017. In World Malaria Report (World Health Organization ed. http://www.who.int/malaria/publications/world-malaria-report-2017/en/: World Health Organization; 2017.
Chiodini PLP, D. Whitty, C J.M. and Lalloo, D.G.: Guidelines for malaria prevention in travellers from the United Kingdom, 2017. England PH ed. www.gov.uk/government/uploads/system/uploads/attachment_data/file/652892/Guidelines_for_malaria_prevention_in_travellers_from_the_UK_2017.pdf: Public Health England; 2017.
WHO. Guidelines for the treatment of malaria. In: Guidelines for the treatment of malaria. 3rd edition. Geneva: World Health Organization; 2015. WHO Guidelines Approved by the Guidelines Review Committee.
Bousema JT, Gouagna LC, Meutstege AM, Okech BE, Akim NI, Githure JI, Beier JC, Sauerwein RW. Treatment failure of pyrimethamine-sulphadoxine and induction of Plasmodium falciparum gametocytaemia in children in western Kenya. Tropical Med Int Health. 2003;8:427–30.
Korsinczky M, Chen N, Kotecka B, Saul A, Rieckmann K, Cheng Q. Mutations in Plasmodium falciparum cytochrome b that are associated with atovaquone resistance are located at a putative drug-binding site. Antimicrob Agents Chemother. 2000;44:2100–8.
Nosten F, van Vugt M, Price R, Luxemburger C, Thway KL, Brockman A, McGready R, ter Kuile F, Looareesuwan S, White NJ. Effects of artesunate-mefloquine combination on incidence of Plasmodium falciparum malaria and mefloquine resistance in western Thailand: a prospective study. Lancet. 2000;356:297–302.
Trape JF, Pison G, Preziosi MP, Enel C, Desgrees du Lou A, Delaunay V, Samb B, Lagarde E, Molez JF, Simondon F. Impact of chloroquine resistance on malaria mortality. C R Acad Sci III. 1998;321:689–97.
Price RN, von Seidlein L, Valecha N, Nosten F, Baird JK, White NJ. Global extent of chloroquine-resistant Plasmodium vivax: a systematic review and meta-analysis. Lancet Infect Dis. 2014;14:982–91.
Davlantes E, Dimbu PR, Ferreira CM, Florinda Joao M, Pode D, Felix J, Sanhangala E, Andrade BN, Dos Santos Souza S, Talundzic E, et al. Efficacy and safety of artemether-lumefantrine, artesunate-amodiaquine, and dihydroartemisinin-piperaquine for the treatment of uncomplicated Plasmodium falciparum malaria in three provinces in Angola, 2017. Malar J. 2018;17:144.
Thanh NV, Thuy-Nhien N, Tuyen NT, Tong NT, Nha-Ca NT, Dong LT, Quang HH, Farrar J, Thwaites G, White NJ, et al. Rapid decline in the susceptibility of Plasmodium falciparum to dihydroartemisinin-piperaquine in the south of Vietnam. Malar J. 2017;16:27.
Ashley EA, Dhorda M, Fairhurst RM, Amaratunga C, Lim P, Suon S, Sreng S, Anderson JM, Mao S, Sam B, et al. Spread of artemisinin resistance in Plasmodium falciparum malaria. N Engl J Med. 2014;371:411–23.
Ecker AL, Fidock MA, D A. Molecular markers of Plasmodium resistance to antimalarials. In: Staines HMK S, editor. Treatment and prevention of malaria. Switzerland: Springer; 2012. https://link.springer.com/chapter/10.1007/978-3-0346-0480-2_13
Bopp S, Magistrado P, Wong W, Schaffner SF, Mukherjee A, Lim P, Dhorda M, Amaratunga C, Woodrow CJ, Ashley EA, et al. Plasmepsin II-III copy number accounts for bimodal piperaquine resistance among Cambodian Plasmodium falciparum. Nat Commun. 2018;9:1769.
WHO: Global report on antimalarial drug efficacy and drug resistance: 2000–2010. (World Health Organization ed. http://www.who.int/malaria/publications/atoz/9789241500470/en/: World Health Organization; 2010.
WHO: Mass drug administration, mass screening and treatment and focal screening and treatment for malaria. In WHO Evidence Review Group meeting report World Health Organisation ed. http://www.who.int/malaria/mpac/mpac-sept2015-erg-mda-report.pdf: World Health Organisation; 2015.
WHO: Eliminating malaria in the Greater Mekong Subregion: united to end a deadly disease. In Malaria (Organisation WH ed. http://www.who.int/malaria/publications/atoz/eliminating-malaria-greater-mekong/en/: World Health Organisation; 2016.
Klein EY. Antimalarial drug resistance: a review of the biology and strategies to delay emergence and spread. Int J Antimicrob Agents. 2013;41:311–7.
Antimalarial Drug Efficacy Maps.
WHO. Chemotherapy of malaria and resistance to antimalarials. Report of a WHO scientific group. World Health Organ Tech Rep Ser. 1973;529:1–121.
Plowe CV. Monitoring antimalarial drug resistance: making the most of the tools at hand. J Exp Biol. 2003;206:3745–52.
Nsanzabana C, Djalle D, Guerin PJ, Menard D, Gonzalez IJ. Tools for surveillance of anti-malarial drug resistance: an assessment of the current landscape. Malar J. 2018;17:75.
Venkatesan M, Gadalla NB, Stepniewska K, Dahal P, Nsanzabana C, Moriera C, Price RN, Martensson A, Rosenthal PJ, Dorsey G, et al. Polymorphisms in Plasmodium falciparum chloroquine resistance transporter and multidrug resistance 1 genes: parasite risk factors that affect treatment outcomes for P. falciparum malaria after artemether-lumefantrine and artesunate-amodiaquine. Am J Trop Med Hyg. 2014;91:833–43.
Picot S, Olliaro P, de Monbrison F, Bienvenu AL, Price RN, Ringwald P. A systematic review and meta-analysis of evidence for correlation between molecular markers of parasite resistance and treatment outcome in falciparum malaria. Malar J. 2009;8:89.
Nsanzabana C, Ariey F, Beck HP, Ding XC, Kamau E, Krishna S, Legrand E, Lucchi N, Miotto O, Nag S, et al. Molecular assays for antimalarial drug resistance surveillance: a target product profile. PLoS One. 2018;13:e0204347.
Sarmah NP, Sarma K, Bhattacharyya DR, Sultan AA, Bansal D, Singh N, Bharti PK, Sehgal R, Mohapatra PK, Parida P, Mahanta J. Antifolate drug resistance: novel mutations and haplotype distribution in dhps and dhfr from Northeast India. J Biosci. 2017;42:531–5.
Mishra S, Bharti PK, Shukla MM, Ali NA, Kashyotia SS, Kumar A, Dhariwal AC, Singh N. Clinical and molecular monitoring of Plasmodium falciparum resistance to antimalarial drug (artesunate+sulphadoxine-pyrimethamine) in two highly malarious district of Madhya Pradesh, Central India from 2012-2014. Pathog Glob Health. 2017;111:186–94.
Burrow R: Sensitivity and specificity of molecular methods for detecting markers of antimalarial drug resistance in clinical samples of Plasmodium falciparum: a systematic review. In Department of Continuing Education: University of Oxford; 2017.
McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, the P-DTAG, Clifford T, Cohen JF, Deeks JJ, Gatsonis C, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA. 2018;319:388–96.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, Group P-P. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, Gotzsche PC, Lasserson T, Tovey D, Group PfA. PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419.
Handbook for diagnostic test accuracy reviews.
CRD. Systematic reviews: CRD’s guidance for undertaking reviews in health care. [3rd ed]. York: Centre for Reviews and Dissemination; 2009.
Higgins J, Green S. Cochrane handbook for systematic reviews of interventions. Oxford: Wiley-Blackwell; 2008.
Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM, Group Q. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–36.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HC, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527.
Popay JR H., Sowden, A., Petticrew, M., Britten, N., Arai, L.: Guidance of the conduct of narrative synthesis in systematic reviews. (Programme ERM ed. https://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications.php: Lancaster University; 2006.
Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58:882–93.
Schunemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, Scholten R, Langendam M, Leeflang MM, Akl EA, et al. GRADE guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. J Clin Epidemiol. 2016;76:89–98.
Thank you to Nia Roberts who advised on the development of a difficult search strategy for the first iteration of this review, and to Chris Butler who advised on the development of the protocol for the first iteration of this review.
All protocol amendments will be described, dated, and justified in the published systematic review and in the registration in PROSPERO.
No funding or sponsorship has been received for this systematic review.
Availability of data and materials
The datasets used and/or analysed during the current study are available from Open Science Framework at https://osf.io/tj93v/.
Ethics approval and consent to participate
Consent for publication
RB is an author of a report of an eligible study. TF and GH declare that they have no conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Burrow, R., Fanshawe, T.R. & Humphreys, G.S. Diagnostic accuracy of molecular methods for detecting markers of antimalarial drug resistance in clinical samples of Plasmodium falciparum: protocol for an update to a systematic review and meta-analysis. Syst Rev 7, 221 (2018). https://doi.org/10.1186/s13643-018-0891-6