- Open Access
- Open Peer Review
Study-based registers reduce waste in systematic reviewing: discussion and case report
Systematic Reviewsvolume 8, Article number: 129 (2019)
Maintained study-based registers (SBRs) have, at their core, study records linked to, potentially, multiple other records such as references, data sets, standard texts and full-text reports. Such registers can minimise and refine searching, de-duplicating, screening and acquisition of full texts. SBRs can facilitate new review titles/updates and, within seconds, inform the team about the potential workload of each task.
We discuss the advantages/disadvantages of SBRs and report a case of how such a register was used to develop a successful grant application and deliver results—reducing considerable redundancy of effort.
SBRs saved time in question-setting and scoping and made rapid production of nine Cochrane systematic reviews possible.
Whilst helping prioritise and conduct systematic reviews, SBRs improve quality. Those funding information specialists for literature reviewing could reasonably stipulate the resulting SBR to be delivered for dissemination and use beyond the life of the project.
Time to complete systematic reviews
There is much redundancy in medical research [1,2,3,4,5,6] and systematic reviewing is no exception [7,8,9,10,11,12,13,14,15,16,17,18]. Usually, the review team runs searches, removes duplicates, screens titles and abstracts, obtains full-text reports, screens full texts, assembles reports of the same study, extracts data, synthesises them and writes the final report. This process has great potential for waste [19,20,21]. For systematic reviews, the median time from search to publication has improved from 14 months in 2008  to 8 in 2013  (mean time to complete was 17 months ; median time between first search and appearance of the review in PubMed was nearly 2 years ). The Cochrane Collaboration, a large organisation undertaking and maintaining systematic reviews of health care, largely works with volunteer health care professionals [26, 27] and the median time from Cochrane protocol to review publication was 2.4 years . Keeping volunteer authors active on the review and the actual length of the review process are two major challenges to swift reviewing . Efficiencies are needed.
Current preparation for reviewing
At the start of a new systematic review or an update for an existing systematic review, there is limited knowledge about the quantity of relevant literature. Although estimation of workload is possible through piloting or scoping searches [30,31,32], this requires time and the exact number of relevant studies may remain unclear. This lack of clarity leaves assembled review teams vulnerable. The predicted investment of effort could be
Overestimated—and eventually review teams have no or very few studies for their new review or update—with the waste this would incur.
Underestimated—and the team is eventually surprised and, perhaps, overwhelmed with many relevant studies, with the risk of
◦ Publishing a protocol but finding completion of the review unaffordable or impossible with the resulting wasteful unfinished or empty review.
◦ Requesting extensions to funding; and/or
◦ Running into delays that may render the final work being immediately out of date.
Accurately estimated—but what remains unclear is as to whether the investment needed to review/update is warranted by any potential to change what is already known.
Waste in systematic reviewing and information supply
The majority of the literature related to waste in systematic review are either focused on methodology [7,8,9,10,11,12,13,14,15,16,17,18] or automation of processes to shorten time-consuming tasks [33,34,35,36]. For over two decades, information specialists have given practical guidance for waste reduction in systematic reviews [32, 37,38,39,40,41]. Information specialists in the Cochrane Collaboration maintain specialised registers to support Cochrane reviews. Some of these registers are highly developed and shorten the systematic review process .
Study-based registers (SBRs) are databases in which all records of same study are linked to one ‘parent’ report. This study report may contain meta-data extracted from the various ‘child’ records of that same study. Often building a SBR involves an information specialist running searches across major bibliographic databases, de-duplicating, screening for eligibility, and obtaining full text of records. Then, there is the process of linking ‘child’ reports to the parent study record, extracting, cleaning and curating meta-data and maintaining the register with updates. In the case of randomised trials, meta-data for the study may be gleaned from the individual records (e.g. details of participants, interventions, controls and outcomes (PICO)) or, working from the other direction, from the overarching review in which the study has been used (e.g. qualitative or quantitative data incorporated within the review relating to that study). Details of creating and maintaining a SBR has been reported elsewhere .
Aims and objectives
To describe how a SBR can be used to almost eliminate certain arduous steps in prospective systematic reviewing. We will illustrate how these steps can be accomplished in a matter of minutes or seconds and how this approach almost negates the early, inhibiting, and, we argue, wasteful, effort experienced by systematic reviewers. Although some benefits of SBRs have already been reported [42, 43], little has been presented on how SBRs can reduce waste whilst assisting prioritisation of systematic review work .
‘Living’ study-based registers
With a well-maintained SBR, an information specialist can provide the following data in a matter of minutes (stipulation of all estimates are review-specific but worked example follows):
Exact number of
◦ Studies/related records in a field (e.g. schizophrenia, tardive dyskinesia);
◦ Studies/related records relevant to a new title or update (e.g. vitamin E for people with tardive dyskinesia);
◦ Studies/related records relevant to a class of interventions (e.g. calcium channel blockers);
◦ Studies that have/have not already been data-extracted, and the extracted data were available;
◦ Existing related reviews on a topic—and quantification of studies/related records within each review;
◦ Comparisons possible to accurately scope existing relevant evidence on a given topic—and quantification of studies/related records within each comparison ;
◦ New studies, records to known studies and novel relevant treatments;
◦ Research gaps in topic areas devoid of/with a dearth of evidence;
For the studies
◦ Concatenated importable references, the output of each study or all the relevant studies;
◦ Full reports of each studies collected into a study folder;
◦ Completed data extraction forms of studies where available.
Essentially, a SBR should be ‘living’. These living curated registers involve minimal analyses and are maintained by an information specialist (Table 1). Such registers have been produced by the Cochrane Dementia (ALIOS), Renal/Kidney (maintained within MeerKat), Pregnancy and Childbirth, Stroke (DORIS) and Schizophrenia (within MeerKat) teams for over two decades. For some existing SBRs, there is further developments to add functions to include extracted data from reviews , links to standard text and to prioritise sharing these data publicly . Unfortunately, CENTRAL and Cochrane Register of Studies (CRS) are, at best, rudimentary SBRs at the time of revising this paper (27 March 2019).
Armed with the information from these sophisticated registers a potential review team should be able to present a much more accurate estimate of workload before embarking on the grant application or the actual review or update. These registers should make it possible to truncate the period immediately after protocol publication, seeding the systematic review with extracted data and preparing for swift meta-analysis.
A case report from schizophrenia
Cochrane Schizophrenia has maintained a SBR of randomised trials for over two decades . Routine searching identifies records that, with some help from automation, are merged into study reports (examples of studies with 10, 50 or even 100 records are not rare) helping minimise the risk of multiple counting with the systematic review. Meta-data (including number randomised) are part of the study record. Although increasingly automated, this process is facilitated by the group’s information specialist (FS). Since search strategies have been saved in bibliographic databases, monthly automatic updates are received through email. Then the information specialist spends three days per month for routine processes of updating the register: (1) 1 day for primary screening of search results and adding references to the register; (2) another day for obtaining full texts and linking them to their references; and finally (3) one last day for indexing the PICO meta-data from each full text and then assembling the separate references of the same study and linking them to that study. This register supports 324 maintained systematic reviews (17 May 2019).
Estimates of costs for the grant application
In applying for NIHR UK Health Technology Assessment (HTA) Project Grant (14/27/02) [49,50,51], a call for reviews relevant to treating people with Tardive Dyskinesia (a problematic adverse effect of antipsychotic drugs) use of the SBR gave a clear advantage. Cochrane Schizophrenia’s information specialist ran a highly specific, highly sensitive search (16 July 2015) in the SBR and identified the exact number of studies relevant to the problem (time spent on task 8 s). This number helped the grant application team provide an accurate assessment of the work to be done—and realistic estimates of costs.
Prediction of best composition of families of reviews
Tardive Dyskinesia is a condition for which many treatments have been used . Arguments exist for ‘lumping and splitting’ at all sorts of levels. At the broadest level of ‘lumping’, the overview could encompass all treatments but this becomes unwieldy and impossible to update. At the finest level of ‘splitting’ each individual comparison of each treatment could be treated as a separate review. Even in a limited topic area such as Tardive Dyskinesia, this would lead to hundreds of separate reviews. Clearly there is a balance to be struck. By use of a controlled vocabulary for the meta-data within the SBR auto-grouping into logical treatment/comparison families for reviews can take place—and, once established, this can take place instantly. This ensures a pragmatic middle road dividing work into clinically logical bite-size reviews for later overviewing if required. Also, the classification of interventions within the register allows reviewing a class of interventions in a review. In the case of Tardive Dyskinesia, 10 separate review groupings were created (Table 2) (time spent on task 2 min and 10 s). This also helped the grant application team provide an accurate assessment of the output the funders could expect.
Prediction of effort needed at data extraction step and saving effort for others
In this particular case, the SBR also contains information on already extracted data. Therefore, the applicants were also informed of exactly how much work has been completed and allowed them to make accurate costing for the necessary remaining efforts (time spent on the task 8 s). Working with such a register affords applicants opportunities to ensure that their request for funding for this part of the effort can be seen as an investment. The extracted study data can, thereafter, be made available to anyone thus reducing future duplication of effort (see below).
Supply of documents
SBR systems such as Microsoft Access ‘MeerKat’ [64,65,66] have capacity to output file batches grouped by review, sub-grouped into relevant study files, in turn containing all relevant records and references (time spent on the task 4 min and 43 s). In this cause, this allowed those applying for the grant to reassure funders that supply of documents was not an issue and, once the grant was given, to waste no time in acquiring papers and piecing together the studies from ‘salami’ or multiple publications of same study.
Future supply of full dataset
In the hope of evolving SBR towards making the level of document supply described above redundant and saving more time in the future—applicants sought and were granted support to extract all data from all randomised studies relevant to Tardive Dyskinesia and to make these data publicly available. This included each part of the data being made traceable to the exact site within the source record . Any new updates of this will involve supply of documents containing tabulated, reliably and verifiably extracted data .
Cochrane recommends biennial update for reviews  but this timing is not always appropriate. Excessive updating wastes resource while inadequate updating could result in outdated or incomplete evidence being used . While there are methods to detect if updating a review could change the current conclusion/practice, almost all require an awareness of the available ‘unused’ relevant literature [48, 70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99], and some degree of screening and data checking to allow an informed decision. Within a well-constructed and maintained study register, this investment has already been made.
As the grant  was drawing to a close and the reviews were being completed. On the 26 April 2017, the SBR allowed the information specialist to run a final ‘just-before-submission’ update search limiting to not-already-identified records (time spent on the task 13 s). Just before publication, this search was used to inform the team that seven of the 10 reviews were fully current but two needed to be updated with a total of five new studies. This allowed the grant holders efficiently update the reviews just pre-publication to ensure they held fully current information.
This search also identified two new drugs (Valbenazine and Deutetrabenazine) entering the market specifically for treatment of people with Tardive Dyskinesia. These new compounds, unrelated to others, necessitate a new review outside of what was supported by the grant . Unlike the decades ago when SBRs did not exist or were not sophisticated, it is now almost impossible to fail to identify a newly emerging treatment. This saves further waste in systematic reviews through inclusiveness of all treatments from all classes.
Feasibility of study-based registers
Although it seems exciting to start a systematic review with the extraction of data, the workload creating a SBR should not be underestimated. The investment of time is a frequent concern. Is it possible for all evidence-synthesis groups to maintain a SBR and what are the necessary requirements in creating such a register?
The short answer is that every systematic review is, in itself, a small SBR. Frequently at completion of any given review, these small registers (reviews) are rendered unusable to others or disassembled necessitating the next interested group of reviewers to have to repeat the construction. This is avoidable waste when collating all the data within a related group of reviews constitutes the embryonic SBR.
In Table 3, we itemise the time and resource required for establishing and maintaining our broad-based schizophrenia SBR.
Small SBRs, in the form of competed reviews, are increasingly prevalent. We maintain that there is a strong argument for creation of broad-based healthcare study-based registers linked to records containing data, text and other relevant information. Not to use already compiled data is wasteful and not to invest to create the SBR is passing cost—and waste—down the line to reviewers. Information specialist investment is already happening—repeatedly. We argue that focus and direction of this investment would avoid the ongoing unnecessary multiplication of effort .
We reported one example of the potential of SBRs for grant application. This is one amongst many. The living property of this register allowed the information specialist with his/her more sophisticated role—to become an integral—and useful—part of the review team.
Finally, the SBR promoted more sophisticated sharing of data from this project facilitating the not-so-distant full automation of living systematic reviews.
Health Technology Assessment
Participants/Problem/Population, Interventions, Control/Comparison, and Outcomes
Research information systems
The centre for systematic reviews on health policy and systems research
Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65. https://doi.org/10.1016/s0140-6736(13)62229-1.
Chan A-W, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66. https://doi.org/10.1016/s0140-6736(13)62296-5.
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. https://doi.org/10.1016/s0140-6736(13)62228-x.
Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75. https://doi.org/10.1016/s0140-6736(13)62227-8.
Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JPA, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4. https://doi.org/10.1016/s0140-6736(13)62329-6.
Salman RA-S, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85. https://doi.org/10.1016/s0140-6736(13)62297-7.
Afshari A, Wetterslev J, Smith AF. Can systematic reviews with sparse data be trusted? Anaesthesia. 2017;72(1):12–16. PMID: 27804113. https://doi.org/10.1111/anae.13730.
Annane D, Jaeschke R, Guyatt G. Are systematic reviews and meta-analyses still useful research? Yes Intensive Care Med 2018;44(4):512–514. PMID: 29663046. https://doi.org/10.1007/s00134-018-5102-3.
Chalmers I, Glasziou P. Systematic reviews and research waste. Lancet. 2016;387(10014):122–123. PMID: 26841991. https://doi.org/10.1016/s0140-6736(15)01353-7
Chevret S, Ferguson ND, Bellomo R. Are systematic reviews and meta-analyses still useful research? No Intensive Care Med. 2018;44(4):515–517. PMID: 29663047https://doi.org/10.1007/s00134-018-5066-3.
Garattini S, Jakobsen JC, Wetterslev J, Bertele V, Banzi R, Rath A, et al. Evidence-based clinical practice: overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016;32:13–21. PMID: 27160381. https://doi.org/10.1016/j.ejim.2016.03.020.
Handoll HH, Langhorne P. In defence of reviews of small trials: underpinning the generation of evidence to inform practice. Cochrane Database Syst Rev. 2015;11(11):ED000106. PMID: 26574167. https://doi.org/10.1002/14651858.ED000106.
Lund H, Juhl C, Christensen R. Systematic reviews and research waste. Lancet (London, England). 2016;387(10014):123–124 PMID: 26841992. https://doi.org/10.1016/s0140-6736(15)01354-9
Moller MH, Ioannidis JPA, Darmon M. Are systematic reviews and meta-analyses still useful research? We are not sure. Intensive Care. 2018;44(4):518–20. PMID: 29663048. https://doi.org/10.1007/s00134-017-5039-y.
Nelson RL. Systematic reviews and research waste. Lancet (London, England). 2016;387(10014):124. PMID: 26841994. https://doi.org/10.1016/s0140-6736(15)01355-0.
Roberts I, Ker K. How systematic reviews cause research waste. Lancet. 2015;386(10003):1536. PMID: 26530621. https://doi.org/10.1016/s0140-6736(15)00489-4.
Roberts I, Ker K. Cochrane: the unfinished symphony of research synthesis. Syst Rev. 2016;5(1):115. PMID: 27416925. https://doi.org/10.1186/s13643-016-0290-9.
Roberts I, Ker K, Edwards P, Beecher D, Manno D, Sydenham E. The knowledge system underpinning healthcare is not fit for purpose and must change. BMJ. 2015;350:h2463. PMID: 26041754. https://doi.org/10.1136/bmj.h2463.
Andrade R, Pereira R, Weir A, Ardern CL, Espregueira-Mendes J. Zombie reviews taking over the PROSPERO systematic review registry. It's time to fight back! Br J Sports Med. 2017. PMID: 29021246. https://doi.org/10.1136/bjsports-2017-098252.
Rathbone J, Carter M, Hoffmann T, Glasziou P. Solving research waste with better duplicate detection. Eur J Pub Health. 2015;25(Suppl 3):14–5. https://doi.org/10.1093/eurpub/ckv167.023.
Tovey D. Systematic Reviews can both prevent and create research waste: what is Cochrane doing now and how can it do better in the future? Edinburgh: REWARD/EQUATOR; 2015.
Sampson M, Shojania KG, Garritty C, Horsley T, Ocampo M, Moher D. Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008;61(6):531–6. PMID: 18471656. https://doi.org/10.1016/j.jclinepi.2008.02.004.
Beller EM, Chen JK, Wang UL, Glasziou PP. Are systematic reviews up-to-date at the time of publication? Syst Rev. 2013;2:36. PMID: 23714302. https://doi.org/10.1186/2046-4053-2-36.
Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545. PMID: 28242767. https://doi.org/10.1136/bmjopen-2016-012545.
Bramer W, Bain P. Updating search strategies for systematic reviews using EndNote. J Med Libr Assoc. 2017;105(3):285–9. PMID: 28670219. https://doi.org/10.5195/jmla.2017.183.
Tharyan P. Evidence of the people, by the people, and for the people. Cochrane Database Syst Rev. 2010;2011:ED000013 PMID: 21833936 DOI: https://doi.org/10.1002/14651858.ED000013
Wilson M. Help us improve the health of people everywhere. Cochrane collaboration. 2019. https://www.cochrane.org/join-cochrane.
Tricco AC, Brehaut J, Chen MH, Moher D. Following 411 Cochrane protocols to completion: a retrospective cohort study. PLoS One. 2008;3(11):e3684. PMID: 18997866. https://doi.org/10.1371/journal.pone.0003684.
Turner T, Green S, Tovey D, McDonald S, Soares-Weiser K, Pestridge C, et al. Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement. Syst Rev. 2017;6(1):147. PMID: 28760162.https://doi.org/10.1186/s13643-017-0542-3.
Clavisi O, Bragge P, Tavender E, Turner T, Gruen RL. Effective stakeholder participation in setting research priorities using a Global Evidence Mapping approach. J Clin Epidemiol. 2013 66(5):496–502.e2. PMID: 22819249. https://doi.org/10.1016/j.jclinepi.2012.04.002.
Hoekstra D, Mutsch M, Kien C, Gerhardus A, Lhachimi SK. Identifying and prioritising systematic review topics with public health stakeholders: a protocol for a modified Delphi study in Switzerland to inform future research agendas. BMJ Open. 2017;7(8):e015500. PMID: 28780546. https://doi.org/10.1136/bmjopen-2016-015500.
Otter MLE, Wright JM, King NV. Developing the Librarians' role in supporting grant applications and reducing waste in research: outcomes from a literature review and survey in the NIHR research design service. New Rev Acad Librariansh. 2017;23(2–3):258–74. https://doi.org/10.1080/13614533.2017.1330219.
Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015;4:78. PMID: 26073888. https://doi.org/10.1186/s13643-015-0066-7.
O'Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS. Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Rev. 2018;7(1):3. PMID: 29316980. https://doi.org/10.1186/s13643-017-0667-4.
Tsafnat G, Dunn A, Glasziou P, Coiera E. The automation of systematic reviews. BMJ. 2013;346:f139. PMID: 23305843. https://doi.org/10.1136/bmj.f139
Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev. 2014;3:74. PMID: 25005128. https://doi.org/10.1186/2046-4053-3-74.
Kirtley S. Can librarians contribute to increasing value and reducing waste in medical research? EQUATOR Network: Enhancing the Quality and Transparency of Health Research 2014.
Kirtley S. Improving the reliability and value of medical research literature: a new role for librarians in promoting good research reporting practices. Rome: 14th EAHIL; 2014. p. 11-3.
Kirtley S. Increasing value and reducing waste in biomedical research: librarians are listening and are part of the answer. Lancet. 2016;387(10028):1601. PMID: 27116055.https://doi.org/10.1016/s0140-6736(16)30241-0.
Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, et al. Towards evidence based research. BMJ. 2016;355:i5440. PMID: 27797786. https://doi.org/10.1136/bmj.i5440.
Moher D, Shamseer L, Cobey KD, Lalu MM, Galipeau J, Avey MT, et al. Stop this waste of people, animals and money. Nature. 2017;549(7670):23–25. PMID: 28880300. https://doi.org/10.1038/549023a.
Shokraneh F, Adams CE. Study-based registers of randomized controlled trials: starting a systematic review with data extraction or meta-analysis. BioImpacts. 2017;7(4):209–217. PMID: 29435428. https://doi.org/10.15171/bi.2017.25.
Shokraneh F, Adams CE. Potentials of registers of randomized controlled trials in automating the systematic reviews. Edinburgh: REWARD/EQUATOR; 2015. https://doi.org/10.13140/RG.2.2.19853.84963.
Shokraneh F, Adams CE. How do the study-based registers reduce waste in systematic reviewing? Edinburgh: 25th Cochrane Colloquium; 2018. https://doi.org/10.13140/RG.2.2.16401.28004.
Shokraneh F, Adams CE. A simple formula for enumerating comparisons in trials and network meta-analysis. F1000Research. 2019;8:38. PMID: 30863537. https://doi.org/10.12688/f1000research.17352.2.
Shokraneh F, Adams CE, Clarke M, Amato L, Bastian H, Beller E, et al. Why Cochrane should prioritise sharing data. BMJ. 2018;362:k3229. PMID: 30061322. https://doi.org/10.1136/bmj.k3229.
Shokraneh F. Register of trials. Cochrane Schizophrenia Group 2018;http://schizophrenia.cochrane.org/register-trials
Akl EA, Fadlallah R, Ghandour L, Kdouh O, Langlois E, Lavis JN, et al. The SPARK tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation. Health Res Policy Syst. 2017;15(1):77. PMID: 28870215. https://doi.org/10.1186/s12961-017-0242-4.
Adams CE, Walker DM, Gray B, Soares-Weiser K. HTA - 14/27/02: a systematic review and network meta-analysis of the safety and clinical effectiveness of interventions for treating or preventing deterioration of symptoms of antipsychotic-induced tardive dyskinesia (TD). 2015. https://www.fundingawards.nihr.ac.uk/award/14/27/02.
Adams CE, Walker DM, Gray B, Soares-Weiser K, Bergman H, Zhao S, et al. Appendix: traceable extracted data from included studies of tardive dyskinesia reviews. 2017. https://doi.org/10.13140/RG.2.2.28907.95529.
Bergman H, Walker DM, Nikolakopoulou A, Soares-Weiser K, Adams CE. Systematic review of interventions for treating or preventing antipsychotic-induced tardive dyskinesia. Health Technol Assess. 2017;21(43):1–218. PMID: 28812541. https://doi.org/10.3310/hta21430.
Soares K, McGrath J, Adams C. Evidence and tardive dyskinesia. Lancet. 1996;347(9016):1696–7. https://doi.org/10.1016/s0140-6736(96)91525-1.
Bergman H, Soares-Weiser K. Anticholinergic medication for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000204. PMID: 29341071. https://doi.org/10.1002/14651858.CD000204.pub2.
Bergman H, Rathbone J, Agarwal V, Soares-Weiser K. Antipsychotic reduction and/or cessation and antipsychotics as specific treatments for tardive dyskinesia. Cochrane Database Syst Rev. 2018;2:CD000459. PMID: 29409162. https://doi.org/10.1002/14651858.CD000459.pub3.
Bergman H, Bhoopathi PS, Soares-Weiser K. Benzodiazepines for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000205. PMID: 29352477. https://doi.org/10.1002/14651858.CD000205.pub3.
Essali A, Soares-Weiser K, Bergman H, Adams CE. Calcium channel blockers for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD000206. PMID: 29578611. https://doi.org/10.1002/14651858.CD000206.pub4.
Tammenmaa-Aho I, Asher R, Soares-Weiser K, Bergman H. Cholinergic medication for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD000207. PMID: 29553158. https://doi.org/10.1002/14651858.CD000207.pub2.
Alabed S, Latifeh Y, Mohammad HA, Bergman H. Gamma-aminobutyric acid agonists for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;4:CD000203. PMID: 29663328. https://doi.org/10.1002/14651858.CD000203.pub4.
Soares-Weiser K, Rathbone J, Ogawa Y, Shinohara K, Bergman H. Miscellaneous treatments for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD000208. PMID: 29552749. https://doi.org/10.1002/14651858.CD000208.pub2.
El-Sayeh HG, Rathbone J, Soares-Weiser K, Bergman H. Non-antipsychotic catecholaminergic drugs for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000458. PMID: 29342497. https://doi.org/10.1002/14651858.CD000458.pub3.
Adelufosi AO, Abayomi O, Ojo TM. Pyridoxal 5 phosphate for neuroleptic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2015;4:CD010501. PMID: 25866243. https://doi.org/10.1002/14651858.CD010501.pub2.
Karl B, Bergman H, Abd El Sayed S, Adams CE. Vesicular monoamine transporter inhibitors versus placebo for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD012986. https://doi.org/10.1002/14651858.cd012986.
Soares-Weiser K, Maayan N, Bergman H. Vitamin E for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000209. PMID: 29341067. https://doi.org/10.1002/14651858.CD000209.pub3.
Kaur S, de Souza M, Fenton M, Adams CE. MeerKat 1.4 – and beyond. Cochrane Schizophrenia Group; 2009.
MeerKat Working G, Editor MeerKat: getting the most out of your study-based specialized register. Melbourne: Cochrane Colloquium; 2005.
Wright J. MeerKat user guide for MeerKat version 1.4. Grant T, editor. Leeds: Cochrane Schizophrenia Group; 2006.
Shokraneh F, Adams CE. Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms. Syst Rev. 2017;6(1):153. PMID: 28778216. https://doi.org/10.1186/s13643-017-0546-z.
Higgins JPT, Green S, Scholten R. Chapter 3: maintaining reviews: updates, amendments and feedback. Cochrane handbook for systematic reviews of interventions version 510 [updated March 2011]: The Cochrane Collaboration; 2011.
Jaidee W, Moher D, Laopaiboon M. Time to update and quantitative changes in the results of cochrane pregnancy and childbirth reviews. PLoS One. 2010;5(7):e11553 PMID: 20644625. https://doi.org/10.1371/journal.pone.0011553.
Chalmers I, Haynes B. Reporting, updating, and correcting systematic reviews of the effects of health care. BMJ. 1994;309(6958):862–5. PMID: 7950620. https://doi.org/10.1136/bmj.309.6958.862.
Barrowman NJ, Fang M, Sampson M, Moher D. Identifying null meta-analyses that are ripe for updating. BMC Med Res Methodol. 2003;3:13. PMID: 12877755. https://doi.org/10.1186/1471-2288-3-13.
Waters E, Doyle J, Jackson N. Evidence-based public health: improving the relevance of Cochrane Collaboration systematic reviews to global public health priorities. J Public Health Med. 2003;25(3):263–6 PMID: 14575208.
Doyle J, Waters E, Yach D, McQueen D, De Francisco A, Stewart T, et al. Global priority setting for Cochrane systematic reviews of health promotion and public health research. J Epidemiol Community Health. 2005;59(3):193–7. PMID: 15709077. https://doi.org/10.1136/jech.2003.019547.
Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. A systematic review identified few methods and strategies describing when and how to update systematic reviews. J Clin Epidemiol. 2007;60(11):1095–104. PMID: 17938050. https://doi.org/10.1016/j.jclinepi.2007.03.008.
Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007;147(4):224. https://doi.org/10.7326/0003-4819-147-4-200708210-00179.
Cohen AM. Optimizing feature representation for automated systematic review work prioritization. AMIA Annu Symp Proc. 2008;6:121–5 PMID: 18998798.
Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. When and how to update systematic reviews. Cochrane Database Syst Rev. 2008(1):Mr000023 PMID: 18254126 DOI: https://doi.org/10.1002/14651858.MR000023.pub3
Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE, et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol. 2008;61(8):755–762. PMID: 18586179. https://doi.org/10.1016/j.jclinepi.2007.10.003.
Cohen AM, Ambert K, McDonagh M. Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc. 2009;16(5):690–704. PMID: 19567792. https://doi.org/10.1197/jamia.M3162.
Sampson MJ. Updating searches for systematic reviews: Aberystwyth University; 2009.
Sutton AJ, Donegan S, Takwoingi Y, Garner P, Gamble C, Donald A. An encouraging assessment of methods to inform priorities for updating systematic reviews. J Clin Epidemiol. 2009;62(3):241–51. PMID: 18783919. https://doi.org/10.1016/j.jclinepi.2008.04.005.
Garritty C, Tsertsvadze A, Tricco AC, Sampson M, Moher D. Updating systematic reviews: an international survey. PLoS One. 2010;5(4):e9914. PMID: 20376338. https://doi.org/10.1371/journal.pone.0009914.
Bastian H, Scheibler F, Knelangen M, Zschorlich B, Nasser M, Waltering A. Choosing health technology assessment and systematic review topics: the development of priority-setting criteria for patients' and consumers' interests. Int J Technol Assess Health Care. 2011;27(4):348–56. PMID: 22004776. https://doi.org/10.1017/s0266462311000547.
Meremikwu M, Udoh E, Nwagbara B, Effa E, Oringanje C, Edet B, et al. Priority setting for systematic review of health care interventions in Nigeria. Health Pol. 2011;99(3):244–9. PMID: 21131092. https://doi.org/10.1016/j.healthpol.2010.11.004.
Chung M, Newberry SJ, Ansari MT, Yu WW, Wu H, Lee J, et al. Two methods provide similar signals for the need to update systematic reviews. J Clin Epidemiol. 2012;65(6):660–8. PMID: 22464414. https://doi.org/10.1016/j.jclinepi.2011.12.004.
Cohen AM, Ambert K, McDonagh M. Studying the potential impact of automated document classification on scheduling a systematic review update. BMC Med Inform Decis Mak. 2012;12:33. PMID: 22515596. https://doi.org/10.1186/1472-6947-12-33.
Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. AHRQ Methods for effective health care. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012.
Hoomans T, Seidenfeld J, Basu A, Meltzer D. AHRQ methods for effective health care. Systematizing the use of value of information analysis in prioritizing systematic reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012.
Pattanittum P, Laopaiboon M, Moher D, Lumbiganon P, Ngamjarus C. A comparison of statistical methods for identifying out-of-date systematic reviews. PLoS One. 2012;7(11):e48894. PMID: 23185281. https://doi.org/10.1371/journal.pone.0048894.
Mickenautsch S, Yengopal V. SYSTEM research note on: the modified Ottawa method for updating systematic reviews. J Minimum Intervention Dentistry. 2013;6:15–29.
Newberry SJ, Ahmadzai N, Motala A, Tsertsvadze A, Maglione M, Ansari MT, et al. AHRQ methods for effective health care. Surveillance and identification of signals for updating systematic reviews: implementation and early experience. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013.
Takwoingi Y, Hopewell S, Tovey D, Sutton AJ. A multicomponent decision tool for prioritising the updating of systematic reviews. Bmj. 2013;347:f7191. PMID: 24336453. https://doi.org/10.1136/bmj.f7191.
Tugwell P, Knottnerus JA, Idzerda L. Methods for setting priorities in systematic reviews. J Clin Epidemiol. 2013;66(5):467–8. PMID: 23548134. https://doi.org/10.1016/j.jclinepi.2013.02.008.
Shekelle PG, Motala A, Johnsen B. Assessment of a method to detect signals for updating systematic reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014.
Shekelle PG, Motala A, Johnsen B, Newberry SJ. Assessment of a method to detect signals for updating systematic reviews. Syst Rev. 2014;3:13. PMID: 24529068. https://doi.org/10.1186/2046-4053-3-13.
Martinez Garcia L, Sanabria AJ, Araya I, Lawson J, Sola I, Vernooij RW, et al. Efficiency of pragmatic search strategies to update clinical guidelines recommendations. BMC Med Res Methodol. 2015;15:57. PMID: 26227021. https://doi.org/10.1186/s12874-015-0058-2.
Garner P, Hopewell S, Chandler J, MacLehose H, Schunemann HJ, Akl EA, et al. When and how to update systematic reviews: consensus and checklist. BMJ. 2016;354:i3507. PMID: 27443385. https://doi.org/10.1136/bmj.i3507.
Sampson M, de Bruijn B, Urquhart C, Shojania K. Complementary approaches to searching MEDLINE may be sufficient for updating systematic reviews. J Clin Epidemiol. 2016;78:108–15. PMID: 26976054. https://doi.org/10.1016/j.jclinepi.2016.03.004.
Martinez Garcia L, Pardo-Hernandez H, Superchi C, Nino de Guzman E, Ballesteros M, Ibargoyen Roteta N, et al. Methodological systematic review identifies major limitations in prioritization processes for updating. J Clin Epidemiol. 2017;86:11–24. PMID: 28549931. https://doi.org/10.1016/j.jclinepi.2017.05.008.
Shokraneh F. Reproducibility and replicability of systematic reviews. World J Meta-Anal. 2019;7(3):66–71. https://doi.org/10.13105/wjma.v7.i3.66
Shokraneh F. Reducing waste and increasing value through embedded replicability and reproducibility in systematic review process and automation. J Clin Epidemiol. 2019. https://doi.org/10.1016/j.jclinepi.2019.04.008.
This manuscript has been supported by NIHR UK HTA Project Grant (14/27/02) . Cochrane Schizophrenia Group’s Study-Based Register of Trials currently is not receiving any direct funding and it has been developed as a side product by Cochrane Schizophrenia Group.
Availability of data and materials
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests. The views expressed in this paper are the authors’ own views not the views of their affiliated organisations.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.