Skip to main content

Advertisement

  • Methodology
  • Open Access
  • Open Peer Review

Study-based registers reduce waste in systematic reviewing: discussion and case report

Systematic Reviews20198:129

https://doi.org/10.1186/s13643-019-1035-3

  • Received: 3 May 2018
  • Accepted: 1 May 2019
  • Published:
Open Peer Review reports

Abstract

Background

Maintained study-based registers (SBRs) have, at their core, study records linked to, potentially, multiple other records such as references, data sets, standard texts and full-text reports. Such registers can minimise and refine searching, de-duplicating, screening and acquisition of full texts. SBRs can facilitate new review titles/updates and, within seconds, inform the team about the potential workload of each task.

Methods

We discuss the advantages/disadvantages of SBRs and report a case of how such a register was used to develop a successful grant application and deliver results—reducing considerable redundancy of effort.

Results

SBRs saved time in question-setting and scoping and made rapid production of nine Cochrane systematic reviews possible.

Conclusion

Whilst helping prioritise and conduct systematic reviews, SBRs improve quality. Those funding information specialists for literature reviewing could reasonably stipulate the resulting SBR to be delivered for dissemination and use beyond the life of the project.

Keywords

  • Study-based registers
  • Grant application
  • Systematic reviews
  • Research prioritisation
  • Reducing waste
  • Increasing value

Background

Time to complete systematic reviews

There is much redundancy in medical research [16] and systematic reviewing is no exception [718]. Usually, the review team runs searches, removes duplicates, screens titles and abstracts, obtains full-text reports, screens full texts, assembles reports of the same study, extracts data, synthesises them and writes the final report. This process has great potential for waste [1921]. For systematic reviews, the median time from search to publication has improved from 14 months in 2008 [22] to 8 in 2013 [23] (mean time to complete was 17 months [24]; median time between first search and appearance of the review in PubMed was nearly 2 years [25]). The Cochrane Collaboration, a large organisation undertaking and maintaining systematic reviews of health care, largely works with volunteer health care professionals [26, 27] and the median time from Cochrane protocol to review publication was 2.4 years [28]. Keeping volunteer authors active on the review and the actual length of the review process are two major challenges to swift reviewing [29]. Efficiencies are needed.

Current preparation for reviewing

At the start of a new systematic review or an update for an existing systematic review, there is limited knowledge about the quantity of relevant literature. Although estimation of workload is possible through piloting or scoping searches [3032], this requires time and the exact number of relevant studies may remain unclear. This lack of clarity leaves assembled review teams vulnerable. The predicted investment of effort could be
  • Overestimated—and eventually review teams have no or very few studies for their new review or update—with the waste this would incur.

  • Underestimated—and the team is eventually surprised and, perhaps, overwhelmed with many relevant studies, with the risk of
    • ◦ Publishing a protocol but finding completion of the review unaffordable or impossible with the resulting wasteful unfinished or empty review.

    • ◦ Requesting extensions to funding; and/or

    • ◦ Running into delays that may render the final work being immediately out of date.

  • Accurately estimated—but what remains unclear is as to whether the investment needed to review/update is warranted by any potential to change what is already known.

Waste in systematic reviewing and information supply

The majority of the literature related to waste in systematic review are either focused on methodology [718] or automation of processes to shorten time-consuming tasks [3336]. For over two decades, information specialists have given practical guidance for waste reduction in systematic reviews [32, 3741]. Information specialists in the Cochrane Collaboration maintain specialised registers to support Cochrane reviews. Some of these registers are highly developed and shorten the systematic review process [42].

Study-based registers

Study-based registers (SBRs) are databases in which all records of same study are linked to one ‘parent’ report. This study report may contain meta-data extracted from the various ‘child’ records of that same study. Often building a SBR involves an information specialist running searches across major bibliographic databases, de-duplicating, screening for eligibility, and obtaining full text of records. Then, there is the process of linking ‘child’ reports to the parent study record, extracting, cleaning and curating meta-data and maintaining the register with updates. In the case of randomised trials, meta-data for the study may be gleaned from the individual records (e.g. details of participants, interventions, controls and outcomes (PICO)) or, working from the other direction, from the overarching review in which the study has been used (e.g. qualitative or quantitative data incorporated within the review relating to that study). Details of creating and maintaining a SBR has been reported elsewhere [42].

Aims and objectives

To describe how a SBR can be used to almost eliminate certain arduous steps in prospective systematic reviewing. We will illustrate how these steps can be accomplished in a matter of minutes or seconds and how this approach almost negates the early, inhibiting, and, we argue, wasteful, effort experienced by systematic reviewers. Although some benefits of SBRs have already been reported [42, 43], little has been presented on how SBRs can reduce waste whilst assisting prioritisation of systematic review work [44].

‘Living’ study-based registers

With a well-maintained SBR, an information specialist can provide the following data in a matter of minutes (stipulation of all estimates are review-specific but worked example follows):
  • Exact number of
    • ◦ Studies/related records in a field (e.g. schizophrenia, tardive dyskinesia);

    • ◦ Studies/related records relevant to a new title or update (e.g. vitamin E for people with tardive dyskinesia);

    • ◦ Studies/related records relevant to a class of interventions (e.g. calcium channel blockers);

    • ◦ Studies that have/have not already been data-extracted, and the extracted data were available;

    • ◦ Existing related reviews on a topic—and quantification of studies/related records within each review;

    • ◦ Comparisons possible to accurately scope existing relevant evidence on a given topic—and quantification of studies/related records within each comparison [45];

  • Alerts to
    • ◦ New studies, records to known studies and novel relevant treatments;

    • ◦ Research gaps in topic areas devoid of/with a dearth of evidence;

  • For the studies
    • ◦ Concatenated importable references, the output of each study or all the relevant studies;

    • ◦ Full reports of each studies collected into a study folder;

    • ◦ Completed data extraction forms of studies where available.

Essentially, a SBR should be ‘living’. These living curated registers involve minimal analyses and are maintained by an information specialist (Table 1). Such registers have been produced by the Cochrane Dementia (ALIOS), Renal/Kidney (maintained within MeerKat), Pregnancy and Childbirth, Stroke (DORIS) and Schizophrenia (within MeerKat) teams for over two decades. For some existing SBRs, there is further developments to add functions to include extracted data from reviews [42], links to standard text and to prioritise sharing these data publicly [46]. Unfortunately, CENTRAL and Cochrane Register of Studies (CRS) are, at best, rudimentary SBRs at the time of revising this paper (27 March 2019).
Table 1

Saved resource by use of study-based registers by stage of systematic reviewing

Armed with the information from these sophisticated registers a potential review team should be able to present a much more accurate estimate of workload before embarking on the grant application or the actual review or update. These registers should make it possible to truncate the period immediately after protocol publication, seeding the systematic review with extracted data and preparing for swift meta-analysis.

A case report from schizophrenia

Cochrane Schizophrenia has maintained a SBR of randomised trials for over two decades [47]. Routine searching identifies records that, with some help from automation, are merged into study reports (examples of studies with 10, 50 or even 100 records are not rare) helping minimise the risk of multiple counting with the systematic review. Meta-data (including number randomised) are part of the study record. Although increasingly automated, this process is facilitated by the group’s information specialist (FS). Since search strategies have been saved in bibliographic databases, monthly automatic updates are received through email. Then the information specialist spends three days per month for routine processes of updating the register: (1) 1 day for primary screening of search results and adding references to the register; (2) another day for obtaining full texts and linking them to their references; and finally (3) one last day for indexing the PICO meta-data from each full text and then assembling the separate references of the same study and linking them to that study. This register supports 324 maintained systematic reviews (17 May 2019).

Using this SBR, prioritisation of work could then proceed with efficiency (Fig. 1) and in line with items 2–6 from module 2 of SPARK, a prioritisation tool for systematic reviews [48].
Fig. 1
Fig. 1

The process of systematic reviewing using a study-based register

Estimates of costs for the grant application

In applying for NIHR UK Health Technology Assessment (HTA) Project Grant (14/27/02) [4951], a call for reviews relevant to treating people with Tardive Dyskinesia (a problematic adverse effect of antipsychotic drugs) use of the SBR gave a clear advantage. Cochrane Schizophrenia’s information specialist ran a highly specific, highly sensitive search (16 July 2015) in the SBR and identified the exact number of studies relevant to the problem (time spent on task 8 s). This number helped the grant application team provide an accurate assessment of the work to be done—and realistic estimates of costs.

Prediction of best composition of families of reviews

Tardive Dyskinesia is a condition for which many treatments have been used [52]. Arguments exist for ‘lumping and splitting’ at all sorts of levels. At the broadest level of ‘lumping’, the overview could encompass all treatments but this becomes unwieldy and impossible to update. At the finest level of ‘splitting’ each individual comparison of each treatment could be treated as a separate review. Even in a limited topic area such as Tardive Dyskinesia, this would lead to hundreds of separate reviews. Clearly there is a balance to be struck. By use of a controlled vocabulary for the meta-data within the SBR auto-grouping into logical treatment/comparison families for reviews can take place—and, once established, this can take place instantly. This ensures a pragmatic middle road dividing work into clinically logical bite-size reviews for later overviewing if required. Also, the classification of interventions within the register allows reviewing a class of interventions in a review. In the case of Tardive Dyskinesia, 10 separate review groupings were created (Table 2) (time spent on task 2 min and 10 s). This also helped the grant application team provide an accurate assessment of the output the funders could expect.
Table 2

Updated/started Cochrane reviews as a result of NIHR HTA Grant (14/27/02) [49]

Anticholinergic medication for antipsychotic-induced tardive dyskinesia [53]

Antipsychotic reduction and/or cessation and antipsychotics as specific treatments for tardive dyskinesia [54]

Benzodiazepines for antipsychotic-induced tardive dyskinesia [55]

Calcium channel blockers for antipsychotic-induced tardive dyskinesia [56]

Cholinergic medication for antipsychotic-induced tardive dyskinesia [57]

Gamma-aminobutyric acid agonists for antipsychotic-induced tardive dyskinesia [58]

Miscellaneous treatments for antipsychotic-induced tardive dyskinesia [59]

Non-antipsychotic catecholaminergic drugs for antipsychotic-induced tardive dyskinesia [60]

Pyridoxal 5 phosphate for neuroleptic-induced tardive dyskinesiaa [61]

Vesicular monoamine transporter inhibitors versus placebo for antipsychotic-induced tardive dyskinesiab [62]

Vitamin E for antipsychotic-induced tardive dyskinesia [63]

aThis review is absent in the published report [51] because there was no new study

bThis review is absent in the published report [51] because we became informed and started this review as a result of update search process in SBR

Prediction of effort needed at data extraction step and saving effort for others

In this particular case, the SBR also contains information on already extracted data. Therefore, the applicants were also informed of exactly how much work has been completed and allowed them to make accurate costing for the necessary remaining efforts (time spent on the task 8 s). Working with such a register affords applicants opportunities to ensure that their request for funding for this part of the effort can be seen as an investment. The extracted study data can, thereafter, be made available to anyone thus reducing future duplication of effort (see below).

Supply of documents

SBR systems such as Microsoft Access ‘MeerKat’ [6466] have capacity to output file batches grouped by review, sub-grouped into relevant study files, in turn containing all relevant records and references (time spent on the task 4 min and 43 s). In this cause, this allowed those applying for the grant to reassure funders that supply of documents was not an issue and, once the grant was given, to waste no time in acquiring papers and piecing together the studies from ‘salami’ or multiple publications of same study.

Future supply of full dataset

In the hope of evolving SBR towards making the level of document supply described above redundant and saving more time in the future—applicants sought and were granted support to extract all data from all randomised studies relevant to Tardive Dyskinesia and to make these data publicly available. This included each part of the data being made traceable to the exact site within the source record [67]. Any new updates of this will involve supply of documents containing tabulated, reliably and verifiably extracted data [50].

Updating

Cochrane recommends biennial update for reviews [68] but this timing is not always appropriate. Excessive updating wastes resource while inadequate updating could result in outdated or incomplete evidence being used [69]. While there are methods to detect if updating a review could change the current conclusion/practice, almost all require an awareness of the available ‘unused’ relevant literature [48, 7099], and some degree of screening and data checking to allow an informed decision. Within a well-constructed and maintained study register, this investment has already been made.

The upside

As the grant [49] was drawing to a close and the reviews were being completed. On the 26 April 2017, the SBR allowed the information specialist to run a final ‘just-before-submission’ update search limiting to not-already-identified records (time spent on the task 13 s). Just before publication, this search was used to inform the team that seven of the 10 reviews were fully current but two needed to be updated with a total of five new studies. This allowed the grant holders efficiently update the reviews just pre-publication to ensure they held fully current information.

The downside

This search also identified two new drugs (Valbenazine and Deutetrabenazine) entering the market specifically for treatment of people with Tardive Dyskinesia. These new compounds, unrelated to others, necessitate a new review outside of what was supported by the grant [62]. Unlike the decades ago when SBRs did not exist or were not sophisticated, it is now almost impossible to fail to identify a newly emerging treatment. This saves further waste in systematic reviews through inclusiveness of all treatments from all classes.

Feasibility of study-based registers

Although it seems exciting to start a systematic review with the extraction of data, the workload creating a SBR should not be underestimated. The investment of time is a frequent concern. Is it possible for all evidence-synthesis groups to maintain a SBR and what are the necessary requirements in creating such a register?

The short answer is that every systematic review is, in itself, a small SBR. Frequently at completion of any given review, these small registers (reviews) are rendered unusable to others or disassembled necessitating the next interested group of reviewers to have to repeat the construction. This is avoidable waste when collating all the data within a related group of reviews constitutes the embryonic SBR.

In Table 3, we itemise the time and resource required for establishing and maintaining our broad-based schizophrenia SBR.
Table 3

Characteristics of the study-based database in this study

Volume

Records:

~ 20,000 studies

~ 30,000 references/reports

PICO meta-data:

~ 230 healthcare conditions;

~ 2700 interventionsa;

~ 13,700 outcomes

Variety

Standard protocols for meta-data:

For references (RIS);

For studies (PICO)

Veracity

Document coverage

 Type:

Any

 Language

All

 Date/time:

Any

 Geography

Worldwide

 Publication status:

Published/unpublished

 Status of study:

Allb

Reliability

 Two independent Information Specialists checked data.

Velocity

Information specialist

Screens 1000–2000 references per month;

Adds 100–200 eligible references to register.

Value

Software: free.

Current number of maintained systematic reviews: 324.

Retracted studies: retraction linked into study record.

Reproducibility and replicability: all SBR’s review-specific steps can be repeated within seconds [100].

Prioritising: sensitive/specific direction of effort

Human resources: skilled information specialist

 Establish register

1 year (F/T) 2–3 years (P/T 50%)

 Maintain register

1 day/week

a Structured, controlled language (e.g. WHO ATC)

b Finished/ongoing/awaiting/terminated/unclear

Conclusions

Small SBRs, in the form of competed reviews, are increasingly prevalent. We maintain that there is a strong argument for creation of broad-based healthcare study-based registers linked to records containing data, text and other relevant information. Not to use already compiled data is wasteful and not to invest to create the SBR is passing cost—and waste—down the line to reviewers. Information specialist investment is already happening—repeatedly. We argue that focus and direction of this investment would avoid the ongoing unnecessary multiplication of effort [101].

We reported one example of the potential of SBRs for grant application. This is one amongst many. The living property of this register allowed the information specialist with his/her more sophisticated role—to become an integral—and useful—part of the review team.

Finally, the SBR promoted more sophisticated sharing of data from this project facilitating the not-so-distant full automation of living systematic reviews.

Abbreviations

HTA: 

Health Technology Assessment

PICO: 

Participants/Problem/Population, Interventions, Control/Comparison, and Outcomes

RIS: 

Research information systems

SBR: 

Study-based register

SPARK: 

The centre for systematic reviews on health policy and systems research

Declarations

Acknowledgements

Not applicable.

Funding

This manuscript has been supported by NIHR UK HTA Project Grant (14/27/02) [49]. Cochrane Schizophrenia Group’s Study-Based Register of Trials currently is not receiving any direct funding and it has been developed as a side product by Cochrane Schizophrenia Group.

Availability of data and materials

Not applicable.

Authors’ contributions

FS started the idea, examined the methods, drafted the first version of manuscript and reviewed the literature. CEA re-wrote the sections, re-organised the sections and commented on six drafts of the manuscript. Both authors read and approved the current version of the submission.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests. The views expressed in this paper are the authors’ own views not the views of their affiliated organisations.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Cochrane Schizophrenia Group, Division of Psychiatry and Applied Psychology, Institute of Mental Health, School of Medicine, University of Nottingham, Nottingham, UK

References

  1. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65. https://doi.org/10.1016/s0140-6736(13)62229-1.View ArticleGoogle Scholar
  2. Chan A-W, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66. https://doi.org/10.1016/s0140-6736(13)62296-5.View ArticleGoogle Scholar
  3. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. https://doi.org/10.1016/s0140-6736(13)62228-x.View ArticleGoogle Scholar
  4. Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75. https://doi.org/10.1016/s0140-6736(13)62227-8.View ArticleGoogle Scholar
  5. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JPA, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4. https://doi.org/10.1016/s0140-6736(13)62329-6.View ArticleGoogle Scholar
  6. Salman RA-S, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85. https://doi.org/10.1016/s0140-6736(13)62297-7.View ArticleGoogle Scholar
  7. Afshari A, Wetterslev J, Smith AF. Can systematic reviews with sparse data be trusted? Anaesthesia. 2017;72(1):12–16. PMID: 27804113. https://doi.org/10.1111/anae.13730.View ArticleGoogle Scholar
  8. Annane D, Jaeschke R, Guyatt G. Are systematic reviews and meta-analyses still useful research? Yes Intensive Care Med 2018;44(4):512–514. PMID: 29663046. https://doi.org/10.1007/s00134-018-5102-3.View ArticleGoogle Scholar
  9. Chalmers I, Glasziou P. Systematic reviews and research waste. Lancet. 2016;387(10014):122–123. PMID: 26841991. https://doi.org/10.1016/s0140-6736(15)01353-7 View ArticleGoogle Scholar
  10. Chevret S, Ferguson ND, Bellomo R. Are systematic reviews and meta-analyses still useful research? No Intensive Care Med. 2018;44(4):515–517. PMID: 29663047https://doi.org/10.1007/s00134-018-5066-3.View ArticleGoogle Scholar
  11. Garattini S, Jakobsen JC, Wetterslev J, Bertele V, Banzi R, Rath A, et al. Evidence-based clinical practice: overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016;32:13–21. PMID: 27160381. https://doi.org/10.1016/j.ejim.2016.03.020.View ArticleGoogle Scholar
  12. Handoll HH, Langhorne P. In defence of reviews of small trials: underpinning the generation of evidence to inform practice. Cochrane Database Syst Rev. 2015;11(11):ED000106. PMID: 26574167. https://doi.org/10.1002/14651858.ED000106.
  13. Lund H, Juhl C, Christensen R. Systematic reviews and research waste. Lancet (London, England). 2016;387(10014):123–124 PMID: 26841992. https://doi.org/10.1016/s0140-6736(15)01354-9 View ArticleGoogle Scholar
  14. Moller MH, Ioannidis JPA, Darmon M. Are systematic reviews and meta-analyses still useful research? We are not sure. Intensive Care. 2018;44(4):518–20. PMID: 29663048. https://doi.org/10.1007/s00134-017-5039-y.View ArticleGoogle Scholar
  15. Nelson RL. Systematic reviews and research waste. Lancet (London, England). 2016;387(10014):124. PMID: 26841994. https://doi.org/10.1016/s0140-6736(15)01355-0.View ArticleGoogle Scholar
  16. Roberts I, Ker K. How systematic reviews cause research waste. Lancet. 2015;386(10003):1536. PMID: 26530621. https://doi.org/10.1016/s0140-6736(15)00489-4.View ArticleGoogle Scholar
  17. Roberts I, Ker K. Cochrane: the unfinished symphony of research synthesis. Syst Rev. 2016;5(1):115. PMID: 27416925. https://doi.org/10.1186/s13643-016-0290-9.
  18. Roberts I, Ker K, Edwards P, Beecher D, Manno D, Sydenham E. The knowledge system underpinning healthcare is not fit for purpose and must change. BMJ. 2015;350:h2463. PMID: 26041754. https://doi.org/10.1136/bmj.h2463.View ArticleGoogle Scholar
  19. Andrade R, Pereira R, Weir A, Ardern CL, Espregueira-Mendes J. Zombie reviews taking over the PROSPERO systematic review registry. It's time to fight back! Br J Sports Med. 2017. PMID: 29021246. https://doi.org/10.1136/bjsports-2017-098252.
  20. Rathbone J, Carter M, Hoffmann T, Glasziou P. Solving research waste with better duplicate detection. Eur J Pub Health. 2015;25(Suppl 3):14–5. https://doi.org/10.1093/eurpub/ckv167.023.
  21. Tovey D. Systematic Reviews can both prevent and create research waste: what is Cochrane doing now and how can it do better in the future? Edinburgh: REWARD/EQUATOR; 2015.Google Scholar
  22. Sampson M, Shojania KG, Garritty C, Horsley T, Ocampo M, Moher D. Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008;61(6):531–6. PMID: 18471656. https://doi.org/10.1016/j.jclinepi.2008.02.004.View ArticleGoogle Scholar
  23. Beller EM, Chen JK, Wang UL, Glasziou PP. Are systematic reviews up-to-date at the time of publication? Syst Rev. 2013;2:36. PMID: 23714302. https://doi.org/10.1186/2046-4053-2-36.
  24. Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545. PMID: 28242767. https://doi.org/10.1136/bmjopen-2016-012545.View ArticleGoogle Scholar
  25. Bramer W, Bain P. Updating search strategies for systematic reviews using EndNote. J Med Libr Assoc. 2017;105(3):285–9. PMID: 28670219. https://doi.org/10.5195/jmla.2017.183.
  26. Tharyan P. Evidence of the people, by the people, and for the people. Cochrane Database Syst Rev. 2010;2011:ED000013 PMID: 21833936 DOI: https://doi.org/10.1002/14651858.ED000013
  27. Wilson M. Help us improve the health of people everywhere. Cochrane collaboration. 2019. https://www.cochrane.org/join-cochrane.
  28. Tricco AC, Brehaut J, Chen MH, Moher D. Following 411 Cochrane protocols to completion: a retrospective cohort study. PLoS One. 2008;3(11):e3684. PMID: 18997866. https://doi.org/10.1371/journal.pone.0003684.View ArticleGoogle Scholar
  29. Turner T, Green S, Tovey D, McDonald S, Soares-Weiser K, Pestridge C, et al. Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement. Syst Rev. 2017;6(1):147. PMID: 28760162.https://doi.org/10.1186/s13643-017-0542-3.
  30. Clavisi O, Bragge P, Tavender E, Turner T, Gruen RL. Effective stakeholder participation in setting research priorities using a Global Evidence Mapping approach. J Clin Epidemiol. 2013 66(5):496–502.e2. PMID: 22819249. https://doi.org/10.1016/j.jclinepi.2012.04.002.View ArticleGoogle Scholar
  31. Hoekstra D, Mutsch M, Kien C, Gerhardus A, Lhachimi SK. Identifying and prioritising systematic review topics with public health stakeholders: a protocol for a modified Delphi study in Switzerland to inform future research agendas. BMJ Open. 2017;7(8):e015500. PMID: 28780546. https://doi.org/10.1136/bmjopen-2016-015500.View ArticleGoogle Scholar
  32. Otter MLE, Wright JM, King NV. Developing the Librarians' role in supporting grant applications and reducing waste in research: outcomes from a literature review and survey in the NIHR research design service. New Rev Acad Librariansh. 2017;23(2–3):258–74. https://doi.org/10.1080/13614533.2017.1330219.View ArticleGoogle Scholar
  33. Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015;4:78. PMID: 26073888. https://doi.org/10.1186/s13643-015-0066-7.
  34. O'Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS. Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Rev. 2018;7(1):3. PMID: 29316980. https://doi.org/10.1186/s13643-017-0667-4.
  35. Tsafnat G, Dunn A, Glasziou P, Coiera E. The automation of systematic reviews. BMJ. 2013;346:f139. PMID: 23305843. https://doi.org/10.1136/bmj.f139 View ArticleGoogle Scholar
  36. Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev. 2014;3:74. PMID: 25005128. https://doi.org/10.1186/2046-4053-3-74.
  37. Kirtley S. Can librarians contribute to increasing value and reducing waste in medical research? EQUATOR Network: Enhancing the Quality and Transparency of Health Research 2014.Google Scholar
  38. Kirtley S. Improving the reliability and value of medical research literature: a new role for librarians in promoting good research reporting practices. Rome: 14th EAHIL; 2014. p. 11-3.Google Scholar
  39. Kirtley S. Increasing value and reducing waste in biomedical research: librarians are listening and are part of the answer. Lancet. 2016;387(10028):1601. PMID: 27116055.https://doi.org/10.1016/s0140-6736(16)30241-0.View ArticleGoogle Scholar
  40. Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, et al. Towards evidence based research. BMJ. 2016;355:i5440. PMID: 27797786. https://doi.org/10.1136/bmj.i5440.
  41. Moher D, Shamseer L, Cobey KD, Lalu MM, Galipeau J, Avey MT, et al. Stop this waste of people, animals and money. Nature. 2017;549(7670):23–25. PMID: 28880300. https://doi.org/10.1038/549023a.View ArticleGoogle Scholar
  42. Shokraneh F, Adams CE. Study-based registers of randomized controlled trials: starting a systematic review with data extraction or meta-analysis. BioImpacts. 2017;7(4):209–217. PMID: 29435428. https://doi.org/10.15171/bi.2017.25.View ArticleGoogle Scholar
  43. Shokraneh F, Adams CE. Potentials of registers of randomized controlled trials in automating the systematic reviews. Edinburgh: REWARD/EQUATOR; 2015. https://doi.org/10.13140/RG.2.2.19853.84963.
  44. Shokraneh F, Adams CE. How do the study-based registers reduce waste in systematic reviewing? Edinburgh: 25th Cochrane Colloquium; 2018. https://doi.org/10.13140/RG.2.2.16401.28004.
  45. Shokraneh F, Adams CE. A simple formula for enumerating comparisons in trials and network meta-analysis. F1000Research. 2019;8:38. PMID: 30863537. https://doi.org/10.12688/f1000research.17352.2.View ArticleGoogle Scholar
  46. Shokraneh F, Adams CE, Clarke M, Amato L, Bastian H, Beller E, et al. Why Cochrane should prioritise sharing data. BMJ. 2018;362:k3229. PMID: 30061322. https://doi.org/10.1136/bmj.k3229.
  47. Shokraneh F. Register of trials. Cochrane Schizophrenia Group 2018;http://schizophrenia.cochrane.org/register-trials
  48. Akl EA, Fadlallah R, Ghandour L, Kdouh O, Langlois E, Lavis JN, et al. The SPARK tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation. Health Res Policy Syst. 2017;15(1):77. PMID: 28870215. https://doi.org/10.1186/s12961-017-0242-4.
  49. Adams CE, Walker DM, Gray B, Soares-Weiser K. HTA - 14/27/02: a systematic review and network meta-analysis of the safety and clinical effectiveness of interventions for treating or preventing deterioration of symptoms of antipsychotic-induced tardive dyskinesia (TD). 2015. https://www.fundingawards.nihr.ac.uk/award/14/27/02.
  50. Adams CE, Walker DM, Gray B, Soares-Weiser K, Bergman H, Zhao S, et al. Appendix: traceable extracted data from included studies of tardive dyskinesia reviews. 2017. https://doi.org/10.13140/RG.2.2.28907.95529.
  51. Bergman H, Walker DM, Nikolakopoulou A, Soares-Weiser K, Adams CE. Systematic review of interventions for treating or preventing antipsychotic-induced tardive dyskinesia. Health Technol Assess. 2017;21(43):1–218. PMID: 28812541. https://doi.org/10.3310/hta21430.View ArticleGoogle Scholar
  52. Soares K, McGrath J, Adams C. Evidence and tardive dyskinesia. Lancet. 1996;347(9016):1696–7. https://doi.org/10.1016/s0140-6736(96)91525-1.View ArticleGoogle Scholar
  53. Bergman H, Soares-Weiser K. Anticholinergic medication for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000204. PMID: 29341071. https://doi.org/10.1002/14651858.CD000204.pub2.
  54. Bergman H, Rathbone J, Agarwal V, Soares-Weiser K. Antipsychotic reduction and/or cessation and antipsychotics as specific treatments for tardive dyskinesia. Cochrane Database Syst Rev. 2018;2:CD000459. PMID: 29409162. https://doi.org/10.1002/14651858.CD000459.pub3.
  55. Bergman H, Bhoopathi PS, Soares-Weiser K. Benzodiazepines for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000205. PMID: 29352477. https://doi.org/10.1002/14651858.CD000205.pub3.
  56. Essali A, Soares-Weiser K, Bergman H, Adams CE. Calcium channel blockers for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD000206. PMID: 29578611. https://doi.org/10.1002/14651858.CD000206.pub4.
  57. Tammenmaa-Aho I, Asher R, Soares-Weiser K, Bergman H. Cholinergic medication for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD000207. PMID: 29553158. https://doi.org/10.1002/14651858.CD000207.pub2.
  58. Alabed S, Latifeh Y, Mohammad HA, Bergman H. Gamma-aminobutyric acid agonists for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;4:CD000203. PMID: 29663328. https://doi.org/10.1002/14651858.CD000203.pub4.
  59. Soares-Weiser K, Rathbone J, Ogawa Y, Shinohara K, Bergman H. Miscellaneous treatments for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD000208. PMID: 29552749. https://doi.org/10.1002/14651858.CD000208.pub2.
  60. El-Sayeh HG, Rathbone J, Soares-Weiser K, Bergman H. Non-antipsychotic catecholaminergic drugs for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000458. PMID: 29342497. https://doi.org/10.1002/14651858.CD000458.pub3.
  61. Adelufosi AO, Abayomi O, Ojo TM. Pyridoxal 5 phosphate for neuroleptic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2015;4:CD010501. PMID: 25866243. https://doi.org/10.1002/14651858.CD010501.pub2.
  62. Karl B, Bergman H, Abd El Sayed S, Adams CE. Vesicular monoamine transporter inhibitors versus placebo for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;3:CD012986. https://doi.org/10.1002/14651858.cd012986.
  63. Soares-Weiser K, Maayan N, Bergman H. Vitamin E for antipsychotic-induced tardive dyskinesia. Cochrane Database Syst Rev. 2018;1:CD000209. PMID: 29341067. https://doi.org/10.1002/14651858.CD000209.pub3.
  64. Kaur S, de Souza M, Fenton M, Adams CE. MeerKat 1.4 – and beyond. Cochrane Schizophrenia Group; 2009.Google Scholar
  65. MeerKat Working G, Editor MeerKat: getting the most out of your study-based specialized register. Melbourne: Cochrane Colloquium; 2005.Google Scholar
  66. Wright J. MeerKat user guide for MeerKat version 1.4. Grant T, editor. Leeds: Cochrane Schizophrenia Group; 2006.Google Scholar
  67. Shokraneh F, Adams CE. Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms. Syst Rev. 2017;6(1):153. PMID: 28778216. https://doi.org/10.1186/s13643-017-0546-z.
  68. Higgins JPT, Green S, Scholten R. Chapter 3: maintaining reviews: updates, amendments and feedback. Cochrane handbook for systematic reviews of interventions version 510 [updated March 2011]: The Cochrane Collaboration; 2011.Google Scholar
  69. Jaidee W, Moher D, Laopaiboon M. Time to update and quantitative changes in the results of cochrane pregnancy and childbirth reviews. PLoS One. 2010;5(7):e11553 PMID: 20644625. https://doi.org/10.1371/journal.pone.0011553.View ArticleGoogle Scholar
  70. Chalmers I, Haynes B. Reporting, updating, and correcting systematic reviews of the effects of health care. BMJ. 1994;309(6958):862–5. PMID: 7950620. https://doi.org/10.1136/bmj.309.6958.862.View ArticleGoogle Scholar
  71. Barrowman NJ, Fang M, Sampson M, Moher D. Identifying null meta-analyses that are ripe for updating. BMC Med Res Methodol. 2003;3:13. PMID: 12877755. https://doi.org/10.1186/1471-2288-3-13.
  72. Waters E, Doyle J, Jackson N. Evidence-based public health: improving the relevance of Cochrane Collaboration systematic reviews to global public health priorities. J Public Health Med. 2003;25(3):263–6 PMID: 14575208.View ArticleGoogle Scholar
  73. Doyle J, Waters E, Yach D, McQueen D, De Francisco A, Stewart T, et al. Global priority setting for Cochrane systematic reviews of health promotion and public health research. J Epidemiol Community Health. 2005;59(3):193–7. PMID: 15709077. https://doi.org/10.1136/jech.2003.019547.View ArticleGoogle Scholar
  74. Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. A systematic review identified few methods and strategies describing when and how to update systematic reviews. J Clin Epidemiol. 2007;60(11):1095–104. PMID: 17938050. https://doi.org/10.1016/j.jclinepi.2007.03.008.View ArticleGoogle Scholar
  75. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007;147(4):224. https://doi.org/10.7326/0003-4819-147-4-200708210-00179.View ArticleGoogle Scholar
  76. Cohen AM. Optimizing feature representation for automated systematic review work prioritization. AMIA Annu Symp Proc. 2008;6:121–5 PMID: 18998798.Google Scholar
  77. Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. When and how to update systematic reviews. Cochrane Database Syst Rev. 2008(1):Mr000023 PMID: 18254126 DOI: https://doi.org/10.1002/14651858.MR000023.pub3
  78. Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE, et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol. 2008;61(8):755–762. PMID: 18586179. https://doi.org/10.1016/j.jclinepi.2007.10.003.View ArticleGoogle Scholar
  79. Cohen AM, Ambert K, McDonagh M. Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc. 2009;16(5):690–704. PMID: 19567792. https://doi.org/10.1197/jamia.M3162.View ArticleGoogle Scholar
  80. Sampson MJ. Updating searches for systematic reviews: Aberystwyth University; 2009.Google Scholar
  81. Sutton AJ, Donegan S, Takwoingi Y, Garner P, Gamble C, Donald A. An encouraging assessment of methods to inform priorities for updating systematic reviews. J Clin Epidemiol. 2009;62(3):241–51. PMID: 18783919. https://doi.org/10.1016/j.jclinepi.2008.04.005.View ArticleGoogle Scholar
  82. Garritty C, Tsertsvadze A, Tricco AC, Sampson M, Moher D. Updating systematic reviews: an international survey. PLoS One. 2010;5(4):e9914. PMID: 20376338. https://doi.org/10.1371/journal.pone.0009914.View ArticleGoogle Scholar
  83. Bastian H, Scheibler F, Knelangen M, Zschorlich B, Nasser M, Waltering A. Choosing health technology assessment and systematic review topics: the development of priority-setting criteria for patients' and consumers' interests. Int J Technol Assess Health Care. 2011;27(4):348–56. PMID: 22004776. https://doi.org/10.1017/s0266462311000547.View ArticleGoogle Scholar
  84. Meremikwu M, Udoh E, Nwagbara B, Effa E, Oringanje C, Edet B, et al. Priority setting for systematic review of health care interventions in Nigeria. Health Pol. 2011;99(3):244–9. PMID: 21131092. https://doi.org/10.1016/j.healthpol.2010.11.004.View ArticleGoogle Scholar
  85. Chung M, Newberry SJ, Ansari MT, Yu WW, Wu H, Lee J, et al. Two methods provide similar signals for the need to update systematic reviews. J Clin Epidemiol. 2012;65(6):660–8. PMID: 22464414. https://doi.org/10.1016/j.jclinepi.2011.12.004.View ArticleGoogle Scholar
  86. Cohen AM, Ambert K, McDonagh M. Studying the potential impact of automated document classification on scheduling a systematic review update. BMC Med Inform Decis Mak. 2012;12:33. PMID: 22515596. https://doi.org/10.1186/1472-6947-12-33.
  87. Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. AHRQ Methods for effective health care. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012.View ArticleGoogle Scholar
  88. Hoomans T, Seidenfeld J, Basu A, Meltzer D. AHRQ methods for effective health care. Systematizing the use of value of information analysis in prioritizing systematic reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012.Google Scholar
  89. Pattanittum P, Laopaiboon M, Moher D, Lumbiganon P, Ngamjarus C. A comparison of statistical methods for identifying out-of-date systematic reviews. PLoS One. 2012;7(11):e48894. PMID: 23185281. https://doi.org/10.1371/journal.pone.0048894.View ArticleGoogle Scholar
  90. Mickenautsch S, Yengopal V. SYSTEM research note on: the modified Ottawa method for updating systematic reviews. J Minimum Intervention Dentistry. 2013;6:15–29.Google Scholar
  91. Newberry SJ, Ahmadzai N, Motala A, Tsertsvadze A, Maglione M, Ansari MT, et al. AHRQ methods for effective health care. Surveillance and identification of signals for updating systematic reviews: implementation and early experience. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013.Google Scholar
  92. Takwoingi Y, Hopewell S, Tovey D, Sutton AJ. A multicomponent decision tool for prioritising the updating of systematic reviews. Bmj. 2013;347:f7191. PMID: 24336453. https://doi.org/10.1136/bmj.f7191.View ArticleGoogle Scholar
  93. Tugwell P, Knottnerus JA, Idzerda L. Methods for setting priorities in systematic reviews. J Clin Epidemiol. 2013;66(5):467–8. PMID: 23548134. https://doi.org/10.1016/j.jclinepi.2013.02.008.View ArticleGoogle Scholar
  94. Shekelle PG, Motala A, Johnsen B. Assessment of a method to detect signals for updating systematic reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014.Google Scholar
  95. Shekelle PG, Motala A, Johnsen B, Newberry SJ. Assessment of a method to detect signals for updating systematic reviews. Syst Rev. 2014;3:13. PMID: 24529068. https://doi.org/10.1186/2046-4053-3-13.
  96. Martinez Garcia L, Sanabria AJ, Araya I, Lawson J, Sola I, Vernooij RW, et al. Efficiency of pragmatic search strategies to update clinical guidelines recommendations. BMC Med Res Methodol. 2015;15:57. PMID: 26227021. https://doi.org/10.1186/s12874-015-0058-2.
  97. Garner P, Hopewell S, Chandler J, MacLehose H, Schunemann HJ, Akl EA, et al. When and how to update systematic reviews: consensus and checklist. BMJ. 2016;354:i3507. PMID: 27443385. https://doi.org/10.1136/bmj.i3507.
  98. Sampson M, de Bruijn B, Urquhart C, Shojania K. Complementary approaches to searching MEDLINE may be sufficient for updating systematic reviews. J Clin Epidemiol. 2016;78:108–15. PMID: 26976054. https://doi.org/10.1016/j.jclinepi.2016.03.004.View ArticleGoogle Scholar
  99. Martinez Garcia L, Pardo-Hernandez H, Superchi C, Nino de Guzman E, Ballesteros M, Ibargoyen Roteta N, et al. Methodological systematic review identifies major limitations in prioritization processes for updating. J Clin Epidemiol. 2017;86:11–24. PMID: 28549931. https://doi.org/10.1016/j.jclinepi.2017.05.008.View ArticleGoogle Scholar
  100. Shokraneh F. Reproducibility and replicability of systematic reviews. World J Meta-Anal. 2019;7(3):66–71. https://doi.org/10.13105/wjma.v7.i3.66 View ArticleGoogle Scholar
  101. Shokraneh F. Reducing waste and increasing value through embedded replicability and reproducibility in systematic review process and automation. J Clin Epidemiol. 2019. https://doi.org/10.1016/j.jclinepi.2019.04.008.

Copyright

© The Author(s). 2019

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement