Skip to main content

Table 1 Examples of machine learning systems available for use in systematic reviews

From: Toward systematic review automation: a practical guide to using machine learning tools in research synthesis

  Example tools Comments
Search—finding RCTs RobotSearch (https://robotsearch.vortext.systems)
Cochrane Register of Studies (https://community.cochrane.org/help/tools-and-software/crs-cochrane-register-studies)
RCT tagger (http://arrowsmith.psych.uic.edu/cgi-bin/arrowsmith_uic/RCT_Tagger.cgi)
• Validated machine learning filters available for identifying RCTs and suitable for fully automatic use
• Conventional topic-specific keyword search strategy still needed
• No widely available tools for non-RCT design currently
Search—literature exploration Thalia (http://nactem-copious.man.ac.uk/Thalia/) Allows search of PubMed for concepts (i.e. chemicals, diseases, drugs, genes, metabolites, proteins, species and anatomical entities)
Screening Abstrackr (http://abstrackr.cebm.brown.edu) [30]
EPPI reviewer (https://eppi.ioe.ac.uk/cms/er4) [31]
RobotAnalyst (http://www.nactem.ac.uk/robotanalyst/) [32]
SWIFT-Review (https://www.sciome.com/swift-review/)
Colandr (https://www.colandrapp.com)
Rayyan (https://rayyan.qcri.org)
• Screening systems automatically sort a search retrieval by relevance
• RobotAnalyst and SWIFT-Review also allow topic modelling, where abstracts relating to similar topics are automatically grouped, allowing the user to explore the search retrieval.
Data extraction ExaCT (http://exactdemo.iit.nrc.ca)
RobotReviewer (https://robotreviewer.vortext.systems)
NaCTeM text mining tools for automatically extracting concepts relating to genes and proteins (NEMine), yeast metabolites (Yeast MetaboliNER), and anatomical entities (AnatomyTagger) (http://www.nactem.ac.uk/software.php)
• These prototype systems automatically extract data elements (e.g. sample sizes, descriptions of PICO elements) from free-texts.
Bias assessment RobotReviewer (https://robotreviewer.vortext.systems) • Automatic assessment of biases in reports of RCTs
• System recommended for semi-automatic use (i.e. with human reviewer checking and correcting the ML suggestions)