Table 1 Examples of machine learning systems available for use in systematic reviews
Example tools | Comments | |
|---|---|---|
Search—finding RCTs | RobotSearch (https://robotsearch.vortext.systems) Cochrane Register of Studies (https://community.cochrane.org/help/tools-and-software/crs-cochrane-register-studies) RCT tagger (http://arrowsmith.psych.uic.edu/cgi-bin/arrowsmith_uic/RCT_Tagger.cgi) | • Validated machine learning filters available for identifying RCTs and suitable for fully automatic use • Conventional topic-specific keyword search strategy still needed • No widely available tools for non-RCT design currently |
Search—literature exploration | Allows search of PubMed for concepts (i.e. chemicals, diseases, drugs, genes, metabolites, proteins, species and anatomical entities) | |
Screening | Abstrackr (http://abstrackr.cebm.brown.edu) [30] EPPI reviewer (https://eppi.ioe.ac.uk/cms/er4) [31] RobotAnalyst (http://www.nactem.ac.uk/robotanalyst/) [32] SWIFT-Review (https://www.sciome.com/swift-review/) Colandr (https://www.colandrapp.com) Rayyan (https://rayyan.qcri.org) | • Screening systems automatically sort a search retrieval by relevance • RobotAnalyst and SWIFT-Review also allow topic modelling, where abstracts relating to similar topics are automatically grouped, allowing the user to explore the search retrieval. |
Data extraction | ExaCT (http://exactdemo.iit.nrc.ca) RobotReviewer (https://robotreviewer.vortext.systems) NaCTeM text mining tools for automatically extracting concepts relating to genes and proteins (NEMine), yeast metabolites (Yeast MetaboliNER), and anatomical entities (AnatomyTagger) (http://www.nactem.ac.uk/software.php) | • These prototype systems automatically extract data elements (e.g. sample sizes, descriptions of PICO elements) from free-texts. |
Bias assessment | RobotReviewer (https://robotreviewer.vortext.systems) | • Automatic assessment of biases in reports of RCTs • System recommended for semi-automatic use (i.e. with human reviewer checking and correcting the ML suggestions) |