Skip to main content

Table 2 Pre-trained models used in the experiments, the corpus type used in their training, and the number of parameters per model

From: Ensemble of deep learning language models to support the creation of living systematic reviews for the COVID-19 literature

Pre-trained models

Corpus type

# Parameters (M)

RoBERTabase [35]

General

110

RoBERTalarge [35]

General

340

COVID-Twitter-BERT [36]

Bio (COVID-19)

110

BioBERT [37]

Bio

110

PubMedBERT [38]

Bio

110

Back to article page

Systematic Reviews

ISSN: 2046-4053