Table 2 AUC-ROC values by combination of MLTs, balancing techniques and balancing ratios across 14 systematic reviews
From: Screening PubMed abstracts: is class imbalance always a challenge to machine learning?
MLT | Systematic review | Method for imbalance | ||||
|---|---|---|---|---|---|---|
None | ROS-35:65 | ROS-50:50 | RUS-35:65 | RUS-50:50 | ||
GLMNet | Cavender et al. 2014 [26] | 0.9667 | 1 | 1 | 0.9988 | 1 |
Chatterjee et al. 2014 [27] | 0.9738 | 0.9667 | 0.9667 | 0.9875 | 0.9963 | |
Douxfils et al. 2014 [23] | 0.9667 | 0.9988 | 0.9988 | 1 | 0.9988 | |
Funakoshi et al 2014 [28] | 0.8851 | 0.9602 | 0.9799 | 0.9794 | 0.9885 | |
Kourbeti et al. 2014 [24] | 0.9518 | 0.9921 | 0.9991 | 0.9918 | 0.9991 | |
Li et al. 2014 [18] | 0.9 | 1 | 1 | 0.9975 | 0.97 | |
Li et al. 2014 [25] | 0.8975 | 0.8975 | 0.9475 | 0.99 | 0.9375 | |
Liu et al. 2014 [22] | 0.915 | 0.98 | 1 | 0.9983 | 0.9975 | |
Lv et al. 2014 [19] | 1 | 1 | 1 | 0.9963 | 0.9963 | |
Meng et al 2014 [16] | 1 | 1 | 1 | 1 | 0.9875 | |
Segelov et al. 2014 [17] | 0.9667 | 1 | 0.9988 | 0.995 | 0.9863 | |
Wang et al. 2015 [20] | 0.9667 | 1 | 1 | 0.9988 | 0.9988 | |
Yang et al. 2014 [15] | 0.975 | 0.975 | 1 | 1 | 1 | |
Zhou at al. 2014 [21] | 1 | 1 | 1 | 1 | 0.98 | |
k-nearest neighbors | Cavender et al. 2014 [26] | 1 | 0.5113 | 0.5063 | 0.5013 | 0.5792 |
Chatterjee et al. 2014 [27] | 0.9988 | 0.5388 | 0.5363 | 0.5063 | 0.6333 | |
Douxfils et al. 2014 [23] | 0.9667 | 0.5213 | 0.5113 | 0.5075 | 0.5625 | |
Funakoshi et al 2014 [28] | 0.9955 | 0.5005 | 0.5 | 0.5 | 0.5885 | |
Kourbeti et al. 2014 [24] | NA | NA | NA | 0.5 | 0.5661 | |
Li et al. 2014 [18] | 0.9775 | 0.63 | 0.6125 | 0.5125 | 0.7775 | |
Li et al. 2014 [25] | 0.7975 | 0.685 | 0.59 | 0.5675 | 0.71 | |
Liu et al. 2014 [22] | 0.9975 | 0.5017 | 0.5017 | 0.5 | 0.5983 | |
Lv et al. 2014 [19] | 1 | 0.5075 | 0.505 | 0.5025 | 0.6996 | |
Meng et al 2014 [16] | 0.9875 | 0.59 | 0.57 | 0.515 | 0.71 | |
Segelov et al. 2014 [17] | 0.9283 | 0.51 | 0.5063 | 0.5 | 0.5625 | |
Wang et al. 2015 [20] | 1 | 0.5056 | 0.5056 | 0.5 | 0.5237 | |
Yang et al. 2014 [15] | 0.9404 | 0.5288 | 0.52 | 0.5025 | 0.6333 | |
Zhou at al. 2014 [21] | 1 | 0.675 | 0.6425 | 0.54 | 0.71 | |
Random forest | Cavender et al. 2014 [26] | 1 | 1 | 1 | 1 | 1 |
Chatterjee et al. 2014 [27] | 0.9167 | 0.975 | 0.975 | 0.9963 | 1 | |
Douxfils et al. 2014 [23] | 1 | 1 | 1 | 1 | 1 | |
Funakoshi et al 2014 [28] | 0.9184 | 0.9517 | 0.9299 | 0.9895 | 0.9895 | |
Kourbeti et al. 2014 [24] | 0.9918 | 0.9854 | 0.9854 | 0.9988 | 0.9984 | |
Li et al. 2014 [18] | 0.95 | 1 | 1 | 1 | 1 | |
Li et al. 2014 [25] | 0.8 | 0.9 | 0.9 | 0.9 | 0.9475 | |
Liu et al. 2014 [22] | 0.98 | 0.9992 | 0.9783 | 0.9992 | 0.9992 | |
Lv et al. 2014 [19] | 1 | 1 | 1 | 0.9988 | 0.9988 | |
Meng et al 2014 [16] | 0.95 | 0.95 | 0.95 | 1 | 1 | |
Segelov et al. 2014 [17] | 0.9988 | 0.9988 | 0.9988 | 0.9975 | 0.9963 | |
Wang et al. 2015 [20] | 0.9815 | 0.9821 | 0.9827 | 0.9994 | 0.9975 | |
Yang et al. 2014 [15] | 0.95 | 0.975 | 0.95 | 0.9083 | 0.9046 | |
Zhou at al. 2014 [21] | 1 | 1 | 1 | 1 | 0.995 | |
Support vector machines | Cavender et al. 2014 [26] | 1 | 1 | 1 | 1 | 0.825 |
Chatterjee et al. 2014 [27] | 1 | 1 | 0.9988 | 1 | 0.9263 | |
Douxfils et al. 2014 [23] | 1 | 1 | 1 | 0.9963 | 0.8338 | |
Funakoshi et al 2014 [28] | 0.999 | 0.999 | 0.9985 | 0.9945 | 0.975 | |
Kourbeti et al. 2014 [24] | 0.9927 | 0.9927 | 0.9991 | 0.9988 | 0.9875 | |
Li et al. 2014 [18] | 1 | 0.9975 | 0.9975 | 0.9325 | 0.5625 | |
Li et al. 2014 [25] | 0.85 | 0.9 | 0.9925 | 0.98 | 0.6775 | |
Liu et al. 2014 [22] | 1 | 1 | 1 | 0.9992 | 0.96 | |
Lv et al. 2014 [19] | 1 | 1 | 1 | 0.9988 | 0.785 | |
Meng et al 2014 [16] | 1 | 1 | 1 | 0.99 | 0.62 | |
Segelov et al. 2014 [17] | 0.9333 | 0.9333 | 1 | 0.995 | 0.8013 | |
Wang et al. 2015 [20] | 1 | 0.9857 | 1 | 0.9988 | 0.9681 | |
Yang et al. 2014 [15] | 0.975 | 0.9417 | 0.9654 | 0.995 | 0.8825 | |
Zhou at al. 2014 [21] | 1 | 1 | 1 | 1 | 0.7425 | |