Skip to main content

Table 4 Summary of screening workload savings for three systematic reviews

From: Automated screening of research studies for systematic reviews using study characteristics

Screening rule TP FP FN TN Pr Re Work saved
Hamra 2014 (n = 615 articles) Max = 97.2%
 All 4 PECO Terms 5 5 12 593 50% 29% 98.4%
 Any 3 PECO Terms 12 24 5 574 33% 71% 94.1%
 Any 2 PECO Terms 17 89 0 509 16% 100% 82.8%
 PEO 11 17 6 581 39% 60% 95.4%
 PE 11 13 6 585 46% 65% 96.1%
 EO 17 65 0 533 21% 100% 86.7%
Johnson 2014 (n = 2470 articles) Max = 99.3%
 All 4 PECO Terms 3 1 14 2455 75% 18% 99.8%
 Any 3 PECO Terms 14 12 3 2441 54% 82% 98.9%
 Any 2 PECO Terms 16 60 1 2393 21% 94% 96.9%
 PEO 13 49 4 2413 25% 76% 97.5%
 PE 13 5 4 1551 72% 76% 99.3%
 EO 16 11 1 2442 59% 94% 98.9%
Thayer 2013 (n = 1880 articles) Max = 99.4%
 All 4 PECO Terms 7 20 13 1840 26% 35% 98.6%
 Any 3 PECO Terms 9 83 2 1786 10% 82% 95.1%
 Any 2 PECO Terms 11 304 0 1565 3% 100% 83.2%
 PEO 7 116 4 1753 6% 64% 93.5%
 PE 14 45 6 1815 24% 70% 96.9%
 EO 11 195 0 1674 5% 100% 89.0%
Average (n = 4965 articles) Max = 99.1%
 All 4 PECO Terms 15 26 39 4888 37% 28% 99.2%
 Any 3 PECO Terms 35 119 10 4801 23% 78% 96.9%
 Any 2 PECO Terms 44 453 1 4467 9% 98% 90.0%
 PEO 31 182 14 4747 15% 69% 95.7%
 PE 38 63 16 3951 38% 70% 98.0%
 EO 44 271 1 4649 14% 98% 93.7%
  1. Work saved is the proportion of all positives in the entire set of n references (i.e., 1 − (TP + FP)/n)
  2. TP true positive, FP false positives, FN false negative, TN true negative, P precision, Re recall