Machine learning reduced workload for the Cochrane COVID-19 Study Register: development and evaluation of the Cochrane COVID-19 Study Classifier

Table 1 Distribution of classifier scores among ‘included’ and ‘excluded’ calibration records and related performance metrics

Classifier score	90–99	80–89	70–79	60–69	50–59	40–49	30–39	20–29	10–19	0–9	Totals
Included N	2853	1059	610	402	284	202	195	180	129	91	6005
Excluded N	83	156	190	237	290	364	578	885	1736	5599	10,118
Totals	2936	1215	800	639	574	566	773	1065	1865	5690	16,123
Precision	0.97	0.87	0.76	0.63	0.49	0.36	0.25	0.17	0.07	0.02
Cumulative recall	0.48	0.65	0.75	0.82	0.87	0.90	0.93	0.96	0.98	1.00
Cumulative precision	0.97	0.94	0.91	0.88	0.84	0.80	0.75	0.68	0.57	0.37
Threshold classifier score (recall >0.99)	7
Screened included N^a	5950
Screened excluded N^a	5487
Precision^a	0.52
Discarded (‘Lost’) included N^a	55
Discarded excluded N^a	4631
Net workload reduction N^a	4686
Net workload reduction %^a	29.1%

ISSN: 2046-4053