Skip to main content

Table 3 The revised QUADAS-2 tool for risk of bias assessment

From: Automation of literature screening using machine learning in medical evidence synthesis: a diagnostic test accuracy systematic review protocol

Domains Signal questions Answers
“Patient” (literature) Selection Risk of bias  
Was a consecutive or random sample of literatures enrolled Yes/no/unclear
Was a case-control design avoided Yes/no/unclear
Did the study avoid inappropriate exclusions Yes/no/unclear
Could the selection of literatures have introduced bias Low/high/unclear risk
Concerns regarding applicability  
Is there concern that the included literatures do not match the review question Low/high/unclear risk
Index test (AI algorithms in literature screening) Risk of bias  
Were the index test results interpreted without knowledge of the results of the reference standard Yes/no/unclear
If a threshold was used, was it pre-specified Yes/no/unclear
Could the conduct or interpretation of the index test have introduced bias Low/high/unclear risk
Concerns regarding applicability  
Is there concern that the index test, its conduct, or interpretation differ from the review question Low/high/unclear risk
Reference standard (results of screening by human investigators) Risk of bias  
Is the reference standard likely to correctly classify the target condition Yes/no/unclear
Were the reference standard results interpreted without knowledge of the results of the index test Yes/no/unclear
Could the reference standard, its conduct, or its interpretation have introduced bias Low/high/unclear risk
Concerns regarding applicability  
Is there concern that the target condition as defined by the reference standard does not match the review question Low/high/unclear risk
Flow and timing Risk of bias  
Did all literatures receive a reference standard Yes/no/unclear
Did literatures receive the same reference standard Yes/no/unclear
Were all literatures included in the analysis Yes/no/unclear
Could the literature flow have introduced bias Low/high/unclear risk