Table 3 Checklist for determining whether GRADE was used appropriately

If the response to all of the following questions is “yes,” then GRADE has been used appropriately at the review level. If any of these criteria are not met, then GRADE was not used appropriately.

• Do the SR authors use GRADE for all outcomes for which a meta-analysis was conducted?

• Are the GRADE assessments compiled in a GRADE evidence table (summary of findings table or evidence profile)?^a

• Do the SR authors refrain from making recommendations?

If the response to all of the following questions is “yes,” then GRADE has been used appropriately at the outcome-level. If any of these criteria are not met, then GRADE was not used appropriately.

• Are all five GRADE domains assessed?

• Do the SR authors refrain from using the criteria for rating up?

• Are explanations provided for all domains that are downgraded?

• Are all the explanations for the downgraded domains informative?^b [15]

    ◦ For study limitations, do the authors indicate the proportion of studies that were at a concern for high risk of bias or the specific RoB assessment criteria that was of most concern?

    ◦ For imprecision, do the authors indicate whether the sample size or number of events was too low or whether the bounds of the CI have different meanings based on thresholds for the optimal information size or the effect size, respectively?

    ◦ For inconsistency, do the authors indicate how heterogeneity was judged (e.g., confidence interval overlap, statistical tests)?

    ◦ For indirectness, do the authors indicate whether it was the population, intervention, comparator, or outcome of the included RCTs that does not align with the SR question and is therefore a reason for concern?

    ◦ For publication bias, do the authors indicate the reason to suspect publication bias (e.g., funnel plot, suspected selective reporting)?

• If the primary outcome is dichotomous, do the SR authors transform relative estimates of effects to absolute estimates in order to assess imprecision?

• For the GRADE domains which were downgraded, is there evidence that the SR authors assessed the domains using the incorrect criteria (e.g, referring to the criteria for indirectness in the explanation for rating down imprecision, creating concerns for whether imprecision and indirectness were appropriately assessed)?