Criterion | Example of low risk of bias | Example of high risk of bias |
---|---|---|
Exchangeability of the control and intervention groups | The study is randomized. | The study is observational with uncontrolled self-selection into the intervention group (e.g., inducing confounding by a pre-existing interest in dietary change). |
Proximity of the outcome measure to actual meat consumption or purchase | The study measures meat consumption using subjects’ actual food choices in a cafeteria. | The study measures subjects’ intended meat consumption. |
Missing data | Nearly all enrolled subjects completed the intervention and provided outcome measures. | Many subjects failed to complete the intervention or were lost to follow-up before the outcome was measured. |
Minimization of social desirability biases and demand characteristics | The intervention was subtly embedded in a decoy task about a topic unrelated to meat consumption, leading subjects to believe the study was not about meat consumption. | Subjects interact with experimenters who are clearly identifiable as animal welfare advocates. |
Potential for selective reporting | The study was preregistered. | The study was not preregistered |
Analytic reproducibility | The study has publicly available data, materials, and code. | The study does not have publicly available data, materials, or code. |