Skip to main content

Table 4 Risk of bias assessment

From: Do wastewater treatment plants increase antibiotic resistant bacteria or genes in the environment? Protocol for a systematic review

Bias domainAssessment questionCriteria
Sample selection biasWere sample locations and sampling methods implemented such that sampling did not introduce systematic differences depending on the value of the exposure variable for each sample (in the case of continuous exposure data) or between the comparison groups (in the case of categorical exposure measures)?1. Criteria for the judgement of “Yes”:
• Method for determining the sampling locations is identical independent of exposure status (i.e. distance or direction from source);
• Restriction of sampling locations is applied in the same way regardless of exposure status (e.g. sampling sites are all agricultural fields with a similar type and level of historical use);
• Time between sampling at all sites is sufficiently close so as to render the outcomes measured at these sites comparable for the sample type in question
2. Criteria for the judgement of “No”:
• Sampling locations are selected differently;
• Restriction of sample locations is applied differently depending on exposure status
3. Risk of bias will be considered “unclear” if there is not enough information to judge sample selection bias criteria as either “yes” or “no”, e.g. if methods for determining sampling locations are not described in enough detail
Information bias“Were outcome ascertainment methods (i.e. methods of gene or bacterial measurement) conducted in a way that ensures the same accuracy regardless of distance or direction from the source(s)?”1. Criteria for the judgement of “Yes”:
• Identical microbiological methods applied to all samples;
• Controlling for laboratory factors, if these are different (e.g. which laboratory, technician, testing date, instrument used);
• Blinding laboratory staff to exposure status
2. Criteria for the judgement of “No”:
• Application of different methods depending on comparison group;
• No adjustment strategy for different laboratory methods
3. Risk of bias will be considered “unclear” if there is not enough information to judge information bias criteria as either “yes” or “no”, e.g. if methods for analyses are not explained sufficiently to reach a judgement
Confoundinga“Were adequate methods to control for potential confounding employed?”1. Criteria for the judgement of “Yes”:
• Restriction of the sample population;
• Analytical confounding control (e.g. stratification, regression adjustment)
2. Criteria for the judgement of “No”:
• Lack of any confounding control despite confounding being likely;
• Inappropriate method of confounding control used;
• Controlling for confounding is correctly applied for some potential confounders, but not for all
3. Risk of bias will be considered “unclear” if there is not enough information to judge information bias criteria as either “yes” or “no”, e.g. if methods to control for confounding are mentioned but the implementation is not explained sufficiently at length to reach a judgement
  1. aSome potential confounders for measuring antibiotic resistant bacteria and genes in environmental samples such as air and water samples include varying bacterial population size across sampling locations, environmental media composition (e.g. water salinity), recent precipitation and other weather events, sample composition and other sources of antibiotics or antibiotic resistance factors [18]