Table 4 Summary of outstanding queries for Domain 2 and 4 of ROBINS-I tool

Domain 2: selection of participants into the study

• Further guidance on the distinction between SQ2.2 and SQ2.3, (“Were the post-intervention variables that influenced selection: likely to be associated with the intervention (SQ2.2); OR likely to be influenced by the outcome or a cause of the outcome (SQ2.3)”. For analysis relying on post hoc classification of intervention status, this may be difficult to assess but may be a critical source of bias. Also, for interventions that address socio-economic determinants of health, it is highly likely that selection to receive the intervention will be on variables such as income or other measures of socio-economic deprivation, which is also a determinant of the outcome, health.

• Start of intervention coinciding with start of follow-up: clarification on how this should be assessed for studies where a baseline assessment of the outcome was made before the participants received the intervention and then at follow-up after the intervention.

• Can SQ2.4 (Do start of follow-up and start of intervention coincide for most participants?) be applied to non-event type outcomes?

• Clarification of whether variation in lengths of exposure to the intervention at follow-up could contribute to selection bias.

Domain 4: deviation from intended interventions

Effect of interest

• Can analysis of post hoc classification of intervention status be interpreted as per protocol?

• Clarification of question “If your aim for this study is to assess the effect of initiating and adhering to the intervention (as in a per protocol analysis)”. Does this relate to the aim for the review as agreed in the review protocol characteristics or the aim of the analysis used in the study being assessed?

Implementation and adherence

• Clarification about what is meant by successful implementation (SQ 4.4) and how authors should decide the level at which implementation failure (SQ 4.4) and adherence (SQ4.5) is assessed (see Table 5).

• How should interventions which are tailored to individual need be assessed?

• Can it be assumed that there is no implementation failure where classification of intervention status is post hoc? By definition, all those in the intervention group may be assessed to have received the intervention, but this will be dependent on the level at which the intervention is being assessed (see Table 5).

Co-interventions

• Clarification about what constitutes a co-intervention (see Table 5).

• Clarification about when a co-intervention should be considered to be “important” (SQ4.3). Should there be an established association with the outcome?

Contamination and switching

• Clarification about the meaning when “contamination” constitutes “switching”, especially in cases where classification of intervention status is post hoc. Guidance implies that contamination is inadvertent but this is difficult to determine in studies, see page 35 of ROBINS-I guidance.