Skip to main content

Table 3 Comparison of domains between NOS and ROB-NRSE

From: Inter-rater reliability and validity of risk of bias instrument for non-randomized studies of exposures: a study protocol

NOS

ROB-NRSE

Degree of overlap

Comparability

C:Comparability of cohorts on the basis of the design or analysis

Bias due to confounding

1.1: Is there potential for confounding of the effect of exposure in this study?

Unique

 

1a: Study controls for the most important factor

 

1.2: Was the analysis based on splitting follow up time according to exposure received?

Unique

 

1b: Study controls for additional factor

 

1.3. Were exposure discontinuations or switches likely to be related to factors that are prognostic for the outcome?

Unique

  

Baseline confounding only

1.4:Did the authors use an appropriate analysis method that controlled for all the important confounding domains?

Complete overlap

   

1.5: Were confounding areas that were adjusted for measured validly and reliably by the variables available in this study?

Unique

   

1.6. Did the authors avoid adjusting for post-exposure variables?

Unique

  

Time-varying confounding only

1.7: Did the authors use an appropriate analysis method that adjusted for all the critically important confounding areas and for time-varying confounding?

Unique

   

1.8: Were confounding areas that were adjusted for measured validly and reliably by the variables available in this study?

Unique

Selection

S1: Representativeness of exposed cohort

Bias in selection of participants into the study

2.1: Was selection of participants into the study (or into the analysis) based on variables measured after the start of the exposure?

Unique

 

1a: Truly representative

 

2.2: Were the post-exposure variables that influenced selection associated with exposure?

Unique

 

1b: Somewhat representative

 

2.3: Were the post-exposure variables that influenced eligibility selection influenced by the outcome or a cause of the outcome?

Unique

 

1c: Selected group of users

 

2.4: Do start of follow-up and start of intervention coincide for most participants?

Unique

 

1d: No description of the derivation of the cohort

 

2.5: Were adjustment techniques used that are likely to correct for the presence of selection biases?

Unique

 

S2: Selection of non-exposed cohort

   
 

2a: Drawn from the same community as the exposed cohort

   
 

2b: Drawn from a different source

   
 

2c: No description of the derivation of the non-exposed cohort

   

Selection

S3: Ascertainment of exposure

Bias in classification of Exposures

3.1: Is exposure status well defined?

Unique

 

3a: Secure record

 

3.2: Did entry into the study begin with start of the exposure?

Unique

 

3b: Structured interview

 

3.3:Was information used to define exposure status recorded prior to outcome assessment?

Partial overlap

 

3c: Written self-report

 

3.4:Could classification of exposure status have been affected by knowledge of the outcome or risk of the outcome?

Partial overlap

 

3d: No description

 

3.5: Were exposure assessment methods robust (including methods used to input data)?

 
 

S4: Demonstration of outcome of interest was not present at start of the study

   
 

4a: Yes

   
 

4b: No

   
  

Bias due to departures from intended exposures

4.1: Is there concern that changes in exposure status occurred among participants?

Unique

   

4.2: Did many participants switch to other exposures?

Unique

   

4.3: Were the critical co-exposures balanced across exposure groups?

Unique

   

4.4: Were adjustment techniques used that are likely to correct for these issues?

Unique

Outcomes

O1: Was follow-up long enough for outcomes to occur (Yes/No)

Bias due to missing data

5.1:Were there missing outcome data?

Partial overlap

 

O3: Adequacy of follow-up of cohorts

 

5.2: Were participants excluded due to missing data on exposure status?

Unique

 

3a: Complete follow-up—all subjects accounted for

 

5.3: Were participants excluded due to missing data on other variables needed for the analysis?

Unique

 

3b: Subjects lost to follow up unlikely to introduce bias -small number lost

 

5.4:Are the proportion of participants and reasons for missing data similar across exposures?

Partial overlap

 

3c: Follow up rate large (%) and no description of those lost

 

5.5: Were appropriate statistical methods used to account for missing data?

Unique

 

3d: No statement

   

Outcomes

O2: Assessment of outcome

Bias in measurement of outcomes

6.1:Could the outcome measure have been influenced by knowledge of the exposure received?

Partial overlap

 

2a: Independent blind assessment

 

6.2: Was the outcome measure sensitive?

Unique

 

2b: Record linkage

 

6.3:Were outcome assessors unaware of the exposure received by study participants?

Partial overlap

 

2c: Self report

 

6.4:Were the methods of outcome assessment comparable across exposure groups?

Partial overlap

 

2d: No description

 

6.5: Were any systematic errors in measurement of the outcome unrelated to exposure received?

 
  

Bias in selection of the reported result

7.1: Is the reported effect estimate likely to be selected, on the basis of the results, from multiple outcome measurements within the outcome domain?

Unique

   

7.2: Is the reported effect estimate likely to be selected, on the basis of the results, from multiple analyses of the exposure-outcome relationship?

Unique

   

7.3: Is the reported effect estimate likely to be selected, on the basis of the results, from different subgroups?

Unique