Measurement property | Rating | Criteria |
---|---|---|
Structural validity | + | Classical Test Theory (CTT) Confirmatory factor analysis: Comparative Fit Index or Tucker Lewis Index or comparable measure > 0.95 OR root mean square error of approximation < 0.06 OR standardised root mean residuals < 0.08 Item Response Theory (IRT)/Rasch No violation of unidimensionality: Comparative Fit Index or Tucker Lewis Index or comparable measure > 0.95 OR root mean square error of approximation < 0.06 OR standardised root mean residuals < 0.08 AND No violation of local independence: residual correlations among the items after controlling for the dominant factor < 0.20 OR Q3’s < 0.37 AND No violation of monotonicity: adequate looking graphs OR item scalability > 0.30 AND Adequate model fit: IRT: χ2 > 0.001 Rasch: infit and outfit mean squares ≥ 0.5 and ≤ 1.5 OR Z-standardized values > − 2 and < 2 |
? | CTT: not all information for ‘+’ reported IRT/Rasch: model fit not reported | |
− | Criteria for ‘+’ not met | |
Internal consistency | + | At least low evidence for sufficient structural validity AND Cronbach’s alpha(s) ≥ 0.70 for each unidimensional scale or subscale |
? | Criteria for ‘At least low evidence for sufficient structural validity’ not met | |
− | At least low evidence for sufficient structural validity AND Cronbach’s alpha(s) < 0.70 for each unidimensional scale or subscale | |
Reliability | + | Interclass Correlation Coefficient (ICC) or weighted kappa ≥ 0.70 |
? | ICC or weighted kappa not reported | |
− | ICC or weighted kappa < 0.70 | |
Measurement error | + | Smallest detectable change (SDC) or limits of agreement (LoA) < minimal important change (MIC) |
? | MIC not defined | |
− | SDC or LoA > MIC | |
Hypotheses testing for construct validity | + | The result is in accordance with the hypothesis |
? | No hypothesis defined (by the review team) | |
− | The result is not in accordance with the hypothesis | |
Cross-cultural validity/measurement invariance | + | No important differences found between group factors (e.g. age and language) in multiple group factor analysis OR no important differential item functioning (DIF) for group factors (McFadden’s R < 0.02) |
? | No multiple group factor analysis OR DIF analysis performed | |
− | Important differences between group factors OR DIF was found | |
Criterion validity | + | Correlation with gold standard ≥ 0.70 OR area under the curve (AUC) ≥ 0.70 |
? | Not all information for ‘+’ reported | |
− | Correlation with gold standard < 0.70 OR AUC < 0.70 | |
Responsiveness | + | The result is in accordance with the hypothesis OR AUC ≥ 0.70 |
? | No hypothesis defined (by the review team) | |
− | The result is not in accordance with the hypothesis OR AUC < 0.70 |