Table 3 Adequacy criteria for measurement properties adapted from [21] and [27]
Property | Rating | Adequacy criteria |
|---|---|---|
Reliability | Â | Â |
Internal consistency (CTT methods applied) | + | Cronbach’s alpha(s) ≥0.70 |
? | Cronbach’s alpha not determined | |
− | Cronbach’s alpha(s) <0.70 | |
Internal consistency (IRT methods applied) | + | Person Separation Index ≥0.70 |
? | Person Separation Index not determined | |
− | Person Separation Index <0.70 | |
Measurement error | + | MIC > SDC OR MIC outside the LoA |
? | MIC not defined | |
− | MIC ≤ SDC OR MIC equals or inside LoA | |
Reliability | + | ICC/weighted Kappa ≥0.70, OR Pearson’s r ≥ 0.80 |
? | Neither ICC/weighted Kappa, nor Pearson’s r determined | |
− | ICC/weighted Kappa <0.70 OR Pearson’s r < 0.80 | |
Validity | Â | Â |
Content validity | + | All items are considered to be relevant for the construct to be measured, for the target population, and for the purpose of the measurement AND the questionnaire is considered to be comprehensive |
? | Not enough information available | |
− | Not all items are considered to be relevant for the construct to be measured, for the target population, and for the purpose of the measurement OR the questionnaire is considered not to be comprehensive | |
Construct validity | Â | Â |
Structural validity (CTT methods applied) | + | Factors should explain at least 50Â % of the variance |
? | Explained variance not mentioned | |
− | Factors explain <50 % of the variance | |
Structural validity (IRT methods applied) | + | Residual correlations among the items after controlling for the dominant factor <0.20 OR Q3’s <0.37, item scalability >0.30, IRT model fit: G2 >0.01, no DIF for important subject characteristics (such as age, gender, education): McFadden’s R 2 <0.02, OR no non-uniform DIF |
? | Important statistics not reported | |
− | Residual correlations among the items after controlling for the dominant factor ≥0.20 OR Q3’s ≥0.37, item scalability ≤0.30, IRT model fit: G2 ≤0.01, important DIF for important subject characteristics (such as age, gender, education): McFadden’s R 2 ≥0.02, OR non-uniform DIF | |
Hypothesis testing (convergent/divergent validity) | + | Correlations with instruments measuring the same construct ≥0.50 OR at least 75 % of the results are in accordance with the hypotheses AND correlation with related constructs is higher than with unrelated constructs |
? | Solely correlations determined with unrelated constructs | |
− | Correlations with instruments measuring the same construct <0.50 OR <75 % of the results are in accordance with the hypotheses OR correlation with related constructs is lower than with unrelated constructs | |
Hypothesis testing (discriminative validity) | + | Differences in scores on the measurement instrument for all evaluated patient subgroups are statistically significant OR ≥75 % of results in accordance with hypotheses |
? | Some differences statistically significant, others not | |
− | Differences in scores on the measurement instrument for all evaluated patient subgroups are not statistically significant OR <75 % of results in accordance with hypotheses | |
Cross-cultural validity | + | No differences in factor structure OR no important DIF between language versions |
? | Multiple group factor analysis not applied AND DIF not assessed | |
− | Differences in factor structure OR important DIF between language versions | |
Responsiveness | Â | Â |
Responsiveness | + | Correlation with changes on instruments measuring the same construct ≥0.50 OR at least 75 % of the results are in accordance with the hypotheses OR AUC ≥0.70 AND correlations with changes in related constructs are higher than with unrelated constructs |
? | Solely correlations determined with unrelated constructs | |
− | Correlations with changes on instruments measuring the same construct <0.50 OR <75 % of the results are in accordance with the hypotheses OR AUC <0.70 OR correlations with changes in related constructs are lower than with unrelated constructs |