Table 4 Well Living House quality appraisal tool
Category | Questions | Scoring criteria |
|---|---|---|
Local community relevance of method and measures /4 | 1. Did the measures of success reflect local Indigenous community understandings of success? | (Yes = 2, Partial = 1, No = 0) Yes: evidence provided explicitly in the text (look for: where did evaluation take place, who collected evaluation data?) Partial: hints of including local community values/beliefs/knowledge systems in text and therefore assumption made by reviewers that evidence is present No: nothing was said or author(s) indicated that success was not defined by the community. |
2. Had methods and tools been tested and validated previously in a similar Indigenous context and reviewed for relevance by appropriate community members? | (Yes = 2, Partial = 1, No = 0) Yes: evidence is provided explicitly in text Partial: hints of using a tool that has been used in Indigenous contexts and therefore assumption made by reviewers that evidence is present No: nothing was said or author(s) said that the evaluation method/tool has not been used before in Indigenous contexts. | |
Rigour and internal validity of the evaluation method /4 | 3. Do the quantitative or qualitative methods meet relevant rigour and internal validity? | (Excellent = 4, Fair = 3, Barely Acceptable = 2, Poor = 1) Is the study design appropriate for evaluation research question(s)? Are the conclusions supported and justified by the results? Quantitative Is the sample size described and justified? Are the instruments/tools already validated? Are threats to validity addressed (such as confounding factors)? Qualitative Are the participants selected using appropriate strategies (such as purposive sample or until saturation is reached)? Is there accurate and adequate documentation of the KT effects described? |
Strength of the evidence /4 | 4. Is the evidence strong? | (Excellent = 4, Fair = 3, Barely Acceptable = 2, Poor = 1) Quantitative Does the evidence have adequate power and statistical significance? Is the response rate reasonable? Qualitative Are there major and convincing themes from triangulation, and/or member checking? |