Table 3 Development of guidance
From: Consideration of health inequalities in systematic reviews: a mapping review of guidance
Guidance | Aim/audience | Development method | Operationalisation | Strengths/limitations |
|---|---|---|---|---|
HI-focused guidance | ||||
Doull et al. [22] Welch V., et al. for the sex/gender methods group. [23] Puil L., et al. for the sex/gender methods group. [24] Doull M., et al. for the sex/gender methods group. [25] | To translate knowledge about sex/gender analysis into a user-friendly briefing note format and evaluate its use in aiding in the implementation of sex/gender analysis in systematic reviews. Aimed at reviewers and editors of Cochrane Aids/ Hypertension and Musculoskeletal Groups | Guidance development process • Informed by literature reviews •Built on existing structured guidance for systematic reviewers • Feedback and revision sought • Evaluated by 19 participants attending a workshop at the 2012 Canadian Cochrane Symposium • Underpinned by diffusion of innovations theory Who else was involved: Members of the Cochrane Collaboration HIV/AIDS, Hypertension, and Musculoskeletal Review Groups with expertise in methodology, sex/gender analysis, systematic reviews, policy and knowledge translation and additional clinical experts | Four sections: Sections 1–3 define the issue, definitions and rationale for considering sex and gender analysis. Section 4 has 13 items to consider in relation to sex and gender (question formulation; context; population; intervention/ comparator; outcomes; study design; searching for studies; data collection; risk of bias; data analysis; additional analyses; presenting results and summary of findings; interpreting and drawing conclusions). Topic specific descriptive examples provided | Strengths: • Wide range of expertise and systematic review experience involved in development • Consensus-based •provides rationale, evidence and examples to operationalise guidance • Piloted and evaluated Open access Limitations: • Evaluated by a self-selecting group attending Cochrane Meeting. •Terminology used (logic model, context) may not be widely accepted or understood |
NIHR CLAHRC North West Coast [26] | To help ensure that all activities of the NIHR CLAHRC NWC have potential to contribute to reducing health inequalities. Aimed specifically at anyone undertaking CLAHRC NWC work (including reviewers) but also anyone wishing to consider HI in their research | Guidance development process: Collaborative process in a series of workshops in 2014/2015 Who was involved: NIHR CLAHRC NWC staff and partners | Four sections: 1. Clarifying the health inequality dimensions of the problem 2. Designing your intervention/action 3. Evaluating and/or monitoring the impact of your activity 4. Planning for wider impacts on health inequalities Included 26 questions Each section also includes a Health Inequalities Assessment of an exemplar proposal for applied research. Links to resources that provide more information about the issues covered in each section. Guidance provided on how to use HIAT. | Strengths: • Revised after feedback from users, plan to revise regularly after further user feedback. encourages involvement of the public/team approach in considering equity in reviews. •Addition of further resources •Worked example provided •open access Limitations: • Limited information on how the guidance was developed or tested. • Long checklist • Worked example is not a systematic review therefore further details on how reviewers can operationalise individual items is required |
O’Neill et al. [12] | To assess the utility of an acronym, place of residence, race/ ethnicity/culture/language, occupation, gender/ sex, religion, education, socioeconomic status, and social capital (“PROGRESS”) to guide the conceptualization of disadvantage, data extraction, and to inform equity analyses in systematic reviews. Aimed at reviewers, researchers and users | Authors demonstrate how an existing framework PROGRESS, the framework for the PRISMA Equity Extension, can be applied to systematic reviews | Asks reviewers to consider variations in health across 8 factors: place of residence, race/ethnicity/culture/ language, occupation, gender/sex, religion, education, socioeconomic status and social capital. For each PROGRESS factor, examples are provided that demonstrate differences in burden of disease and an effective intervention that could reduce that burden. | Strengths: • Considers multiple equity dimensions Limitations: • Limited examples provided further detail on how reviewers can operationalise individual items is required • Not evaluated • Not open access |
Nasser et al. [27] | To develop and pilot an equity lens to help researchers develop a more equity-oriented approach toward priority setting and agenda setting in systematic reviews Aimed at reviewers. | Development process • A workshop presenting survey results from a previous project • Literature review • Workshop for refinement of the equity lens • Piloted • Underpinned by conceptual framework for priority setting Who was involved: 15 people attending the 2008 Cochrane Colloquium attended the first workshop, 12 attending the 2009 Cochrane Collaboration attended the second workshop | Two checklists: 1. 9 questions assessing priority setting, from identifying the questions and stakeholders to the evaluation strategy. 2. 8 questions assessing the outcome evaluation of priority setting | Strengths: • Piloted Limitations: • Evaluated by a self-selecting group attending Cochrane Meeting • Not open access |
Oxman et al. [28] | To present a structured approach to considering the impacts of policy and programme options on inequities, to inform decisions about what options to implement and how to implement them. Aimed at users | Not reported | 4 questions that can be used to guide considerations when using systematic reviews regarding impacts on inequities. | Strengths: • Descriptive examples provided • Open access Limitations: • No information available on how the guidance was developed or evaluated. • Terminology used may not be widely accepted or understood • Greater detail required on how reviewers can operationalise the items |
Tugwell et al. [29] Ueffing E, et al. for the Campbell and Cochrane Equity Methods Group. [30] | To provide guidance on assessing equity for users and authors of systematic reviews of interventions. Aimed at reviewers, users and journal editors. | Development process: • 4 working sessions • Built on previous work by the members of the Measurement and Evidence Knowledge Network • Panel members reviewed the evidence and drafted guidance • Feedback and revision sought Who was involved: International leaders in systematic reviews and health equity, mixed methods experts, social scientists, economists, experts in systematic reviews, experts in public health and health equity, experts from low and middle income countries and policy advisers who use systematic reviews. Members of the Campbell and Cochrane Equity Methods Group and the Measurement and Evidence Knowledge Network | 7 recommendations underpinned by 16 checklist items. Examples provided | Strengths: • Wide range of expertise involved in development • Consensus-based •Descriptive examples provided •Addition of resources to signpost reviewers to sources of help when attempting to answer the questions. Limitations: • Terminology used may not be widely accepted or understood • Greater detail required on how reviewers can operationalise the items |
Tugwell et al. [31] | “Propose an evidence based framework – or “cascade” – for equity-orientated knowledge translation.” Aimed at reviewers, researchers and users | Development process: Not reported Who was involved: Not reported | 5 steps Examples demonstrate how the steps are applied to 2 systematic reviews | Strengths: • Descriptive examples provided to operationalise items • Open access Limitations: • No information available on how the guidance was developed or tested. • Does not define equity |
Welch et al. [32] Welch et al. [33] Burford et al. [34] | “To provide structured guidance on transparently reporting methods and results for equity focused reviews. To legitimise and emphasize the importance of reporting health equity results.” Aimed at reviewers | Development process: • Consensus-based - Followed guidance for developing reporting guidelines • Identifying need • Reviewing the literature (systematic review and methodological study) • Gathering expert opinion (online survey) • Exploring consensus • Piloting Who was involved: • Equity researchers, decision-makers, clinical epidemiologists, systematic review methodologists, journal editors, funders, practitioners, review authors with LMIC focus, methodologists/statisticians, novice systematic reviewers and established systematic reviewers involved with equity and/or complex population intervention systematic reviews | 14-item equity extension of existing guidance for the reporting of systematic reviews. Provides detailed rationale, evidence, whenever available, an exemplar for recommending each item and examples of good practice. | Strengths: • Wide range of expertise involved in development • Involved non-expert reviewers in development • Consensus-based • Followed guidance on developing reporting guidelines • Provides rationale, evidence, exemplars and examples to operationalise items • Evaluated • Open access Limitations: • Terminology used may not be widely accepted or understood • Greater detail required on how reviewers can operationalise some items, e.g. approach to logic model |
Welch et al. [35] | To develop and assess inter-rater agreement for an algorithm for systematic review authors to predict whether differences in effect measures are likely for disadvantaged populations relative to advantaged populations. Aimed at reviewers. | Development process: • Follows established methods of checklist development • Review of existing guidance • Systematic review of methods for assessing effects on health equity • Survey of practitioners/managers • Evaluated face and conceptual validity amongst four clinical methodologists • Inter-rater reliability assessed amongst 35 methodologists, clinicians, users of SRs assessed the algorithm against a pre-selected sample of 10 SRs. • Piloted Who was involved: Authors, practitioners/managers, clinicians, methodologists, users, members of Cochrane Collaboration | 3 questions. Examples operationalise how each of the questions may result in differential effects | Strengths: • Wide range of expertise involved in development • Follows established methods of checklist development • Descriptive examples provided to operationalise items • Evaluated • Open access Limitations: • Low inter-rater reliability • Tested by individuals rather than review teams who evaluated the algorithm against summarised information from the reviews • Subject expertise of the raters is unclear, this may have impacted on whether they would anticipate differential effects. • Multi-component questions cover several factors |
Welch et al. [36] | To provide guidance on how to conduct equity-focused systematic reviews consistent with the recommendations of PRISMA-E 2012 to facilitate the use of both guidance documents. This article also discusses challenges related to knowledge translation for equity-focused systematic reviews. Aimed at reviewers. | Development process: • Series of methodology meetings • Systematic review of methods to assess equity in systematic reviews • Methods study • WHO Task force on evidence-informed policies about health systems • PRISMA-Equity (2012) guidance Who was involved:Campbell and Cochrane Equity Methods Group, Cochrane Public Health Review Group, methodologists, funders, journal editors, clinicians and public health practitioners | 10 steps to considering health equity in reviews. Recommendations with a few brief examples from exemplar reviews | Strengths: • Wide range of expertise involved in development • Descriptive examples provided to operationalise items • Open access Limitations: • Terminology used may not be widely accepted or understood • Greater detail required on how reviewers can operationalise the items |
Generic focused guidance | ||||
Armstrong R, Waters E, Doyle J (editors) [37] Chapter 21: Reviews in health promotion and public health. In Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. Rebecca Armstrong, Elizabeth Waters on behalf of the Guidelines for Systematic Reviews in Health Promotion and Public Health Taskforce. [38] Systematic Reviews of health promotion and public health Interventions. | Guidance to authors for the preparation of Cochrane Intervention reviews (including Cochrane Overviews of reviews). Aimed at reviewers. | Not reported | N/Aa | |
Chambers and Wilson [39] | To enable researchers to present and contextualize evidence from systematic reviews and other sources of synthesized and quality-assessed evidence. Aimed at researchers. | Uses the Oxman et al. [28] criteria | 4b | Strengths: • As above for Oxman et al. [28] • Authors offer advice on operationalising guidance in absence of evidence in reviews, “by information gathered locally, using documents produced by or relevant to the NHS, such as Joint Strategic Needs Assessments and equity audits.” Limitations: As above for Oxman et al. [28] |
CRD [40] | To promote high standards in commissioning and conduct, by providing practical guidance for undertaking systematic reviews evaluating the effects of health interventions. Aimed at reviewers. | Not reported | N/Ac | |