Skip to main content


Consideration of health inequalities in systematic reviews: a mapping review of guidance



Given that we know that interventions shown to be effective in improving the health of a population may actually widen the health inequalities gap while others reduce it, it is imperative that all systematic reviewers consider how the findings of their reviews may impact (reduce or increase) on the health inequality gap. This study reviewed existing guidance on incorporating considerations of health inequalities in systematic reviews in order to examine the extent to which they can help reviewers to incorporate such issues.


A mapping review was undertaken to identify guidance documents that purported to inform reviewers on whether and how to incorporate considerations of health inequalities. Searches were undertaken in Medline, CINAHL and The Cochrane Library Methodology Register. Review guidance manuals prepared by international organisations engaged in undertaking systematic reviews, and their associated websites were scanned. Studies were included if they provided an overview or discussed the development and testing of guidance for dealing with the incorporation of considerations of health inequalities in evidence synthesis. Results are summarised in narrative and tabular forms.


Twenty guidance documents published between 2009 and 2016 were included. Guidance has been produced to inform considerations of health inequalities at different stages of the systematic review process. The Campbell and Cochrane Equity Group have been instrumental in developing and promoting such guidance. Definitions of health inequalities and guidance differed across the included studies. All but one guidance document were transparent in their method of production. Formal methods of evaluation were reported for six guidance documents. Most of the guidance was operationalised in the form of examples taken from published systematic reviews. The number of guidance items to operationalise ranges from 3 up to 26 with a considerable overlap noted.


Adhering to the guidance will require more work for the reviewers. It requires a deeper understanding of how reviewers can operationalise the guidance taking into consideration the barriers and facilitators involved. This has implications not only for understanding the usefulness and burden of the guidance but also for the uptake of guidance and its ultimate goal of improving health inequalities considerations in systematic reviews.


Health inequalities are avoidable and unjust differences in health between individuals or populations [1]. Given that we know that interventions shown to be effective in improving the health of a population may actually widen the health inequalities gap while others reduce it, it is imperative that systematic reviewers consider how the findings of their reviews may impact (reduce or increase) the health inequality gap [24]. Furthermore, the existence of social inequalities, defined as “systematic differences in health between different socioeconomic groups within a society” (1, p. 473, [5]), increases the argument for all reviewers, not just those with a focus on health inequalities (HI), to consider the potential for their findings to reduce or increase HI. This is echoed by Cochrane and Campbell Collaborations who have called for the effects of interventions on HI to be considered by systematic review authors [6, 7].

Incorporating considerations of how review findings impact on health inequalities also aims to overcome one of the major barriers in using systematic reviews to inform decision-making, policy-making and practice [8]. Moving away from simply assessing what works to considering how review findings impact on disadvantaged populations, by assessing for example differential effects by subgroup populations can improve the applicability of the review findings to the local population[9, 10] thus increasing their fit for purpose in supporting decision-making and practice. Going beyond simply does it work to examine under what circumstances it works for whom and why [11] holds even more resonance when considering the impact on HI. As O’Neill et al. ([12], p. 57) point out, “the intervention has to be accessible, acceptable, effective in, and used by the most disadvantaged group within that population to be truly effective at reducing inequities in health” (whilst health inequalities are defined as avoidable differences in health outcomes across individuals or between populations, the narrower but related term health equity is often referred to in the literature as health inequalities which are also considered unfair and unjust [13].)

Methodological research has highlighted an absence of evidence with regards to the extent to which systematic reviews taking into account issues of HI when analysing and making recommendations for further research and practice [3, 14, 15]. Furthermore, recent methodological studies of systematic reviews demonstrated that very few (<5%) addressed differential impacts across socio-economic groups [16, 17]. The extent to which systematic reviewers in the past have failed to consider how their review findings impact on HI, is due in part to the focus reviewers placed on the effectiveness of interventions and also due to a lack of relevant data reported in the primary literature to assess such differential effects [18]. In addition, the lack of guidance or awareness of the existence of such guidance in this area may also have worsened the situation. More importantly, however, it is also due to a failure on the part of review authors to even consider differential impacts in reviews where HI is not the focus [8, 17, 19].

This paper aims to review existing guidance on incorporating considerations of HI in systematic reviews to examine the extent to which they can help reviewers incorporate such considerations in systematic reviews.

Aim and objectives

The aim of this study was to undertake a mapping review of existing guidance documents currently provided to assist reviewers when determining whether and how to incorporate considerations of HI. A mapping review aims to map out and categorise the literature according to key features (e.g. study design) on a particular topic and to identify gaps in the research literature [20]. The objectives were (1) to provide an overview on the types of guidance, in particular the focus, scope and purpose of the guidance, (2) to explore how the guidance is defined by authors, (3) to describe the methods used to develop the guidance, (4) to examine the comprehensiveness, overlap and operationalisation of the guidance.

Search strategy

A systematic approach to identifying the literature was undertaken in a two-tiered approach. Firstly, more generic guidelines to evidence synthesis were located and searched to identify specific guidance relating to the incorporation of HI. A search of review guidance manuals prepared by international organisations engaged in undertaking evidence synthesis was undertaken (see Table 1). Publications listed on the Campbell and Cochrane Equity Group website were also scanned.

Table 1 International evidence synthesis organisation websites

In addition to this, a search was undertaken in Medline, CINAHL and The Cochrane Methodology Register. Key terms searched on included thesaurus and textwords terms comprising of synonyms for health inequalities, evidence synthesis and methodology/guidance/tools (see Appendix 1). A pre-published search strategy designed to capture HI studies was reviewed and utilised [21]. A practical approach to developing a search strategy to identify different types of evidence synthesis was adopted. This approach was informed by published systematic review filters and related evidence synthesis terms (e.g. realist review, realist synthesis, integrative). Searches were undertaken in September 2015, and search alerts were set up to capture relevant articles added into the databases after this date. No restrictions by year were applied, but publications were limited to English language only studies.

Requests for guidance were made via relevant email discussion lists and contacting experts in the Campbell and Cochrane Equity Group. For guidance that had been updated, the most recent update was included. Where multiple publications discuss the same guidance they were considered together.

Inclusion criteria

Any type of study was included if it provided an overview or discussed the development and testing of a conceptual or practical framework, tool, model or guidance (advice or formal recommendations) for dealing with the incorporation of considerations of HI in evidence synthesis. For the purpose of this review, HI is defined according to Whitehead’s ([1], p. 473) definition in which inequalities in the British context—and increasingly also across Europe—carries the same connotations of unfairness and injustice as the term inequities. Both generic guidance (e.g. guidelines published by evidence synthesis organisations or collaborations) that incorporated considerations of HI and HI-specific guidance (e.g. scholarly methodological studies presenting specific guidance for incorporating considerations of HI in systematic reviews) were included. Studies that primarily offer a theoretical discussion or comments on if and why HI should be included in evidence synthesis, or frameworks and guidance for the incorporation of HI for purposes other than evidence synthesis, were excluded. For practical reasons, studies were limited to English language publications. Screening of studies was undertaken by the author.

Data extraction

Data were extracted by the author on targeted audience (e.g. for reviewers, users of evidence synthesis), purpose (e.g. tools are to be used for planning, conducting, reporting, disseminating or using systematic reviews), scope (e.g. to inform systematic reviews, other research), how the tool was developed, operationalisation of the tool (how reviewers are instructed to apply the guidance) and whether and how they define health inequalities. Included studies were also categorised as to their focus, i.e. whether they were HI, or had a generic focus but which accounted for HI.

Data synthesis

Results are summarised in narrative and tabular forms. Strengths and weaknesses of the guidance in assisting systematic reviewers were assessed based on methods of development including evaluation, accessibility and transparency in operationalisation.


The results of the search are summarised in Fig. 1. Eight hundred and thirty-six references were identified of which 20 were included in the review. All guidance documents [12, 2240] incorporating considerations of HI in systematic reviews were published between 2009 and 2016. Table 2 outlines the characteristics of the included studies. Doull et al. [22] report on three guidance documents within the same study [2325]. Tugwell et al. [29] and Ueffing et al. [30] both discuss the Cochrane Equity Checklist, whilst Welch et al. [32, 33] and Burford et al. [34] report on the PRISMA-Equity 2012 Extension. The majority of the guidance has been produced with the involvement of members of the Campbell and Cochrane Equity Methods Group.

Fig. 1

Flow of search results

Table 2 Characteristics of included studies

Focus, scope and purpose of guidance

Table 2 outlines the focus, scope and purpose of the guidance. The majority of the guidance documents have an HI focus; three focus on sex and gender [2325], one on socio-economic status (SES) [26] and one on both sex and SES analysis in systematic reviews [35] All four generic guidance documents in which HI was considered, signposted reviewers onto HI-focused guidance produced by the Campbell and Cochrane Equity Group or the PROGRESS framework [12]. All guidance were produced for informing the production of systematic reviews with two guidance documents also applicable to other types of research [12, 26]. Guidance has been produced to inform considerations of HI at different stages of the systematic review process with the most guidance produced to support the conduct of systematic reviews.

Guidance definitions

The documents defined their guidance in different ways, for example, as briefing notes [2225] equity lens [12, 26, 27, 29, 30], recommendations [29, 30, 36], plausibility algorithm [35], tool [28] framework [31, 39], guidance [37, 38, 40] and guidelines [3234].

All but three guidance documents [31, 39, 40] define what is meant by HI, equity or inequity. Where definitions are recorded, they differed across the studies. Whitehead’s [13] definition of health equity and HI were the most commonly reported within the guidance documents, although a number of different authors were cited for HI [8, 13, 41, 42], health inequity [13, 41, 43] and health equity [13, 4448]. Four of the five guidance documents with a sex and gender HI focus all define what it meant by sex and gender in the same way [2225]. Neither of the two guidance documents with a socio-economic focus define socio-economic status [26, 35].

Development of guidance

Table 3 outlines the guidance development process of the included studies. The majority of the guidance documents were transparent in outlining how the guidance had been produced. Of those reporting methods of guidance development, all were informed by a literature review.

Table 3 Development of guidance

The guidance development process for most involved seeking feedback and revision from people with a wide range of expertise (including researchers, HI experts, review methodologists, decision-makers, clinical epidemiologists, practitioners and journal editors) and systematic review experience, the majority of whom were either members of the Cochrane Collaboration or were attending Cochrane Workshops. Burford et al. [34] and Doull et al. [22] specifically report involving novice reviewers in the development of their guidance. The PRISMA-E 2012 reporting guidelines [32, 33] were also informed by consensus methods. Whilst all of the guidance documents were produced from an HI perspective, only five had theoretical underpinnings or followed established methods in developing their guidance [22, 27, 32, 33, 35].

Operationalisation of guidance

As a means of demonstrating what reviewers should consider in the application of the items, most of the guidance provide examples from published systematic reviews. For example, when asking reviewers to consider whether there are known or possible differences by sex/gender, Welch et al. [23] use the following example, “In a systematic review on quality of life after total hip and total knee arthroplasty, men appeared to benefit more from the intervention in the few studies that addressed this issue.”

Despite the different purposes and audiences, there is a considerable overlap in what users of the guidance are asked to consider at different stages of the review process. Table 4 highlights this using the example of whether to expect differential effects across SES population characteristics.

Table 4 Overlap of guidance items on anticipating differential effects across SES in relation to population characteristics

The rationale for why HI should be considered was also provided in some guidance. For example, Oxman et al. ([28], “Questions to consider”, no.3) ask “are there likely to be different baseline conditions across groups or settings such that the absolute effectiveness of the option would be different, and the problem more or less important, for disadvantaged groups or settings?” They then outline the rationale, “Typically, baseline risks are larger in disadvantaged populations and a larger absolute effect could therefore be expected.” ([28], “Questions to consider”, no.3).

When addressing how reviewers can operationalise the items, the comprehensiveness and application with which this is detailed differs across the guidance. For example, the number of items reviewers are asked to consider in the guidance documents ranges from 3 up to 26. Many guidance documents recommend using a theory-based approach, using programme theory or logic models to understand the assumptions behind how and why the intervention may work differently across disadvantaged populations and the influence of context on the outcome [2226, 29, 30, 32, 33, 35, 36]. However, there is a lack of detail on how this could be implemented in practice. Tugwell et al. ([29], 2.“Defining disadvantage”, para. 2), for example, state that “implications on inequities are dependent on context, so authors of HI orientated reviews must strive to understand and explore the mediating effect of context”, yet they do not define what is meant by context and what data could be collected to explore this. Welch et al. [32] suggest that one limitation of the guidance is the use of terminology such as logic model, analytic framework, context and process evaluation, terms that are not widely accepted. A lack of consistency was noted in applying such terms across the guidance with some referring instead to causal pathway analysis, program theory and mechanisms of action.

Few guidance documents discuss who should be involved in making decisions on if and how HIs matter in systematic reviews. Tugwell et al. [29] suggest that relevant stakeholders should be involved in defining the review question, whilst HIAT [26] recommend involving members of the public (e.g. service users, carers, people living in disadvantaged neighbourhoods) in the planning stages. Only Welch et al. [35] started to explore how people came to their decisions of whether to expect differential effects across sex/SES and found that these decisions were made based on theory, personal experience, empirical data and guesses, but called for further research to investigate how these are used to make judgements.

Evaluation of guidance

Formal methods of evaluation were reported in three studies [22, 34, 35] for six of the guidance documents [2325, 32, 33, 35]. Burford et al. [34] surveyed 151 systematic review authors on the perceived utility of the PRISMA-E 2012 [32, 33]. Reported advantages of using PRISMA-E 2012 include improved awareness of HI considerations in systematic reviews and improved consistency of reporting of HI in systematic reviews. Barriers reported include time required to apply guidance, length of guidance, increase length and complexity of reviews and lack of data in primary studies to operationalise some of the guidance.

Doull et al. [22] undertook a workshop evaluation involving 19 participants including potential users (researchers, practitioners and policy-makers) with little or some knowledge of the concepts of sex/gender to evaluate the content, readability and comprehensiveness of their briefing notes [2325]. Although respondents reported that the briefing notes “provided clear methodological guidance to address sex/gender in reviews” ([22], Results, para. 2) and rated all aspects highly, responses were mixed on the level of complexity within the methods section.

Finally, in Welch et al. [35] four clinical methodologists evaluated the face and construct validity of a plausibility algorithm in predicting the likelihood of differential effects across sex and SES. Thirty-five review users, methodologists and clinicians also assessed the inter-rater reliability of the algorithm against ten pre-selected systematic reviews. The results found a low to no agreement beyond chance between raters for each of the three questions across sex and socioeconomic considerations. The authors suggest several reasons for the poor agreement relating to the design of the algorithm (use of multi-component questions covering several factors, omission of a don’t know category for responses), individual characteristics of respondents and poor choice of proxy or gold standard set of reviews to test the algorithm against. Whether any of the guidance has resulted in an uptake of considerations of HI in systematic reviews is still to be determined [22, 33].


This review identified 20 guidance documents for incorporating considerations of equity in systematic reviews spanning the whole spectrum of the review process, from planning, conduct and reporting through to considerations of applicability to disadvantaged populations and knowledge translation when using reviews to inform decision-making and policy. Many of the documents have been published in the last 3 years, highlighting the fact that methodological research in this field is an important, emerging and evolving area of interest. The majority of the guidance is published in open access format in the journal literature. Whilst this increases the accessibility of the guidance, omission from standard review guidance handbooks places a greater emphasis on guidance authors to raise awareness of the existence of such tools to encourage greater uptake. For example, Welch et al. [32, 33] recognise the importance of widespread dissemination amongst journal editors, funding bodies and ethics committees in order to encourage the adoption of the PRISMA-Equity 2012 Extension reporting guideline.

Increasing awareness of such guidance is even more important when reviewers are faced with the different terminology used by authors to describe it. Use of multiple terms such as algorithm, equity lens, tool, may make them harder to locate within the journal literature. The Campbell and Cochrane Equity Methods Group have started to collate guidance on HI considerations for review authors [49], yet the list does not cover all guidance identified in this review and focuses to a greater extent on the guidance developed to support Cochrane Review authors.

Whitehead’s [13] definition of HI as “Inequality in health is a term commonly used in some countries to indicate systematic, avoidable and important differences” and health inequity as “refers to differences in health which are not only unnecessary and avoidable but, in addition, are considered unfair and unjust” were most commonly applied in the guidance. However, rather than demonstrate an acceptance of a common definition for HI or health equity, most of the guidance citing Whitehead [13] were produced by the same group (The Campbell and Cochrane Equity Methods Group). The different definitions of HI and health equity used by the guidance documents supports the view by Tugwell et al. [29] that these terms are used in different ways by different authors, and that there is no agreed definition of health equity or HI. This stresses the importance therefore for HI guidance to define health equity and/or HI to help reviewers to operationalise the guidance.

This is particularly true for guidance incorporating considerations of SES in systematic reviews. Neither of the two guidance documents with a focus on SES [26, 35] defined the term. Several of the guidance documents refer reviewers to the PROGRESS framework [12, 50] when asking reviewers to consider disadvantage. PROGRESS although does not explicitly define SES, relates it to income, while considering education and occupation separately. Yet SES has been defined more broadly as “a composite measure that typically incorporates economic status, measured by income, social status, measured by education; and work status measured by occupation” ([51], pp. 30), [52, 53]. The classification of individuals by SES has implications for reviewers in relation to types of SES indicators to collect and therefore, the definition and classification of HI terms such as SES needs to be operationalised within the guidance. This is further supported by the findings of Runnels et al. [54] who in a survey examining the challenges of including sex/gender analysis in systematic reviews found that one of the significant challenges was clarifying the concepts of sex and gender.

Much of the guidance is written from the perspective of HI having already being identified as the focus of the review and from a whole HI perspective rather than focusing on a specific HI dimension. More recently, however, guidance is being tailored more towards specific HI domains, in particular, sex/gender [2325, 35] and SES [26, 35]. This may reflect the interests of the groups involved in producing the guidance. For example, HIAT [26] was developed by the National Institute of Health Research Collaboration for Leadership in Applied Health Research and Care North West Coast (NIHR CLAHRC NWC) whose remit is to ensure that all of the research it produces has socio-economic relation to HI as its core focus (

Guidance on incorporating sex/gender analysis in systematic reviews was produced by the Cochrane Sex/Gender Methods Group (a subgroup of the Campbell and Cochrane Equity Methods Group). However, the development of more HI-specific guidance may also suggest that once reviewers have identified which HI domain(s) to consider, they may have difficulty operationalising the more generic HI-focused tools and require more tailored guidance. Or it may be that given that it would be impossible for reviewers to incorporate considerations across all HI dimensions, that there is a debate developing that certain HI domains may be considered more important than others to incorporate in systematic reviews. Indeed, in the application of the PROGRESS framework to systematic reviews, O’Neill et al. [12, p. 62] state that the framework is “not intended to encourage data dredging but to identify the most important factors that drive inequities in health”.

The Campbell and Cochrane Equity Group have been instrumental in driving forward methodological advances in evidence synthesis for the incorporation of HI in systematic reviews, and although not specifically stated, most of the guidance produced appears most relevant to reviewers undertaking systematic reviews measuring the effectiveness of interventions or at users assessing the applicability of reviews of effectiveness. This is no doubt partly due to the focus of the Cochrane Collaboration on reviews of effectiveness, yet how well this guidance translates to other types of HI systematic reviews (e.g. qualitative) therefore is unclear, and further research investigating the usefulness of the guidance for other types of reviews is required.

One potential challenge facing reviewers in operationalising the guidance is that methodological approaches to incorporating considerations of HI are still in development, and even the guidance authors themselves recognise that one of the limitations of the guidance may relate to the terminology used, such as logic model, process evaluation, mechanisms of action, terms which may not be accepted [32] or understood [55]. Encouraging reviewers to consider what works for disadvantaged populations, why, how and under what circumstances not only requires as Tugwell et al. ([29], Conclusions, para. 2) suggest “a paradigm shift in the generation and synthesis of evidence”, but also an acceptance of the terminology along with an understanding of how these methods can be applied in practice. Part of the problem is that current review methods cannot necessarily be transferred across to consider more complex issues and HI [3, 11] and methods to incorporate such complexity in systematic reviews are only just emerging [9, 11, 5661]. Given the potential complexity of the process therefore, further research examining the challenges and barriers in incorporating considerations of HI in systematic reviews is required to better support reviewers in undertaking HI-focused reviews.

There is also perhaps, a greater need to understand how guidance items can be operationalised. Most of the guidance is operationalised by means of descriptive examples from published systematic reviews, and there is evidence to indicate that reviewers find them useful. For example, Doull et al., [22] in testing their guidance on incorporating issues of sex/gender analysis in systematic reviews found that reviewers wanted even more examples.

Much of the guidance is written with an underlying assumption that reviewers will recognise if and how equity matters. Empirical evidence however suggests otherwise [35, 54]. A recent survey by Runnels et al. [54] conducted on the challenges of including sex/gender analysis in systematic reviews seems to support this view. They found that concerns were raised over the construction of knowledge and cites one respondent saying “the biggest challenges are much more fundamental and have to do with the way that we arrive at decisions as to what is important for us to study, why it is important for us to study, and how we determine the way to study” ([53], Conceptual challenges, para. 3).

Welch et al., [35] in developing their equity plausibility algorithm, start to explore the rationale behind reviewers’ decisions and found that empirical data, theory or personal experience were often used to explain their reasons but call for further research to enhance understanding of how this was used and the contribution of individual characteristics to the process. In order to help reviewers to operationalise the guidance therefore, it may be useful to explore the rationale behind how reviewers are making decisions when applying the guidance and the contribution made to those decisions by different individuals (e.g. stakeholders).

Furthermore, if, as Welch et al. ([35], Discussion, para. 1) suggest, reviewers “need to have a deep understanding of the content area” to make judgements about likely differential effects, then single examples drawn from topic-specific reviews may not be the best way to demonstrate guidance application. Without a comprehensive understanding of the different ways in which HI issues may contribute towards differential effects in health outcomes, it may be difficult for some reviewers, particularly those new to HI, to recognise the need to incorporate or operationalise such issues in systematic reviews. Building on this, research is currently underway by the author (MM) to identify a comprehensive set of considerations to help reviewers operationalise the influence of socio-economic contextual factors on how and why an intervention may work differently across disadvantaged populations.

Strengths of the guidance evaluation methods include the involvement of a wide range of expertise (e.g. reviewers, methodologists, decision-makers, HI experts), consensus methods and piloting of the guidance. However, the use of self-selecting samples may not necessarily be representative of the wider population expected to utilise the guidance. Assessment of face validity alone, i.e. a subjective assessment of the relevance of the questions [62] rather than evaluating how well the guidance works when applied prospectively may not identify problems operationalising items. Burford et al. [34] used a prospective design to assess the utility of the reporting guidelines but, largely due to the type of guidance, did not ask reviewers to discuss how they had operationalised the checklist items. In addition, evaluation of the guidance appears to be undertaken by individuals rather than reflecting the collaborative approach that a systematic review encourages. This may be a further reason why the Welch et al. [35] plausibility algorithm had poor inter-rater reliability. Evaluating guidance using a case study approach, may better capture how well the guidance is interpreted or operationalised by those it is designed to assist. Involving novice reviewers in the guidance evaluation may also identify challenges in interpreting and operationalising the guidance that may not necessarily be considered when piloting is undertaken by the guidance developers or expert reviewers alone.

Strengths and limitations of the review

This study is the first to summarise the range of guidance available on the incorporation of HI in systematic reviews. One of the strengths of this study is detailing what guidance is available for considering HI at various stages of the systematic review process. There is no validated search filter for HI, however, terms were based on those used in a Cochrane methodological review exploring how effects on HI are assessed in systematic reviews [21]. The review did not seek to critique the individual items/questions in the guidance or recommend one guidance over another, but rather offer an overview of guidance available to reviewers when incorporating considerations of HI at different stages of the review process.

A potential limitation of this review is that as this study is part of a PhD study, only one person was involved in the selection of studies, data extraction and synthesis. One limitation of this review is the focus on English language literature when it is acknowledged that other languages, such as Spanish, may offer extensive coverage of literature regarding inequalities. As this is not a systematic review, the search was restricted to a small number of key databases in health, further databases outside of health could have been searched. Instead, a targeted approach to the search was adopted using a number of different search approaches, including scanning of relevant systematic review organisational websites, reference checking and contacting known experts in the field. Given the diverse nature of the guidance documents included in this review, no formal quality appraisal was undertaken instead each guidance document was assessed as to whether or not it formally evaluated.


Given the recent growing interest in the incorporation of HI in systematic reviews, it is not surprising that methodological guidance exploring how considerations of HI can be incorporated into evidence synthesis is a relatively new and emerging area of research [7, 36]. Above all, the strength of all the guidance documents reviewed in this study is in highlighting the importance of incorporating considerations of HI in systematic reviews, yet due to the fairly recent introduction of the guidance there is little evidence on the guidance uptake or whether it has led to an improvement in considerations of HI in systematic reviews. It is clear, however, that operationalising the guidance will require more work for the reviewer but aside from the Runnels et al. [54] survey, there is limited evidence on the challenges facing reviewers when incorporating considerations of HI. Furthermore, understanding how reviewers can operationalise the guidance and the challenges in doing so have implications not only for understanding the usefulness and burden of the guidance [34], but also has implications for the uptake of guidance and its ultimate goal of improving HI considerations in systematic reviews. There is currently a gap in the evidence examining how reviewers can operationalise the guidance and the barriers and facilitators involved. The results of this review will be used to inform the development of a framework to help reviewers rationalise whether or not to incorporate considerations of HI in systematic reviews.


  1. 1.

    Whitehead M. A typology of actions to tackle social inequalities in health. J Epidemiol Community Health. 2007;61(6):473–8.

  2. 2.

    Arblaster L, Lambert M, Entwistle V, Forster M, Fullerton D, Sheldon T, Watt I. A systematic review of the effectiveness of health service interventions aimed at reducing health inequalities in health. J Health Serv Res Policy. 1996;1(2):93–103.

  3. 3.

    Blaxter M. Evidence for the effect on inequalities in health of interventions designed to change behaviour. Bristol: Department of Social Medicine, University of Bristol. NICE BC 6-5; 2007.

  4. 4.

    Lorenc T, Petticrew M, Welch V, Tugwell P. What types of interventions generate inequalities? Evidence from systematic reviews. J Epidemiol Community Health. Online First: 8 August 2012.

  5. 5.

    Plachta-Danielzik S, Landsberg B, Lange D, Seiberl J, Muller MJ. Eight-year follow-up of school-based intervention on childhood overweight - the Kiel obesity prevention study. Obes Facts. 2011;4(1):43.

  6. 6.

    Tugwell P, Petticrew M, Robinson V, Kristjansson E, Maxwell L for the Cochrane Equity Field Editorial Team. Cochrane and Campbell Collaborations and health equity. The Lancet. 2006;367:1128–30.

  7. 7.

    Petticrew M, Welch V, Tugwell P. ‘It is surely a great criticism of our profession…’ The next 20 years of equity-focused systematic reviews. J Epidemiol Community Health. 2014;68(4):291–2. doi:10.1136/jech-2013-203400.

  8. 8.

    Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public policy on inequalitites: 1: the reality according to policymakers. J Epidemiol Community Health. 2004;58:811–6.

  9. 9.

    Burford B, Lewin S, Welch V, Rehfuess E, Waters E. Assessing applicability in systematic reviews of complex interventions. J Clin Epidemiol. 2013;66:1251–126.

  10. 10.

    Lavis JN, Davies HTO, Gruen RL. Working within and beyond the Cochrane Collaboration to make systematic reviews more useful to healthcare managers and policy makers. Healthc Policy. 2006;1:21–33.

  11. 11.

    Squires JE, Valentine JC, Grimshaw JM. Systematic reviews of complex interventions: framing the review question. J Clin Epidemiol. 2013;66:1215–22.

  12. 12.

    O'Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M, Evans T, Pardo Pardo J, Waters E, White H, Tugwell P. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67:56–64. doi:10.1016/j.jclinepi.2013.08.005.

  13. 13.

    Whitehead M. The concepts and principles of equity and health. Int J Health Serv. 1992;22(3):429–45.

  14. 14.

    Welch V. What is the role of systematic reviews in tackling health inequity? University of Ottawa; 2010. PhD Thesis.

  15. 15.

    Bambra C, Gibson M, Sowden A, Wright K, Whitehead M, Petticrew M. Tackling the wider social determinants of health and health inequalities: evidence from systematic reviews. J Epidemiol Community Health. 2010;64:284–91.

  16. 16.

    Main C, Thomas S, Ogilvie D, Stirk L, Petticrew M, Whitehead M, Sowden A. Population tobacco control interventions and their effects on social inequalities in smoking: placing an equity lens on existing systematic reviews. BMC Public Health. 2008;8:178.

  17. 17.

    Welch V, Petticrew M, Ueffing E, Benkhalti Jandu M, Brand K, Dhaliwal B, Kristjansson E, Smylie J, Wells GA, Tugwell P. Does consideration and assessment of effects on health equity affect the conclusions of systematic reviews? A methodology study. PLoS ONE. 2012;7(3), e31360.

  18. 18.

    Paul-Ebhohimhen V, Avenell A. Systematic review of the use of financial incentives in treatments for obesity and overweight. Obes Rev. 2008;9:355–67.

  19. 19.

    Gibson M, Petticrew M, Bambra C, Sowden AJ, Wright KE, Whitehead M. Housing and health inequalities: a synthesis of systematic reviews of interventions aimed at different pathways linking housing and health. Health Place. 2011;17(1):175–84.

  20. 20.

    Booth A, Sutton A, Papaioannou D. Systematic approaches to a successful literature review. 2nd ed. London: SAGE Publications Ltd; 2016.

  21. 21.

    Welch V, Tugwell P, Petticrew M, de Morntigny J, Ueffing E, Kristjansson E, McGowan J, Benkhalti J,M, Wells G,A, Brand K, Smylie J. How effects of health equity are assessed in systematic reviews of interventions. Cochrane Database of Systematic Reviews. 2010, Issue 12. Art No: MR000028.

  22. 22.

    Doull M, Welch V, Puil L, Runnels V, Coen SE, Shea B, O'Neill J, Borkhoff C, Tudiver S, Boscoe M. Development and evaluation of 'briefing notes' as a novel knowledge translation tool to aid the implementation of sex/gender analysis in systematic reviews: a pilot study. PLOS One. 2014. doi:10.1371/journal.pone.0110786.

  23. 23.

    Welch V, Puil L, Shea B, Runnels V, Doull M, Tudiver S, Boscoe M, for the Sex/Gender Methods Group. Addressing sex/gender in systematic reviews: Cochrane Musculoskeletal Group briefing note. Version 2014-01. Accessed at:

  24. 24.

    Puil L, Doull M, Runnels V, Welch V, Shea B, Tudiver S, Boscoe M, for the Sex/Gender Methods Group. Addressing sex/gender in systematic reviews: Cochrane Hypertension Group briefing note. Version 2014-01. Accessed at:

  25. 25.

    Doull M, Shea B, Puil L, Runnels V, Welch V, Tudiver S, Boscoe M, for the Sex/Gender Methods Group. Addressing sex/gender in systematic reviews: Cochrane HIV/AIDS Group briefing note. Version 2014-01. Accessed at:

  26. 26.

    NIHR CLAHRC North West Coast. Health inequalities assessment toolkit (HIAT); 2015. Accessed 1 Feb 2016.

  27. 27.

    Nasser M, Ueffing E, Welch V, Tugwell P. An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. J Clin Epidemiol. 2013;66:511–21.

  28. 28.

    Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP). 10. Taking equity into consideration when assessing the findings of a systematic review. Health Res Policy Syst. 2009;7 Suppl 1:S10.

  29. 29.

    Tugwell P, Petticrew M, Kristjansson E, Welch V, Ueffing E, Waters E, Bonnefoy J, Morgan A, Doohan E, Kelly M. Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ. 2010;341:c4739.

  30. 30.

    Ueffing E, Tugwell P, Welch V, Petticrew M, Kristjansson E for the Campbell and Cochrane Equity Methods Group. Equity checklist for systematic review authors. Version 2012-10-02. Accessed at on 30 Jan 2016.

  31. 31.

    Tugwell P, Robinson V, Grimshaw J, Santesso N. Systematic reviews and knowledge translation. Bull World Health Organ. 2006;84(8):643–51.

  32. 32.

    Welch V, Petticrew M, Tugwell P, Moher D, O’Neill J, Waters E, White H, PRISMA-Equity Bellagio group. PRISMA-Equity 2012 Extension: Reporting Guidelines for Systematic Reviews with a Focus on Health Equity. PLoS Med. 2012;9(10):e1001333. doi:10.1371/journal.pmed.1001333.

  33. 33.

    Welch V, Petticrew M, Petkovic J, Moher D, Waters E, White H, Tugwell P, PRISMA-Equity Bellagio group. Extending the PRISMA statement to equity-focused systematic reviews (PRISMA-E 2012): explanation and elaboration. J Clin Epidemiol. 2016;70:68–89.

  34. 34.

    Burford BJ, Welch V, Waters E, et al. Testing the PRISMA-equity 2012 reporting guideline: the perspectives of systematic review authors. PLoS ONE. 2013;8, e75122.

  35. 35.

    Welch V, Brand K, Kristjansson E, Smylie J, Wells G, Tugwell P. Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm. BMC Med Res Methodol. 2012;12:187.

  36. 36.

    Welch VA, Petticrew M, O'Neill J, Waters E, Armstrong R, Bhutta ZA, Francis D, Perez Koehlmoos T, Kristjansson E, Pantoja T, Tugwell P. Health equity: evidence synthesis and knowledge translation methods. Syst Rev. 2013;2:43. doi:10.1186/2046-4053-2-43.

  37. 37.

    Armstrong R, Waters E, Doyle J (editors). Chapter 21: Reviews in health promotion and public health. In Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from

  38. 38.

    Armstrong R, Waters E, on behalf of the Guidelines for Systematic Reviews in Health Promotion and Public Health Taskforce. Systematic reviews of health promotion and public health Interventions. July, 2007.

  39. 39.

    Chambers D, Wilson P. A framework for production of systematic review based briefings to support evidence-informed decision-making. Syst Rev. 2012;9(1):32.

  40. 40.

    Centre for Reviews and Dissemination (CRD). Systematic reviews. CRD’s guidance for undertaking reviews in health care. University of York: Centre for Reviews and Dissemination; 2009.

  41. 41.

    Kawachi I, Subramanian SV, Almeida-Filho N. A glossary for health inequalities. J Epidemiology Community Health. 2002;56:647–52.

  42. 42.

    WHO. Health impact assessment, glossary of terms. World Health Organization.

  43. 43.

    Whitehead M. The concepts and principles of equity and health. Health Promot Int. 1991;6(3):217–28.

  44. 44.

    Braveman P, Gruskin S. Defining equity in health. J Epidemiol Community Health. 2003;57:254–8.

  45. 45.

    Commission on Social Determinants of Health. Closing the gap in a generation: health equity through action on the social determinants of health. Final Report of the Commission on Social Determinants of Health. Geneva: World Health Organization; 2008.

  46. 46.

    Institute of Medicine. Unequal treatment: confronting racial and ethnic disparities in health care. Washington, DC: National Academy Press; 2002.

  47. 47.

    Macinko JA, Starfield B. Annotated bibliography on equity in health, 1980-2001. Int J Equity Health. 2002;1:1.

  48. 48.

    Solar O, Irwin A. Towards a conceptual framework for analysis and action on the social determinants of health. WHO, Commission on Social Determinants of Health, 2007

  49. 49.

    The Campbell and Cochrane Equity Methods Group. Resources for review authors.

  50. 50.

    Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector reform. Inj Control Saf Promot. 2003;10(1-2):11–2.

  51. 51.

    Dutton DB, Levine S. Overview, methodological critique, and reformulation. In: Bunker JP, Gomby DS, Kehrer BH, editors. Pathways to health (pp 26-69). Menlo Park: The Henry J Kaiser Family Foundation; 1989.

  52. 52.

    Adler NE, Boyce T, Chesney MA, Cohen S, Folkman S, Kahn RL, Syme SL. Socioeconomic status and health. The challenge of the gradient. Am Psychol. 1994;49(1):15–24.

  53. 53.

    Graham H, editor. Understanding health inequalities. 2nd ed. Maidenhead: Open University Press; 2009.

  54. 54.

    Runnels V, Tudiver S, Doull M, Boscoe M. The challenges of including sex/gender analysis in systematic reviews: a qualitative survey. Syst Rev. 2014;3:33. doi:10.1186/2046-4053-3-33.

  55. 55.

    Guise JM, Chang C, Viswanathan M, Glick S, Treadwell J, Umscheid C, Whitlock E, Fu R, Berliner E, Paynter R, Anderson J, Motu’apuaka M, Trikalinos T. Systematic reviews of complex multicomponent health care interventions. Research white paper. AHRQ Publication No. 14-EHC003-EF. Rockville: Agency for Healthcare Research and Quality; 2014.

  56. 56.

    Petticrew M, Anderson L, Elder R, Grimshaw J, Hopkins D, Hahn R, et al. Complex interventions and their implications for systematic reviews: a pragmatic approach. J Clin Epidemiol. 2013;66:1209–14.

  57. 57.

    Anderson L, Oliver S, Michie S, Rehfuess E, Noyes J, Shemilt I. Investigating complexity in systematic reviews of interventions using a spectrum of methods. J Clin Epidemiol. 2013;66:1223–9.

  58. 58.

    Petticrew M, Rehfuess E, Noyes J, Higgins JPT, Mayhew A, Pantoja T, et al. Synthesising evidence on complex interventions: the contribution of meta-analytic, qualitative and mixed-method approaches. J Clin Epidemiol. 2013;66:1230–43.

  59. 59.

    Pigott T, Sheppard S. Identifying, documenting and examining heterogeneity in systematic reviews of complex interventions. J Clin Epidemiol. 2013;66:1244–50.

  60. 60.

    Noyes J, Gough D, Lewin S, Mayhew A, Michie S, Pantoja T, et al. Systematic reviews that ask complex questions about complex interventions: a research and development agenda. J Clin Epidemiol. 2013;66:1262–70.

  61. 61.

    Snilstveit B. Systematic reviews: from ‘bare bones’ reviews to policy relevance. J Dev Effect. 2012;4(3):388–408.

  62. 62.

    Bruce N, Pope D, Stanistreet D. Quantitative methods for health research: a practical guide to epidemiology and statistics. 1st ed. Chichester: Wiley-Interscience; 2008.

Download references


The author would like to thank her supervisors, Rumona Dickson (University of Liverpool), Suzy Paisley (University of Sheffield) and Mark Gabbay (University of Liverpool), for their comments on earlier drafts of this paper.


This review was undertaken as part of a wider PhD examining the incorporation of HI in evidence synthesis funded by The National Institute of Health Research (CLAHRC NWC).

Availability of data and materials

Not applicable.

Authors’ contributions

MM contributed to this study in its entirety.

Competing interests

Not applicable.

Consent for publication

The views expressed are those of the author and not necessarily those of the NHS, the NIHR or the Department of Health.

Ethics approval and consent to participate

Not applicable.

Author information

Correspondence to Michelle Maden.

Appendix 1: Medline search strategy

OVID Medline

1 exp Meta-Analysis as Topic/

2 systematic

3 meta-analys$.tw.

4 meta-epidemiolog$.tw.

5 exp "Review Literature as Topic"/

6 (Cochrane adj2 review).tw.

7 ((qualitative or realist or evidence or narrative or knowledge) adj synthesis).tw

8 integrative



11 or/1-10

12 (gender-based OR gender-related OR gender differences OR gender factors).tw.

13 ((sex OR gender) adj2 (analysis OR specific OR difference? OR factor? OR inequit$ OR disparit$ OR inequalit$)).tw.

14 exp sex factors/

15 exp geriatrics/

16 ((ethnic$ OR race OR racial OR religio$ OR cultur$ OR minorit$ OR refugee OR indigenous OR aboriginal) adj3 (analysis OR difference$ OR specific OR disparit$ OR inequalit$ OR inequit$)).tw.

17 exp homosexuality/

18 exp disabled persons/

19 ((poverty OR low-income OR socioeconomic$ OR social) adj2 (analysis OR disadvantage$ OR specific OR difference? OR factor? OR inequalit$ OR depriv$ OR inequit$ OR disparit$)).tw.

20 exp Educational Status/

21 exp Socioeconomic Factors/

22 ((discriminat$ OR social exclu$ OR social inclu$) adj3 (religion OR culture OR race OR racial OR aboriginal OR indigenous OR ethnic$)).tw.

23 ((urban OR rural OR inner-city OR slum) adj2 (difference$ OR specific OR analysis OR inequit$ OR disparit$ OR inequalit$)).tw.

24 ((resource-poor OR (low-income adj countr$) OR (middle income adj countr$) OR africa OR developing countr$ OR south america OR china OR asia OR latin america) adj2 (relevance OR analysis OR specific OR difference OR applicab$ OR inequit$ OR disparit$ OR inequalit$)).tw.

25 health adj2 inequalit*.tw

26 health adj2 equit*.tw

27 health adj2 inequit*.tw

28 ((social gradient* or gap) adj3 (reduc* or difference* or disparit* or increase* or inequit* or inequalit* or equit* or disadvantage*)).tw

29 exp Health Status Disparities/

30 or/12-29

31 (guidance or guideline* or tool* or method* or framework* or model*).ti.

32 11 AND 30 AND 31

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark


  • Health inequalities
  • Health equity
  • Systematic reviews
  • Guidance
  • Mapping review


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.