Searching for health equity: validation of a search filter for ethnic and socioeconomic inequalities in transport
Systematic Reviews volume 8, Article number: 94 (2019)
Efforts to improve health equity should be informed by the best available evidence. However, equity-related research is inconsistently indexed, and uses a variety of terms to describe key concepts, making it difficult to reliably identify all relevant studies. We report the development and validation of a search strategy for studies investigating whether the effects of interventions differ by ethnicity or socio-economic status, using the field of transport and health as an example.
Adapting previously described methods, we followed four steps: generation of a test set of eligible studies, search strategy development, search strategy validation, and documentation.
Drawing from 12 systematic reviews, supplemented by additional studies identified by experts and colleagues, we identified a test set of 11 studies that met our eligibility criteria. We assigned five studies to a development set, which we used to develop and refine our search strategy. We assigned the remaining six studies to a validation set, against which we tested our final search strategy. The final search strategy identified all studies in both validation and development sets.
The validated search strategy derived in this study facilitates the conduct of systematic reviews and other literature searches investigating whether the effects of interventions differ by ethnicity or socio-economic status and may be further developed in future for other equity-focused searches and reviews.
It is generally accepted that efforts to improve health should be informed by the best available evidence, and that systematic reviews are an important tool for forming a reliable representation of all available evidence. Evidence-informed practice is just as important for health equity as for other fields, yet there are still relatively few equity-focused systematic reviews . A prerequisite for reviews of this kind, as well as for other research-related activities such as guideline development, is effective literature searching.
Searching for health equity literature can be challenging, and this may explain why systematic reviews with this focus are scant. In particular, search strategies using equity-related text filters to screen out unrelated studies run the risk of missing relevant literature. This is because studies in this field use a variety of terms to describe key concepts (e.g. inequities, inequalities, disparities) and equity terms are not indexed consistently . This issue is particularly salient for systematic reviews, which must use sensitive search strategies to ensure findings are not biased by missing studies.
One approach to reduce the risk of missing studies is to use an unrestricted search strategy that does not include equity filters. However, such a search strategy may have very low specificity, and return simply too many citations to review, making a systematic review not feasible. Another approach is to limit the scope of the search to a narrow subtopic. However, this could lead to a review that identified few or no eligible studies  or may exclude other subtopics that are of interest. A third strategy is to carry out a review or overview of systematic reviews (also called umbrella reviews), drawing on equity-relevant papers that were included within previous non-equity specific reviews . However, these overviews may struggle to provide an effective synthesis if different systematic reviews have focused on different populations, outcomes, or quality appraisal methods . They may also fail to include papers published since the relevant systematic reviews were completed. Therefore, these strategies, while useful, may lead to problems with feasibility, scope, or timeliness.
An alternative approach is to use a validated equity-focused search filter that has a low risk of missing relevant studies. Such filters have been developed and validated for some population subgroups , but do not cover all important dimensions of health equity. In particular, we are aware of no validated filters for studies addressing socio-economic status or for ethnicity (with the exception of specific indigenous populations in Canada and Australia [6, 7]).
Therefore, in this study, we developed and validated a search filter for studies investigating whether the effects of interventions differ by ethnicity or socio-economic status. We developed and tested this filter in the field of transport, a social determinant of health with important effects on both health and equity, with the goal of producing a validated strategy that could subsequently be applied or adapted in other areas of health. Transport policies and interventions can affect physical activity and air pollution levels, as well as road traffic injury risk, and lead to substantial health impacts . Health effects arising from transport are seldom distributed equally between different social groups, and there is evidence that ethnicity and social deprivation can be associated with a higher burden of transport-related disease [9, 10].
We adapted the general approach described by Hausner et al. , first generating a test set of eligible studies, followed by development, validation, and documentation of the search strategy.
Generation of test set of eligible studies
We generated a test set of eligible studies by reviewing the reference lists of systematic reviews of transport interventions. We also contacted experts and colleagues working in this field. Given the absence of comprehensive systematic reviews of transport and equity, we included systematic reviews that were not specific to health equity, and then appraised studies against a set of eligibility criteria (Table 1) in order to identify studies that investigated whether the effects of transport interventions differed by ethnicity or socio-economic status.
Search strategy development
We developed a MEDLINE search strategy in two parts: a ‘transport intervention’ search strategy and an ‘equity-focused’ search for studies addressing ethnicity or socioeconomic status. Our transport intervention search strategy was adapted from a previous Cochrane systematic review of transport and health . Our equity-focused search strategy was adapted from a previous systematic review investigating socio-economic differences in the effectiveness of nutrition interventions . This draft search strategy was further refined using a trial-and-error approach in which search term combinations were tested in MEDLINE to see if they identified the journal articles in the ‘development set’. We aimed to achieve 100% sensitivity while maximising specificity as far as possible. The final search strategy combined the transport intervention and equity-focused search strategies using the ‘AND’ operator.
Search strategy validation
We assessed the validity of the final search strategy by testing it against a validation set, following the approach used in Hausner et al. . We recorded which studies in the validation set were identified by the final search strategy and which were not.
Generation of test set of eligible studies
We identified 10 systematic reviews of transport interventions through database searches and prior knowledge [12, 14,15,16,17,18,19,20,21,22]. Experts and colleagues working in the field identified two further systematic reviews [23, 24]. From the studies included in those systematic reviews, we identified 125 potentially eligible journal articles. Experts and colleagues suggested a further 33 journal articles, giving a total of 158 journal articles. We retrieved the full text for all 158 articles and assessed them against the systematic review eligibility criteria, resulting in 11 eligible studies.
This test set of 11 articles was randomly divided into a development set and a ‘validation set’ using random number generation in Microsoft Excel. Five studies were assigned to the development set [25,26,27,28,29] and six studies were assigned to the validation set [30,31,32,33,34,35].
Search strategy development and validation
Application of the draft search strategy to the development set identified several additional search terms that were added to the equity-focused search strategy to improve sensitivity. Other search terms were removed in order to increase specificity.
The transport intervention search strategy identified 113,133 citations, and the final equity-focused search strategy identified 618,121 citations. Combining the two search strategies using the ‘AND’ operator yielded 8170 citations (Table 2). This final combined search strategy identified all five studies (100%) in the development set. It was then tested against the validation set, in which it identified all six studies (100%).
In this paper, we report a validation exercise for a search strategy for studies investigating whether the effects of interventions differ by ethnicity or socio-economic status, using the field of transport and health as an example. Our search strategy, which included equity-focused text word limits for ethnicity and socio-economic status, successfully identified all studies in our development set (n = 5) and validation set (n = 6) of eligible studies, a sensitivity of 100%. The equity-focused filter also greatly improved specificity, reducing the number of citations returned from > 100,000 (for the unfiltered transport intervention search strategy) to < 10,000.
A limitation of this validation exercise is that there were only 11 eligible studies. It would be desirable to test the search strategy with a larger set of eligible studies and in other fields beyond transport. However, in the absence of other validated search strategy filters for socio-economic status and ethnicity (except for single ethnic groupings [6, 7]), this strategy facilitates the conduct of systematic reviews relating to ethnicity and socio-economic status, particularly in the transport field.
We are aware of no published systematic reviews related to transport interventions and health equity, though one ‘umbrella review’ of other systematic reviews found no primary studies of 20 mph zones and health inequalities . Our validation exercise suggests that there may be relatively few primary studies in this area; using previously published systematic reviews and expert advice, we identified only 11 eligible studies addressing ethnicity or socio-economic status. Our search strategy validation could provide a starting point for systematic reviews that could identify additional studies.
Our search strategy successfully identified all studies in our ‘test set’ of eligible studies, which included studies assessing whether intervention effects differed for different ethnic or socio-economic groups. We did not include studies that only addressed ethnicity/SES as a confounder of intervention effects, so our search strategy should not be used to search for such studies. Studies that adjust for ethnicity/SES as a confounder may be less likely to mention ethnicity or SES in the study abstract, making a sensitive search strategy difficult to design.
We conclude that it is likely to be very difficult to undertake a systematic review in this field without health equity text word limits, due to the very large number of ineligible citations identified (> 100,000 in our transport intervention search strategy). On the other hand, strategies that use unvalidated health equity text word limits run the risk of failing to identify all eligible studies, since equity-related terms are not indexed consistently. We developed and tested a sensitive search strategy that successfully identified all eligible studies in our test set (100% sensitivity), while greatly improving specificity. This validation exercise may provide a useful basis for future systematic reviews relating to ethnicity or socio-economic status, particularly in the transport field, and our search strategy may be further developed in future for other equity-focused searches and reviews.
Controlled before-after study
Interrupted time series
Welch VA, Petticrew M, O’Neill J, Waters E, Armstrong R, Bhutta ZA, et al. Health equity: evidence synthesis and knowledge translation methods. Systematic Reviews. 2013;2(1):1–10.
Hosking JE, Ameratunga SN, Bramley DM, Crengle SM. Reducing ethnic disparities in the quality of trauma care: an important research gap. Ann Surg. 2011;253(2):233–7.
Bosch-Capblanch X, Zuske M-K, Auer C. Research on subgroups is not research on equity attributes: evidence from an overview of systematic reviews on vaccination. Int J Equity Health. 2017;16(1):95.
Hartling L, Vandermeer B, Fernandes RM. Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evidence-Based Child Health: A Cochrane Review Journal. 2014;9(2):486–94.
InterTASC Information Specialists' Sub-Group. The InterTASC Information Specialists’ Sub-Group Search Filter Resource 2016 [Dec 2016]. Available from: https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/other-filters.
Campbell S, Dorgan M, Tjosvold L. Creating provincial and territorial search filters to retrieve studies related to Canadian indigenous peoples from Ovid MEDLINE. J Can Health Libr Assoc. 2014;35(1):5–10.
Tieman JJ, Lawrence MA, Damarell RA, Sladek RM, Nikolof A. LIt.Search: fast tracking access to aboriginal and Torres Strait islander health literature. Aust Health Rev. 2014;38(5):541–5.
Woodcock J, Edwards P, Tonne C, Armstrong BG, Ashiru O, Banister D, et al. Public health benefits of strategies to reduce greenhouse-gas emissions: urban land transport. Lancet. 2009;374(9705):1930–43.
Hosking J, Ameratunga S, Exeter D, Stewart J, Bell A. Ethnic, socio-economic and geographical inequalities in road traffic injury rates in the Auckland region. Aust N Z J Public Health. 2013;37:162–7.
Fecht D, Fischer P, Fortunato L, Hoek G, de Hoogh K, Marra M, et al. Associations between air pollution and socioeconomic characteristics, ethnicity and age profile of neighbourhoods in England and the Netherlands. Environ Pollut. 2015;198:201–10.
Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):1–10.
Hosking J, Macmillan A, Connor J, Bullen C, Ameratunga S. Organisational travel plans for improving health. Cochrane Database Syst Rev. (2010, 3):CD005575.
McGill R, Anwar E, Orton L, Bromley H, Lloyd-Williams F, O'Flaherty M, et al. Are interventions to promote healthy eating equally effective for all? Systematic review of socioeconomic inequalities in impact. BMC Public Health. 2015;15:457.
Arnott B, Rehackova L, Errington L, Sniehotta FF, Roberts J, Araujo-Soares V. Efficacy of behavioural interventions for transport behaviour change: systematic review, meta-analysis and intervention coding. Int J Behav Nutr Phys Act. 2014;11(1):133.
Cavill N, Kahlmeier S, Rutter H, Racioppi F, Oja P. Economic analyses of transport infrastructure and policies including health effects related to cycling and walking: a systematic review. Transp Policy. 2008;15(5):291–304.
Chillón P, Evenson KR, Vaughn A, Ward DS. A systematic review of interventions for promoting active transportation to school. Int J Behav Nutr Phys Act. 2011;8(1):10.
Fraser SD, Lock K. Cycling for transport and public health: a systematic review of the effect of the environment on cycling. Eur J Pub Health. 2011;21(6):738–43.
Ogilvie D, Foster CE, Rothnie H, Cavill N, Hamilton V, Fitzsimons CF, et al. Interventions to promote walking: systematic review. BMJ. 2007.
Pucher J, Dill J, Handy S. Infrastructure, programs, and policies to increase bicycling: an international review. Prev Med. 2010;50(Supplement):S106–S25.
Scheepers CE, Wendel-Vos GCW, den Broeder JM, van Kempen EEMM, van Wesemael PJV, Schuit AJ. Shifting from car to active transport: a systematic review of the effectiveness of interventions. Transp Res A Policy Pract. 2014;70:264–80.
Shaw C, Hales S, Howden-Chapman P, Edwards R. Health co-benefits of climate change mitigation policies in the transport sector. Nature Clim Change. 2014;4(6):427–33.
Yang L, Sahlqvist S, McMinn A, Griffin SJ, Ogilvie D. Interventions to promote cycling: systematic review. BMJ. 2010;341.
Benmarhnia T, Rey L, Cartier Y, Clary CM, Deguen S, Brousselle A. Addressing equity in interventions to reduce air pollution in urban areas: a systematic review. Int J Public Health. 2014;59(6):933–44.
Mueller N, Rojas-Rueda D, Cole-Hunter T, de Nazelle A, Dons E, Gerike R, et al. Health impact assessment of active transportation: a systematic review. Prev Med. 2015;76:103–14.
Cesaroni G, Boogaard H, Jonkers S, Porta D, Badaloni C, Cattani G, et al. Health benefits of traffic-related air pollution reduction in different socioeconomic groups: the effect of low-emission zoning in Rome. Occupational & Environmental Medicine. 2012;69(2):133–9.
Goodman A, Panter J, Sharp SJ, Ogilvie D. Effectiveness and equity impacts of town-wide cycling initiatives in England: a longitudinal, controlled natural experimental study. [Erratum appears in Soc Sci Med. 2014 Mar;104:41. Soc Sci Med. 2013;97:228–37.
Goodman A, Sahlqvist S, Ogilvie D, iConnect Consortium. New walking and cycling routes and increased physical activity: one- and 2-year findings from the UK iConnect study. Am J Public Health 2014;104(9):e38–e46.
Jones SJ, Lyons RA, John A, Palmer SR. Traffic calming policy can reduce inequalities in child pedestrian injuries: database study. Injury Prevention. 2005;11(3):152–6.
Lindsay G, Macmillan A, Woodward A. Moving urban trips from cars to bicycles: impact on health and emissions. Australian & New Zealand Journal of Public Health. 2011;35(1):54–60.
Brownson RC, Baker EA, Boyd RL, Caito NM, Duggan K, Housemann RA, et al. A community-based approach to promoting walking in rural areas. Am J Prev Med. 2004;27(1):28–34.
Edwards P, Steinbach R, Green J, Petticrew M, Goodman A, Jones A, et al. Health impacts of free bus travel for young people: evaluation of a natural experiment in London. J Epidemiol Community Health. 2013;67(8):641–7.
Nies MA, Chruscial HL, Hepworth JT. An intervention to promote walking in sedentary women in the community. Am J Health Behav. 2003;27(5):524–35.
Panter J, Heinen E, Mackett R, Ogilvie D. Impact of new transport infrastructure on walking, cycling, and physical activity. Am J Prev Med. 2016;50(2):e45–53.
Steinbach R, Grundy C, Edwards P, Wilkinson P, Green J. The impact of 20 mph traffic speed zones on inequalities in road casualties in London. J Epidemiol Community Health. 2011;65(10):921–6.
Tonne C, Beevers S, Armstrong B, Kelly F, Wilkinson P. Air pollution and mortality benefits of the London congestion charge: spatial and socioeconomic inequalities. Occupational & Environmental Medicine. 2008;65(9):620–7.
Cairns J, Warren J, Garthwaite K, Greig G, Bambra C. Go slow: an umbrella review of the effects of 20 mph zones and limits on health and health inequalities. J Public Health. 2014.
Availability of data and supporting materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
This work was funded in part by a research grant from the Ministry for Business, Innovation and Employment (TERX1201). The funding body had no role in the design of the study, the collection, analysis, and interpretation of data, or in writing the manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hosking, J., Macmillan, A., Jones, R. et al. Searching for health equity: validation of a search filter for ethnic and socioeconomic inequalities in transport. Syst Rev 8, 94 (2019). https://doi.org/10.1186/s13643-019-1009-5
- Socio-economic status