Characteristics and use of urban health indicator tools by municipal built environment policy and decision-makers: a systematic review protocol
Systematic Reviews volume 6, Article number: 2 (2017)
There is wide agreement that there is a lack of attention to health in municipal environmental policy-making, such as urban planning and regeneration. Explanations for this include differing professional norms between health and urban environment professionals, system complexity and limited evidence for causality between attributes of the built environment and health outcomes. Data from urban health indicator (UHI) tools are potentially a valuable form of evidence for local government policy and decision-makers. Although many UHI tools have been specifically developed to inform policy, there is poor understanding of how they are used. This study aims to identify the nature and characteristics of UHI tools and their use by municipal built environment policy and decision-makers.
Health and social sciences databases (ASSIA, Campbell Library, EMBASE, MEDLINE, Scopus, Social Policy and Practice and Web of Science Core Collection) will be searched for studies using UHI tools alongside hand-searching of key journals and citation searches of included studies. Advanced searches of practitioner websites and Google will also be used to find grey literature. Search results will be screened for UHI tools, and for studies which report on or evaluate the use of such tools. Data about UHI tools will be extracted to compile a census and taxonomy of existing tools based on their specific characteristics and purpose. In addition, qualitative and quantitative studies about the use of these tools will be appraised using quality appraisal tools produced by the UK National Institute for Health and Care Excellence (NICE) and synthesised in order to gain insight into the perceptions, value and use of UHI tools in the municipal built environment policy and decision-making process. This review is not registered with PROSPERO.
This systematic review focuses specifically on UHI tools that assess the physical environment’s impact on health (such as transport, housing, air quality and greenspace). This study will help indicator producers understand whether this form of evidence is of value to built environment policy and decision-makers and how such tools should be tailored for this audience.
Systematic review registration
The impact of the urban environment on health outcomes has been widely documented [1–3], but there remains a lack of attention to health in municipal environmental policy-making, such as urban planning and regeneration [4, 5]. Researchers point to several potential explanations for the perceived lack of action from policy-makers, including differing professional norms, system complexity and limited evidence for causality between attributes of the built environment and health outcomes [2, 5, 6]. Urban health indicators are a resource which could help overcome some of these challenges and inform evidence-based municipal policies on the social determinants of health. The WHO Healthy Cities projects and a variety of other research and policy programmes have used data from urban health indicators (UHIs) as evidence to inform strategies and policies [7–10].
The use of evidence to inform policy and decision-making in municipal services that impact the wider determinants of health is an under-researched area [11, 12]. Recent studies have found that built environment professionals value data and expertise about the local context over academic evidence [12, 13]. Many indicator tools claim to be evidence-based and are underpinned by research evidence and expert involvement. Therefore urban health indicators may be one route through which research evidence informs policy, if they are used in the policy-making process . Several recent reviews of urban health indicators have focused on describing the nature of indicators and challenges with their development—none have evaluated studies on their use by policy and decision-makers [7, 9, 14, 15]. It is therefore unclear whether this form of evidence is reaching its intended audience and aiding with the process of policy and decision-making.
Urban health and environmental health indicators have traditionally focused on health outcomes and environmental risks such as pollution . The rise of non-communicable diseases and links to urban form and development patterns have broadened the scope and purpose of urban health indicators . There have been recent attempts to conceptualise the interconnections and complex relations between the environmental determinants of health and other related policy objectives such as liveability, quality of life, wellbeing and sustainability [2, 5, 7]. A number of recent indicator tools are attempting to make these relations more explicit with the intention of informing and shaping policies and decisions that meet these aligned objectives.Footnote 1 These tools provide indicators on multiple aspects of the urban environment simultaneously, therefore recognising and highlighting the complexity of the system. Examples of urban health indicators include access to recreational facilities (measured by the San Francisco Indicator Project as ‘proportion of population within 1/4 mile of a public recreation facility’) and access to public transport (measured by the Community Indicators Victoria tool as ‘average distance to nearest public transport stop including tram, bus and train stops (kilometers)’. It is these tools which could be of most use to urban planners and other municipal built environment policy and decision-makers who need to balance multiple sustainability objectives with competing interests, such as economic viability and local acceptability . Although researchers argue that indicators can be used to help policy-makers understand and respond to complex systems, this claim is not supported by research evidence on the use of indicators for this purpose [9, 19].
Defining urban health indicator tools
A number of concepts were explored in the health and urban planning literature to develop a definition for ‘urban health indicator tool’ for this review. Galea and Vlahov define urban health as ‘the determinants of health and diseases in urban areas and with the urban context itself as the exposure of interest’ . There are numerous definitions for an ‘indicator’ which vary by the policy fields for which they were created. Kotval describes an indicator as ‘a measure or a set of measures that describes a complex social, economic or physical reality’ . While Pencheon refers to indicators as ‘succinct measures that aim to describe as much about a system as possible in as few points as possible’ . This review defines an ‘urban health indicator tool’ as a collection of summary measures about the physical urban environment’s contribution to human health and wellbeing. This definition broadens the scope of ‘health’ to include related concepts of quality of life, liveability and wellbeing.
This study aims to investigate the nature and characteristics of urban health indicator tools and their perceived value and use by municipal built environment policy and decision-makers. The specific objectives are the following:
To create a census and taxonomy of urban health indicator tools
To understand how UHI tools are used in the policy and decision-making process
To explore the perceived impact of UHI tools on policy and decision-making
To investigate the value of UHI tools in relation to simplifying, representing or addressing complex systems
This systematic review will include studies and grey literature to identify UHI tools and investigate their use by policy and decision-makers. The PRISMA-P guidelines have been followed, and the checklist is available in Additional file 1. The protocol was developed iteratively following a scoping review of relevant studies and urban health indicator tools. It was informed by similar social sciences systematic reviews regarding the relevant population of policy-makers [13, 22]. Draft iterations of the protocol were shared with an advisory group of senior researchers across health, policy and built environment fields and subsequently improved. This review is not registered with PROSPERO.
There are two parts to this systematic review, with each part having distinct criteria. The first part (part A) is a census of urban health indicator tools. Any UHI tool referred to in peer-reviewed or grey literature documents will be eligible for inclusion if it meets the definition of an urban health indicator tool outlined above and is published in English. Non-English language publications are excluded due to resource constraints and this is recognised as a limitation in the review which will be considered during analysis and reporting of results. UHI tools which only regard one aspect of the physical urban environment (such as air quality) are too narrow to meet the definition provided in this protocol and will therefore be excluded.
The second part of the review (part B) relates to studies about the use of UHI tools and includes any study design (including case studies). Studies will be included if they meet the following criteria:
Reports substantive data on views, attitudes or knowledge about the use of an urban health indicator tool in the policy-making or decision-making process, or about the implementation of specific policies, interventions or programmes informed by these (modified from )
Includes policy and/or decision-makers from one of the following policy fields in local government: housing, transport, urban planning and regeneration
Reports qualitative or quantitative data
Published in English
Studies reported in any country will be included initially. It may be necessary to limit studies to those that are similar to a UK context if the cultures of practise appear to be sufficiently different. There are no date restrictions.
Two University College London (UCL) librarians specialising in systematic reviews have helped to identify the search strategy and appropriate bibliographic databases for the review. The following health and social sciences databases will be searched: Applied Social Sciences Index and Abstracts (ASSIA), Campbell Library, EMBASE, MEDLINE, Scopus, Social Policy and Practice and Web of Science Core Collection (includes Social Sciences Citation Index). In addition, a hand-search of the following key journals will be conducted: Annual Review of Public Health, Social Science and Medicine, BMC Public Health and Social Indicators Research. Citation searches of included studies will be performed. Following advice from the UCL librarian, Google Advanced Search will be used to systematically search using specified search terms, including a focus on the following practitioner websites: Town and Country Planning Association (UK), Royal Town Planning Institute (UK), Planning Institute of Australia, American Planning Association, Built Environment and Public Health Clearinghouse and the World Health Organization Europe.
The search terms and MeSH subject headings were identified through a scoping study of urban health indicator publications. Search terms and indicators identified in similar reviews were also examined and trialled to identify the key terms [7, 9, 15]. The key terms for the inclusion criteria were related to the urban environment (e.g. urban, metropolitan, city, environment, neighbourhood, community), health and related concepts (e.g. determinant, public, health, wellbeing, wellness, quality, liveability) and indicator terms (e.g. benchmark, tool, indicator, index, indices, measure, metric, profile, assessment, score, standard). In Scopus, Web of Science and Ovid (EMBASE and MEDLINE), subject areas were limited to refine results (e.g. subjects such as pharmacology and dentistry were excluded). Boolean and adjacency operators were also used to construct the search and refine results. An example of the Ovid MEDLINE search is included in Additional file 2.
Data management and selection
All records will be imported into EppiReviewer, specialist systematic review software, and duplicates will be removed. A second reviewer (KG) will screen a randomly selected sample of 10% of titles and abstracts. Inter-rater agreement percentages will be reported for both screening stages. Conflicts will be discussed and agreed upon with a third researcher (HR). Records will be removed at this stage if they do not meet the inclusion criteria for part A (i.e. they do not mention an urban health indicator tool or are not published in English). The second reviewer will screen a randomly selected sample of 10% of full papers. This pool of studies will include records for part A and part B. Full papers will be screened simultaneously for the inclusion criteria in part A and part B. The result will be a set of included urban health indicator tools and a set of included studies about the use of these tools.
For each included urban health indicator tool (identified in part A), we will use the provided references or a Google search to determine if there is a stand-alone website and/or further documentation about the indicator and its methodology. The data will be extracted from the indicator tool website or other documentation by the indicator tool producer where possible. The preference is to find information about the indicator tool directly from its producer rather than third-party summaries or evaluations. If the information is only available through the included study, then the data will be extracted from there. The information source will be logged as part of the data extraction. A draft data extraction form in Excel has been developed (see Additional file 3). The form was iteratively developed using information from the scoping review. The data extraction will include the following categories (developed from the scoping review):
Scale—At what scales can the system be applied or measured? (e.g. neighbourhood or city)
Geography—Which areas can this system be applied in (e.g. specific cities or nations)?
Scope—What aspects are analysed (e.g. built environment, health outcomes, demographics)?
Producer—Which organisation developed the system? What type of organisation?
Funders—Which organisations funded the indicator system?
Purpose—What is the stated purpose? (e.g. research and/or informing policy)
Methodology—Is there a published methodology and what are its characteristics?
Evidence base—Does the methodology refer to evidence which was used to inform the system? What is the nature of this evidence?
Weighting—Is there a weighting system and what are its characteristics?
Complexity—Does the methodology refer to complexity and, if so, in what context?
Uncertainty—Does the methodology refer to uncertainty and, if so, in what context?
Maps—Is there an option to view the data on maps?
Publication date—When was the system published?
Source—Where was this information found?
Indicators—Which indicators are reported?
Studies that meet inclusion criteria for part B will be included in a narrative synthesis. The following data will be extracted in an Excel sheet (see Additional file 4) for each study included in part B (informed by [13, 22]):
Year that study was carried out
Urban health indicator tool being evaluated
Data collection methods
Population and sample selection
Conflicts of interest
A data extraction form has been created in Excel and any quantitative data will be analysed using Excel. Qualitative data will be synthesised using NVivo qualitative data analysis Software (QSR International Pty Ltd., Version 11, 2015). Data will be coded using an open code set. These will be updated in an iterative process as new factors regarding the perceptions and use of urban health indicator tools are identified.
Studies about the use of UHI tools (part B) will be appraised using the quality appraisal tools for qualitative and quantitative studies produced by the UK National Institute for Health and Care Excellence (NICE) . A modified version of the NICE tool for quantitative studies reporting correlations and associations will be used as there is no suitable version for study designs reporting quantitative data on participants’ perceptions. For qualitative studies, the NICE appraisal checklist includes assessment of the following: theoretical approach, study design, data collection, trustworthiness, analysis and ethics. For quantitative studies, the following topics are appraised: population, method of selection, outcomes, analyses and summary. A copy of the completed checklists will be published with the review results as an additional file.
There will be two components to the data synthesis. Data about the UHI tools will be analysed to create a taxonomy of the types of tools available to municipal built environment policy and decision-makers. This will include quantitative analysis of the physical urban environment features being measured, such as the percentage of UHI tools measuring particular features. The analysis will also assess the proportion of tools operating in particular geographic scales, the number of tools published each year, the proportion of UHI tools which mention complexity and the proportion of tools developed by different organisation types. The narrative synthesis of qualitative data from part B of this review will identify any recurrent themes across the studies regarding the perceptions and value of urban health indicator tools by policy and decision-makers.
Many urban health indicator tools are created with the goal of informing policy- and decision-makers who influence the social determinants of health in urban environments. However, there is little clarity about what type of evidence this audience uses and whether urban health indicator tools form part of their evidence toolbox. This review focuses specifically on municipal built environment policy and decision-makers and their perceptions of UHI tools. The findings will be of value to UHI tool producers who wish to target their tools to this audience with the aim of improving the health impact of urban environments.
Complexity is emphasised as a key challenge in relation to policy-making for health and the built environment, in response to which indicators have been promoted as a solution. This review will seek to understand whether and how UHI tools aim to address the complexity of the systems they measure. An initial scoping review showed that this topic is not widely addressed in indicator methodology documents. This review will seek to understand whether UHI tools are perceived as assisting with complexity in the policy and decision-making process. However, it is recognised that studies identified for this review may not address this topic. Therefore, this review will also help to establish the current research evidence supporting the claim that indicators are a tool to support policy and decision-makers operating in this complex system.
For example, see the San Francisco Indicator Project developed by the SF Department of Public Health and the City and County of San Francisco (http://www.sfindicatorproject.org/ accessed 29 Mar 2016) and the Community Indicators Victoria tool developed by McCaughey Centre, VicHealth and the University of Melbourne (http://www.communityindicators.net.au/ accessed 17 Feb 2016).
Applied Social Sciences Index and Abstracts
Medical Subject Headings
National Institute for Health and Care Excellence
Preferred Reporting Items for Systematic review and Meta-Analysis Protocols
University College London
Urban health indicator
Galea S, Vlahov D. Urban health: evidence, challenges, and directions. Annu Rev Public Health. 2005;26:341–65.
Rydin Y, Bleahu A, Davies M, Dávila JD, Friel S, De Grandis G, et al. Shaping cities for health: complexity and the planning of urban environments in the 21st century. Lancet. 2012;379:2079–108.
World Health Organisation. Closing the gap in a generation: health equity through action on the social determinants of health. Geneva: World Health Organization; 2008.
Barton H. Healthy urban planning: setting the scene. Built Environ. 2005;31:281–7.
Verbeek T, Boelens L. Environmental health in the complex city: a co-evolutionary approach. J Environ Plan Manag. 2016;59(11):1–20.
Sarkar C, Webster C, Gallacher J. UK Biobank Urban Morphometric Platform (UKBUMP)—a nationwide resource for evidence-based healthy city planning and public health interventions. Ann GIS. 2015;21:135–48.
Badland H, Whitzman C, Lowe M, Davern M, Aye L, Butterworth I, et al. Urban liveability: emerging lessons from Australia for exploring the potential for indicators to measure the social determinants of health. Soc Sci Med. 2014;111:64–73.
Bhatia R. Case study: San Francisco’s use of neighborhood indicators to encourage healthy urban development. Health Aff. 2014;33:1914–22.
Lawrence RJ. Urban environmental health indicators: appraisal and policy directives. Rev Environ Health. 2008;23:299–325.
Lowe M, Whitzman C, Badland H, Davern M, Aye L, Hes D, et al. Planning healthy, liveable and sustainable cities: how can indicators inform policy? Urban Policy Res. 2015;33:131–44.
Phillips G, Green J. Working for the public health: politics, localism and epistemologies of practice. Sociol Health Illn. 2015. doi:10.1111/1467-9566.12214.
McGill E, Egan M, Petticrew M, Mountford L, Milton S, Whitehead M, et al. Trading quality for relevance: non-health decision-makers’ use of evidence on the social determinants of health. BMJ Open. 2015;5:e007053.
Lorenc T, Tyner EF, Petticrew M, Duffy S, Martineau FP, Phillips G, et al. Cultures of evidence across policy sectors: systematic review of qualitative evidence. Eur J Pub Health. 2014;24:1041–7.
Prasad A, Gray CB, Ross A, Kano M. Metrics in urban health: current developments and future prospects. Annu Rev Public Health. 2016. doi:10.1146/annurev-publhealth-032315-021749.
Rothenberg R, Stauber C, Weaver S, Dai D, Prasad A, Kano M. Urban health indicators and indices—current status. BMC Public Health. 2015;15:494.
Briggs DJ. Environmental health indicators: framework and methodologies. World Health Organisation; 1999. http://apps.who.int/iris/handle/10665/66016. Accessed 26 Nov 2015.
Northridge DME, Sclar DED, Biswas MP. Sorting out the connections between the built environment and health: a conceptual framework for navigating pathways and planning healthy cities. J Urban Health. 2003;80:556–68.
Durand CP, Andalib M, Dunton GF, Wolch J, Pentz MA. A systematic review of built environment factors related to physical activity and obesity risk: implications for smart growth urban planning. Obes Rev. 2011;12:e173–82.
Innes JE, Booher DE. Indicators for sustainable communities: a strategy building on complexity theory and distributed intelligence. Plann Theory Pract. 2000;1:173–86.
Kotval Z. Measuring the effectiveness of downtown revitalization strategies. In: Balsas CJL, editor. Urbanismo Comercial em Portugal: a necessidade de uma nova gestão urbana actas. Lisbon: URBE & CMPV; 2001. p. 43–9.
Pencheon D. The good indicators guide: understanding how to use and choose indicators. NHS Institute for Innovation and Improvement; 2008. http://www.swpho.nhs.uk/resource/item.aspx?RID=56629. Accessed 7 Jun 2015.
Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.
National Institute for Health and Care Excellence. Methods for the development of NICE public health guidelines (Third Edition). 2012. https://www.nice.org.uk/process/pmg4/chapter/introduction. Accessed 13 Nov 2015.
The authors would like to thank Karen Lock and Kathryn Oliver for their contribution to the study design.
The majority of the research was undertaken through an industry sponsored PhD. The industrial sponsor is the Building Research Establishment Ltd.
Availability of data and materials
HP designed the study, undertook the scoping review, performed the searches, imported results into EppiReviewer, screened results and wrote the protocol. NZ, KG, HR, PW and MD advised on the study design and contributed to the writing of this protocol. All authors read and approved the final manuscript.
HP is a PhD student funded by the Building Research Establishment Ltd. The other authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
PRISMA-P Checklist. Completed PRISMA-P Checklist. (DOCX 37 kb)
Title of data: MEDLINE search. MEDLINE (Ovid) search strategy. (DOCX 12 kb)
Data extraction form part A. Data extraction form part A. (XLS 33 kb)
Data extraction form part B. Data extraction form part B. (XLS 22 kb)
About this article
Cite this article
Pineo, H., Glonti, K., Rutter, H. et al. Characteristics and use of urban health indicator tools by municipal built environment policy and decision-makers: a systematic review protocol. Syst Rev 6, 2 (2017). https://doi.org/10.1186/s13643-017-0406-x
- Urban metrics
- Built environment
- Urban health
- Social determinants of health