Implementing health promotion in schools: protocol for a realist systematic review of research and experience in the United Kingdom (UK)
© Pearson et al.; licensee BioMed Central Ltd. 2012
Received: 1 August 2012
Accepted: 5 October 2012
Published: 20 October 2012
School-based interventions and campaigns are used to promote health and address a wide variety of public health problems. Schools are considered to be key sites for the implementation of health promotion programmes for their potential to reach the whole population in particular age-groups and instil healthy patterns of behavior early in life. However, evidence for the effectiveness of school-based health promotion interventions is highly variable. Systematic reviews of the evidence of school-based interventions tend to be highly problem- or intervention- specific, thereby missing potential generic insights into implementation and effectiveness of such programmes across problems.
A realist systematic review will be undertaken to explain how, why and in what circumstances schools can provide feasible settings for effective health promotion programmes in the United Kingdom (UK). The review will be conducted in two phases. Phase 1 will identify programme theories about implementation (ideas about what enables or inhibits effective health promotion to be delivered in a school setting). Phase 2 will test the programme theories so that they can be challenged, endorsed and/or refined. A Review Advisory Group of education and health professionals will be convened to help identify and choose potential programme theories, provide a ‘reality check’ on the clarity and explanatory strength of the mechanisms to be tested, and help shape the presentation of findings to be usable by practitioners and decision-makers. Review findings will be disseminated through liaison with decision-makers, and voluntary and professional groups in the fields of education and health.
KeywordsHealth promotion Schools Implementation Realist review Systematic review
they are universal, capturing the whole population in the relevant age-group 
they capitalize on school children being a captive audience
schools and schoolchildren can be used as a catalyst for change in families and the wider community 
in schools, some sensitive health issues, such as sexual health and contraception, can be handled without parental oversight
teachers or school peers may be more effective at delivering certain types of health message or changing health-related attitudes than others; conversely, outsiders in the classroom may work better
Key school or wider environmental contexts.
Required steps in the implementation process.
The nature of health promotion activities which are suited to different types of schools.
A realist systematic review aims to attain a contextualized understanding of how and why complex interventions achieve particular effects (in realist language, how mechanisms lead to outcomes in particular contexts). This understanding is pursued by testing ‘programme theories’, which are often expressed as a model linking outcomes to programme activities and the underlying theoretical assumptions . Contained within programme theories, even if not explicitly stated, are ideas about how a problem can be best addressed and how factors that may undermine the actions of the programme can themselves be addressed . Realist review methods have been specifically advocated for evaluating evidence about complex interventions and their implementation [5, 8], and can provide explanatory insight into variations in the effectiveness of programmes that, for example, a Cochrane review of randomized controlled trials may be less likely to provide .
practicalities of fitting a programme into the timetable of the school day
availability of specialist equipment or materials
beliefs and attitudes of teachers and head teachers about the importance of the health problem
content and change targets of the programme
beliefs, attitudes and perceptions of the people delivering the programme
competencies of the people delivering the programme
mode of delivery of the programme
beliefs, attitudes and other characteristics of the school children’s parents
beliefs and attitudes of school children (at different ages) about the importance of health problems or specific behaviors
development of behavioral skills of the children themselves
compatibility with subjects within the National Curriculum or the goals of Personal, Social, and Health and Education (PSHE) and with other initiatives, such as Schools4life or Social and Emotional Aspects of Learning (SEAL)
There are two related and ongoing systematic reviews led by UK-based researchers: (1) Langford et al., a Cochrane systematic review evaluating the effectiveness of the World Health Organization (WHO) Health Promoting School framework; and (2) Bonell et al., on the health effects of school environments and school environment-based interventions to improve health.
Our inclusion/exclusion criteria are designed to focus the review on the implementation processes of gaining access to schools and working within schools to promote health. This focus will enable the review to maximize the insights gained by looking across health promotion for different topics conducted in schools, and aims to complement the ongoing Langford et al.  and Bonell et al. systematic reviews. We recognize that the extent to which a focus on the implementation (including feasibility and sustainability) of public health programmes can be a separate focus of systematic reviews is a current area of methodological debate and development .
In addition to the specific focus of programme implementation, Phase 2 of the review will be exclusively focused on explaining the success of such initiatives in UK schools. This geographical focus is for two reasons. First, it reflects the policy and practice focus of the research funding body (the National Institute for Health Research (NIHR) School for Public Health Research). Second, and arguably more importantly, it reflects a presumption that many of the issues and mechanisms that determine the implementation of health promotion in schools will be context-specific. They will depend on aspects of school life, the curriculum, socio-economic variations and diversity in communities, which are specific to UK primary and secondary schools.
To inform this realist systematic review, we have already conducted a rapid review of systematic reviews to identify the key dimensions on which school-based health promotion programmes differ and identify some initial programme theories (ideas about how programmes aim to achieve their goals).
This systematic review is registered on the PROSPERO database (registration number: CRD42012002640).
The objective is to use a theory-driven evidence synthesis to identify what influences successful implementation of health promotion in UK schools. The review will have two phases. Phase 1 will identify programme theories about implementation (ideas about what enables or inhibits effective health promotion to be delivered in a school setting) from a range of published and other sources. Phase 2 will test these programme theories, using published and unpublished empirical evidence, through the process of reasoning, as detailed under ‘Data synthesis’.
What are the main factors or mechanisms that are thought to explain the success or failure of the implementation of health promotion in schools?
Is there an association between these factors and mechanisms and the successful implementation of health promotion in schools?
What public health problems and in what circumstances do schools provide a feasible and sustainable setting for effective health promotion in the UK?
There will be a number of search phases for Phase 1 (identification of programme theories about implementation in any Organisation for Economic Co-operation and Development (OECD country)) and Phase 2 (testing of programme theories about implementation in the UK only) of the review.
An initial search will be guided by search terms specifying the type of interventions of interest (for example health promotion, health education and health improvement) and the setting (primary or secondary schools). The search will be designed to locate a broad range of sources, including editorials, opinion pieces and commentaries (primarily for identifying programme theories); as well as comparative effectiveness studies, process evaluations and qualitative research (primarily for testing programme theories). Subsequent bibliographic database searches will be developed to locate sources relating to specific aspects of programme theories not identified in the initial search and to locate studies that will enable selected programme theories to be tested.
The development of search terms will also be informed by the findings of our rapid review of reviews and input from our Review Advisory Group. Scoping and test searches will explore the value and limitations of using country filters. The iterative nature of a theory-driven review means that refinements to the search strategy may be required in response to emerging findings. However, an example of the initial search strategy is included in the search protocol (Additional file 1).
The databases to be searched will include, but not necessarily be limited to: Applied Social Science Index and Abstracts (ASSIA), CINAHL, MEDLINE, PsycINFO, Social Science Citation Index (SSCI), Science Citation Index (SCI), Social Policy and Practice; selected Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) databases, for example Current Education and Children's Services Research UK (CERUKplus); British Education Index, Education Resources Information Center (ERIC), Database of Abstracts of Reviews of Effectiveness (DARE), Health Technology Assessment (HTA), the Cochrane Database of Systematic Reviews (all in the Cochrane Library) and the Campbell Collaboration. Ongoing programmes from the UK or unpublished evaluations of health promotion in UK schools will be identified by searching NIHR, Department of Health, Department for Education and other research registers or programme funders, where they can be feasibly searched.
citations contained in the reference lists of included papers
sources identified through contact with professional networks
‘cited by’ articles search
Study inclusion criteria (Phase 1)
Sources that provide rich descriptions of the delivery of school-based health promotion for children aged 5 to 16 years in any OECD country. These are likely to include editorials, opinion pieces, commentaries, comparative effectiveness studies, process evaluations, qualitative research, programme manuals and systematic reviews. We will include health promotion programmes and activities that are delivered to school children primarily in a school setting, including those outside of school hours.
Since the aim of Phase 1 is to locate a range of programme theories to test, it is not appropriate to foreclose potentially important perspectives by pre-specifying exclusion criteria.
Study inclusion criteria (Phase 2)
Children aged 5 to 16 years attending primary or secondary schools in the UK (both state schools and private schools (so-called ‘public’ schools), regardless of Academy status). Since there is no uniformity in the organization of secondary and further education settings, a programme aimed at children aged 5 to 16 years, but which includes a minority of school children aged 17 or 18 years, would still be included.
Non-UK schools, pre-schools (and pre-school-age children), sixth form and further education colleges (because most children will be over 16-years-old and the structure, education culture and organization will be different from secondary schools).
Health promotion programmes that are being, or have been, delivered in UK primary or secondary schools.
Community-based health promotion programmes, where the school is one of several local settings for promoting the health of both adults and children. In practice, this means that if the primary starting point for a programme is not a school, it would be excluded.
Children aged 5 to 16 years attending schools in the UK, where a different health promotion programme is delivered, or no health promotion programme is delivered.
Since the testing of programme theories will involve a range of study designs, including process evaluations and qualitative research, the comparator criteria will only be applied to comparative effectiveness studies.
Types of study to be included
Comparative effectiveness studies, where intervention and comparator groups are assigned by the research (randomized and non-randomized controlled trials).
Process evaluations, which aim to document how and why school-based health promotion programmes are successfully implemented or not, or are effective or not. These may be quantitative or qualitative, and alongside a comparative effectiveness study or stand-alone.
If a low number of comparative effectiveness studies are located, we shall consider including (in order of preference) uncontrolled before and after studies, and cohort studies.
standardized measures of physical, emotional or mental health
intermediate measures of health, for example observed or self-reported health behaviors or body mass index
health-related knowledge and attitudes
Quality assessment (Phase 1)
We shall use a hybrid appraisal tool based on previous critical appraisal work [12, 13], which enables sources to be classified as a conceptually-rich (thick) or thin (weaker) description. We found this tool to be practical and useful in a recent theory-driven review  for the way in which it enabled a focus on the stronger sources of programme theories without simply excluding weaker sources that may still make an important, if lesser, contribution.
Quality assessment (Phase 2)
For assessing the internal validity of comparative effectiveness studies we propose to use a standard tool for assessing the risk of bias (threats to internal validity). We aim to use the most up-to-date version of the Cochrane risk of bias tool, or an adaptation of it suitable for both randomized and non-randomized studies.
Other study types, such as process evaluations, will be assessed using a modified version of the Wallace et al. quality appraisal tool for qualitative studies . This tool encompasses key components of rigor (for example sampling, data collection and data analysis) that are relevant across different fields of research practice. Importantly, given that the number of included studies may be large, the Wallace et al. tool focuses on key elements of critical appraisal without becoming so detailed as to become unusable in the context of conducting a review to a reasonable timescale. This tool will also enable us to pinpoint whether or not certain aspects of a study are of higher or lower quality. This is particularly important in a realist review where both relevance and rigor are considered in tandem, meaning that an otherwise poorly conducted study may contribute to the synthesis if the aspect concerned is of sufficient rigor.
Data extraction (Phase 1)
Consistent with a theory-driven approach, sources that contribute to theory development will be ‘engaged with’ by the reviewers (through a process of note-taking, annotation and conceptualization) rather than ‘extracted’. At the same time we will refine our conceptualization of the key stages, factors or processes which constitute health programme implementation in UK schools.
Data extraction (Phase 2)
At the stage of testing the programme theories, relevant data/information from included studies and reports will be extracted to a standard data extraction form. A sample of these will be checked by the other reviewer for accuracy, for those data extraction fields which involve quantitative data or key information, such as study design. The data extraction process itself will involve critical discussion between reviewers and the wider team so that data are not simply ‘classified’ but are used to begin to develop a line of argument that feeds into the final synthesis stage.
juxtaposition of sources of evidence, for example when evidence about implementation in one source enables insights into evidence about outcomes in another source
reconciling of sources of evidence, when results differ in apparently similar circumstances, further investigation is appropriate in order to find explanations for why these different results occurred
adjudication of sources of evidence, on the basis of methodological strengths or weaknesses
consolidation of sources of evidence, when evidence about mechanisms and outcomes is complementary and enables a multi-faceted explanation to be built
situating sources of evidence, when outcomes differ in particular contexts, an explanation can be constructed of how and why these outcomes occur differently
The transparency of a synthesis in a realist review is achieved by documenting these reasoning processes, describing how they are grounded in the empirical evidence and the justification of inferential shifts that occur through this engagement with the evidence.
In addition for Phase 2, we aim to code studies according to: (1) the presence/absence of selected theoretical mechanisms and contexts that are believed to promote successful/unsuccessful programme implementation in schools, and (2) markers of successful implementation. For example, the proportion of schools approached who accepted the intervention and the proportion of schools that initially accepted the intervention, but went on to withdraw or not deliver it fully in some way. As a possible final phase of evidence synthesis, these combinations of mechanisms, contexts and markers of implementation success may be used to try and explain between study (or between programme) differences in programme effectiveness. We hope that this might be achieved through tabulating these data and findings, graphically exploring potential associations or possibly by more formal techniques, such as Qualitative Comparative Analysis [17, 18].
Review advisory group
As well as being based on documentary sources and their synthesis, as described in this protocol, the selection and prioritization of programme theories will be informed by two or three meetings with a Review Advisory Group. It is intended that at least three members of the Review Advisory Group will be directly from schools or the education sector (for example local education authority department, and/or head teachers, PSHEleaders or special educational needs co-ordinators). The Review Advisory Group will help identify and choose amongst different potential programme theories, and provide a ‘reality check’ on the clarity and explanatory strength of the selected theories/mechanisms which become a focus of Phase 2 of the review.
Separately from the Review Advisory Group, we shall also involve and share emerging findings with some school children, through School Councils or recruited through a local ‘family faculty’ (which has been established for the purposes of public involvement in other research conducted by Peninsula Medical School, Universities of Exeter and Plymouth).
Primary and secondary education, for example British Educational Research Association (BERA) and National Association for Special Educational Needs (NASEN).
Behavior change, for example UK Society for Behavioural Medicine (UKSBM) and The British Psychological Society (BPS).
Public health, for example The Royal Society for Public Health (RSPH).
Mental health, for example Association of Child and Adolescent Mental Health (ACAMH), Royal College of Psychiatrists (RCPsych) and YoungMinds.
These lists will be expanded as part of the work of the Review Advisory Group. Other key decision-makers who contribute to the policy making process (for example in the Department of Health and Department for Education) will be identified so that dialogue can be initiated and taken forward with regard to the review’s findings.
Applied Social Sciences Index and Abstracts
Current Education and Children's Services Research UK
Database of Abstracts of Reviews of Effectiveness
Evidence for Policy and Practice Information and Co-ordinating Centre
Educational Resources Information Center
Health Technology Assessment
National Institute for Health Research
Organisation for Economic Co-operation and Development
Personal Social, Health education
Science Citation Index
Social and Emotional Aspects of Learning
Social Science Citation Index
World Health Organization.
The systematic review for which this protocol has been developed is funded by the NIHR School for Public Health Research. The views expressed in this publication are those of the authors and not necessarily those of the National Health Service, NIHR or the Department of Health.
- World Health Organization (WHO): Promoting Health Through Schools: Report of a WHO Expert Committee on Comprehensive School Health Education and Promotion. 1997, WHO, Geneva
- Greenberg M, Domitrovich C, Graczyk P, Zins J: The Study of Implementation in School-based Preventive Interventions: Theory, Research, and Practice (Volume 3). 2005, Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, Rockville
- Brown T, Summerbell C: Systematic review of school-based interventions that focus on changing dietary intake and physical activity levels to prevent childhood obesity: An update to the obesity guidance produced by the National Institute for Health and Clinical Excellence. Obes Rev. 2009, 10: 110-141. 10.1111/j.1467-789X.2008.00515.x.View ArticlePubMed
- Lister-Sharp D, Chapman S, Stewart-Brown S, Sowden A: Health promoting schools and health promotion in schools: two systematic reviews. Health Technol Assess. 1999, 3: 1-207.PubMed
- Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review: a new method of systematic review for complex policy interventions. J Health Serv Res Policy. 2005, 10: 21-34. 10.1258/1355819054308530.View ArticlePubMed
- WK Kellogg Foundation: Logic Model Development Guide: Using Logic Models to Bring Together Planning, Evaluation, and Action. 2004, WK Kellogg Foundation, Michigan
- Pawson R, Owen L, Wong G: The Today Programme's contribution to Evidence-Based Policy. Evaluation. 2010, 16: 211-213. 10.1177/1356389010369636.View Article
- Berwick DM: The science of improvement. J Am Med Assoc. 2008, 299: 1182-1184. 10.1001/jama.299.10.1182.View Article
- Greenhalgh T, Kristjansson E, Robinson V: Realist review to understand the efficacy of school feeding programmes. Br Med J. 2007, 335: 858-861. 10.1136/bmj.39359.525174.AD.View Article
- Langford R, Campbell R, Magnus D, Bonell CP, Murphy SM, Waters E, Komro KA, Gibbs LF: The WHO Health Promoting Schools framework for improving the health and well-being of students and staff. Cochrane Database Syst Rev. 2011, 1: CD008958-
- Bonell CP, Harden A, Wells H, Jamal F, Fletcher A, Petticrew M, Thomas J, Whitehead M, Campbell R, Murphy S, Moore L: Protocol for a systematic review of the effects of schools and school-environment interventions on health: evidence mapping and syntheses. BMC Publ Health. 2011, 11: 453-10.1186/1471-2458-11-453.View Article
- Roen K, Arai L, Roberts H, Popay J: Extending systematic reviews to include evidence on implementation: methodological work on a review of community-based initiatives to prevent injuries. Soc Sci Med. 2006, 63: 1060-1071. 10.1016/j.socscimed.2006.02.013.View ArticlePubMed
- Ritzer G: Meta-theorizing in Sociology. 1991, Lexington Books, New York
- Pearson M, Hunt H, Cooper C, Shepperd S, Pawson R, Anderson R: Intermediate care: a realist review and conceptual framework. 2012, Final report. NIHR Service Delivery and Organisation Programme
- Wallace A, Croucher K, Bevan M, Jackson K, O'Malley L, Quilgars D: Evidence for policy making: some reflections on the application of systematic reviews to housing research. Hous Stud. 2006, 21: 297-314. 10.1080/02673030500484935.View Article
- Pawson R: Evidence-Based Policy: A Realist Perspective. 2006, Sage Publications, London
- Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods. 2007, Open University Press, Maidenhead
- Ragin C: The comparative method: moving beyond qualitative and quantitative strategies. 1987, University of California Press, Los Angeles
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.