Skip to main content
Help us understand how you use our websites. Take part in our 30 minute study now.

How effective are digital interventions in increasing flu vaccination amongst pregnant women? A systematic review protocol



Pregnant women and their unborn babies are at increased risk of complications as a result of flu, yet uptake of the flu vaccination in the UK remains low. Digital interventions have proven effectiveness in changing health behaviour, but their effectiveness in increasing flu vaccination amongst pregnant women has not been examined. This protocol details the design and methodology of a systematic review and meta-analysis, examining the effectiveness of digital interventions in increasing flu vaccination amongst pregnant women.


Bibliographic databases will be searched using appropriate search terms related to vaccination, pregnancy and flu. Randomised, non-randomised, quasi randomised controlled trials and other quantitative study designs will be eligible for inclusion, and studies will present the rate of flu vaccination amongst pregnant women of digital interventions compared to non-digital interventions, or usual care. No date or study country restrictions will be put on included studies, but studies will be published in English.


This is the first known systematic review to examine the effectiveness of digital interventions in increasing the rate of flu vaccination amongst pregnant women. This review has the potential to inform whether digital interventions are an appropriate and successful method of increasing flu vaccination amongst pregnant women, and to determine which mode of digital intervention is most effective.

Trial registration

This systematic review is registered on the international prospective register of systematic reviews (PROSPERO). Registration number pending.

Peer Review reports


Pregnant women and their unborn babies are at increased risk of complications as a result of flu, with pregnant women being at increased chance of hospitalisation and death due to physiological and immunological changes that happen during pregnancy [1,2,3], and unborn babies being at increased risk of premature birth, stillbirth and below average birthweight as a result of maternal flu [1]. Despite the increased risk of severe consequences from flu, and despite the flu vaccination having a good record of safety and effectiveness [4,5,6], uptake of the flu vaccination in England is annually below national targets, with 45.2% of pregnant women receiving the flu vaccination in the 2018/2019 flu season [7].

Access to internet use has increased rapidly over recent years, with 87% of all adults using the internet daily or almost daily during 2019, and with almost two thirds of households in the UK having access to mobile broadband [8]. The popularity of internet use is expected to continue to increase, with nearly 54 million people estimated to use an internet enabled smartphone in the UK by 2022 [9].

Digital interventions are one approach to increasing health behaviours that often utilise internet capabilities. Digital interventions have proven effectiveness in increasing behaviours such as smoking cessation [10], physical activity [11] and asthma self-management [12]. If digital interventions are effective in increasing flu vaccination amongst pregnant women, it suggests a mechanism for primary care services to impact the health of pregnant women and unborn babies, and reduce associated costs resulting from maternal flu. For the purposes of this review, the term digital behavioural interventions is defined as an intervention that attempts to change pregnant women’s vaccination behaviour that are delivered via a digital or mobile device directly to participants. This includes text messages (including text, video or audio based messages), interventions delivered by the internet (including by websites, mobile applications (apps) or social media sites), or other digital strategies [13].

To date, the effectiveness of digital interventions at increasing the rate of flu vaccination amongst pregnant women have yet to be determined. This review aims to establish whether digital interventions are effective at increasing flu vaccination amongst pregnant women, and to determine the size of the effect.

Research question

How effective are digital interventions in increasing flu vaccination amongst pregnant women?


  1. 1.

    To examine the effectiveness of digital interventions in increasing flu vaccination amongst pregnant women.

  2. 2.

    To compare the effectiveness of different types of digital interventions in increasing flu vaccination amongst pregnant women


Completed PRISMA-P checklist, showing recommended items to include in a systematic review protocol, and where in the protocol each item is located, can be found in Additional file 1.

Eligibility criteria

Studies will be included that test the effectiveness of digital behavioural interventions in increasing the rate of flu vaccination amongst pregnant women. Comparators in included studies will be either usual care, a wait-list comparator, a historical control group where the digital intervention is not present, a digital intervention that is not about flu vaccination, or comparison to a non-digital intervention. The outcome being studied is the rate of flu vaccination amongst pregnant women. Only original research studies will be included, with any systematic reviews, protocols, commentaries and conference abstracts being excluded. Included studies will be either a randomised controlled trial, a non-randomised controlled trial, quasi-randomised controlled trial or other type of quantitative study that reports the rate of flu vaccination (for example, before and after trials) following the implementation of a digital intervention, which also contains a comparator group. Quantitative studies such as case series, case studies and case reports will also be excluded. No date or country restrictions will be placed on the search, but studies will be required to be published in English (Table 1).

Table 1 Inclusion and exclusion criteria

Outcome measures

The outcome measure will be the rate of flu vaccination amongst pregnant women after receiving a specific digital intervention, compared to the comparator group. This will either be self-reported flu vaccination status or rate obtained from electronic patient records.

In studies where the main outcome is intention to vaccinate rather than actual vaccination behaviour (uptake of vaccination), this rate will be used for this review.

Information sources

Electronic bibliography databases will be searched during April and May 2020. MEDLINE, Embase, Web of Science, Scopus, Cochrane database, PsycINFO, and Cochrane Central Register of Controlled Trials (CENTRAL) will be searched for all eligible studies. Clinical trial registers will be searched for in-progress trials (e.g. ClinicalTrials.Gov) Search terms will include all possible terms relating to ‘vaccination’, ‘influenza’, ‘pregnancy’, and variations of ‘digital interventions’ to include interventions that contain significant influence from text messages, video, internet, or mobile phone applications (apps) [13, 14]. Boolean strategies of ‘AND’ and ‘OR’ will be employed.

Reference sections of included studies will also be screened by hand to identify any other eligible studies, and papers citing included studies will be searched.

Search strategy

Specialist advice on the search strategy has been sought in developing the search strategy for this review, to ensure searches are comprehensive and capture all relevant studies in the searches. An example of the full search strategy for one database can be found in Additional file 2. Keywords will be searched in both titles and abstracts.

Data management and screening process

Results of searches will be managed using Endnote software. Results from all databases will be combined, alphabetised and duplicates will be removed.

The first stage of screening will consist of all titles and abstracts being screened. Any studies that appear to meet the inclusion criteria will be subject to stage two of screening, which will consist of full text screening, where full papers will be obtained and screened against the eligibility criteria. Screening will be conducted by two researchers independently, and any discrepancies will be resolved with discussion. If a consensus is not reached, advice will be sought by a third researcher, until a full and final set of studies that meet the inclusion criteria are obtained. In addition to this, reference sections of all included studies will be screened for any studies that were not captured by the searches.

Data will be extracted from all included studies. This will be conducted by two researchers independently, using a pre-defined extraction form. Eligibility for inclusion in the meta-analysis will also be determined. The following information will be extracted from each included study: name of author, year of publication, study design, study setting, participants, details about intervention (including the mode of digital intervention such as text message, video, mobile phone app), comparison/control condition, rate of flu vaccination, and size of effect of intervention (if reported). Any discrepancies between data extraction carried out by the two researchers will be discussed, and a third author will be consulted if a consensus is not reached.

Quality assessment

Risk of bias in Randomised Controlled Trials will be assessed using the Cochrane Risk of Bias tool [15]. Studies will be assessed on the domains below and classified as either low risk, medium risk, or high risk of bias:

  1. 1.

    Sequence generation

  2. 2.

    Allocation concealment

  3. 3.

    Blinding of participants, personnel and outcome assessors

  4. 4.

    Incomplete outcome data

  5. 5.

    Selective outcome reporting

  6. 6.

    Other sources of bias

Risk of bias in non-randomised controlled trials (including quasi-randomised controlled trials and quantitative studies) will be assessed using the Cochrane Risk of Bias in Non-randomised Studies of Interventions [16]. Studies will be assessed on the domains below and classified as either low risk, moderate risk, serious risk, critical risk of bias, or no information to make a judgement:


  1. 1.

    Bias due to confounding

  2. 2.

    Bias in selection of participants into the study

At intervention

  1. 3.

    Bias in classification of intervention

Post intervention

  1. 4.

    Bias due to deviation from intended intervention

  2. 5.

    Bias due to missing data

  3. 6.

    Bias in measurement of outcomes

  4. 7.

    Bias in selection of the reported result

Data synthesis

Key information that is extracted from each included study will be synthesised. This will consist of descriptive information about the type and content of intervention in each condition as presented in the paper. The rate of flu vaccination will also be extracted from included studies and will be synthesised and discussed to determine if digital interventions are effective at increasing flu vaccination amongst pregnant women. Discussion about the risk of bias of included papers will also be included.

Data analysis

Statistical information about the rate of flu vaccination in pregnant women will be presented, to establish whether digital interventions are more effective at increasing flu vaccination amongst pregnant women, than other types of interventions or comparison groups.

If studies report the rate of intention to vaccinate instead of actual vaccination behaviour (vaccination uptake), this data will be reported instead. Separate analysis for vaccination and intention to vaccinate will be conducted. If studies report both vaccination rate and intervention to vaccinate rate, then the actual vaccination rate will be used.

If sufficient statistical information is provided by included studies, a meta-analysis will be conducted to establish the pooled and weighted size of the effect of digital interventions in increasing flu vaccination amongst pregnant women. Standardised study effect sizes, using standardised mean differences (with 95% confidence intervals) will be calculated for the increase of flu vaccination rates in digital intervention conditions, compared to other intervention types or control/comparison conditions, using a random-effects meta-analysis model, where the statistical information provided allows. If applicable, separate meta-analyses will be conducted for vaccination intention and actual vaccination uptake. In studies where insufficient statistical information is include, study authors will be contacted to ask for further information. If no further information is provided, then the study will be excluded from the meta-analysis.


As it is anticipated that interventions within included studies will differ, random effects model will be used for the meta-analysis. Heterogeneity between studies will be assessed using the chi-squared statistic and I2, to examine what percentage of variability between studies is due to heterogeneity rather than chance. Where I2 is over 40% moderate heterogeneity between studies is indicated [17].

Subgroup analysis

If sufficient studies are included, and there is sufficient variation between the types of digital intervention used to try to increase flu vaccination amongst pregnant women, subgroup analysis will be performed to determine which type of digital intervention (for example, comparing text messages, mobile apps, websites) has the greater effect on increasing flu vaccination amongst pregnant women.

If there are sufficient studies, and sufficient variation between studies, additional sub-group analysis will be performed to determine the rate of self-reported flu vaccination verses validated flu vaccination (from medical records). This will be compared with the combined rate of flu vaccination, to determine if there is any difference between the rate depending on the way it is measured.

If there is sufficient variation in the risk of bias ratings of included studies, sensitivity analysis will be performed to determine whether results of the meta-analysis would change if high-risk studies were removed from the analysis.

Publication bias

Funnel plots will be examined to identify the presence of publication bias. If missing studies as a result of publication bias are detected, a Trim and Fill analysis will be completed to account for the missing studies in the effect size [18].


Digital healthcare interventions are increasing in popularity and use, contributed by the increase in accessibility of technology and internet use. Examining the effectiveness of such interventions for flu vaccination amongst pregnant women provides useful insight into the research area.


The inclusion criteria set may restrict the number of studies that are eligible for the review. The review is examining only one type of vaccination (flu vaccination) in one population group (pregnant women), which will limit the number of papers, but will provide vital information about the potential role that digital interventions can play in increasing the uptake of flu vaccination for this vulnerable patient group.

Flu vaccination uptake is lower in pregnant women from low socio-demographic groups [19,20,21]. There is the possibility that a digital intervention is beyond the reach of some pregnant women within this lower socio-demographic group which may further increase the divide between pregnant women that do and do not have the flu vaccination.


To date, no review has examined the effectiveness of digital interventions in increasing flu vaccination uptake amongst pregnant women. A recent review has examined interventions that have been used to increase flu vaccination uptake, but current advancements in technology and internet use, and previous demonstrated effectiveness of digital health interventions [22, 10,11,12], it is relevant and timely to examine digital flu vaccination interventions.

Potential implications

This review has the potential to increase current knowledge about whether digital interventions are likely to be an effective means of increasing flu vaccination amongst pregnant women and will hopefully highlight which modes of digital interventions are most effective. This knowledge will inform the design of future interventions, by allowing the most effective mode to be identified and used for maximum behaviour change.

Registration of review

The protocol for this review was registered on the international prospective register of systematic reviews (PROSPERO). Registration number pending. Any amendments to the protocol will be amended on this registration.

Availability of data and materials

Not applicable


  1. 1.

    Campbell H, Van Hoek AJ, Bedford H, Craig L, Yeowell A, Green D, Yarwood J, Ramsay M, Amirthalingam G. Attitudes to Immunisation in Pregnancy among Women in the UK Targeted by such Programmes. Br J Midwifery. 2015;23:8.

    Article  Google Scholar 

  2. 2.

    Knight M, Kenyon S, Brocklehurst P, Neilson J, Shakespeare J, Kurinczuk JJ (Eds.) on behalf of MBRRACE-UK. Saving Lives, Improving Mothers’ Care - Lessons learned to inform future maternity care from the UK and Ireland Confidential Enquiries into Maternal Deaths and Morbidity 2009–12. Oxford: National Perinatal Epidemiology Unit, University of Oxford 2014.

  3. 3.

    Tamma PD, Ault KA, del Rio C, Steinhoff MC, Halsey NA, Omer SB. Safety of Influenza Vaccination during Pregnancy. Am J Obstet Gynecol. 2009;201(6):547–52.

    CAS  Article  Google Scholar 

  4. 4.

    Legge A, Dodds L, MacDonald NE, Scott J, McNeil S. Rates and Determinants of Seasonal Influenza Vaccination in Pregnancy and Association with Neonatal Outcomes. CMAJ : Canadian Medical Association Journal = Journal De l'Association Medicale Canadienne. 2014;186(4):E157–64.

    Article  Google Scholar 

  5. 5.

    Madhi SA, Cutland CL, Kuwanda L, Weinberg A, Hugo A, Jones S, Adrian PV, Van Niekerk N, Treurnicht F, Ortiz JR. Influenza Vaccination of Pregnant Women and Protection of their Infants. N Engl J Med. 2014;371(10):918–31.

    Article  Google Scholar 

  6. 6.

    Regan AK, Tracey L, Blyth CC, Mak DB, Richmond PC, Shellam G, Talbot C, Effler PV. A prospective cohort study comparing the reactogenicity of trivalent influenza vaccine in pregnant and non-pregnant women. BMC Pregnancy and Childbirth. 2015;15(1):1.

    CAS  Article  Google Scholar 

  7. 7.

    Public Health England (2019). Seasonal flu vaccine uptake in GP patients: provisional monthly data for 1 September 2018 to 28 February 2019 by local team. Rerieved from

  8. 8.

    Office of National Statistics (2019). Internet access- households and individuals, Great Britain: 2019. Available from

  9. 9.

    Statistica 2018. Smartphone adoption in the United Kingdom (UK) from 2011 to 2017 . Retrieved May 2018 from

  10. 10.

    Griffiths, S.E., Parsons, J., Naughton, F., Tombor, I., Fulton, E.A., & Brown, K. E (2018). Are digital interventions for smoking cessation in pregnancy effective? A Systematic Review and Meta-analysis. Health Psychology Review.

  11. 11.

    Roberts A, Fisher A, Smith L, Heinrich M, Potts H. Digital health behaviour change interventions targeting physical activity and diet in cancer survivors: a systematic review and meta-analysis. Journal of Cancer Survivorship; Research and Practice. 2017;11(6):704–19.

    Article  Google Scholar 

  12. 12.

    Morrison D, Wyke S, Agur K, Cameron EJ, Docking RI, Mackenzie AM, McConnachie A, Raghuvir V, Thomson NC, Mair FS. Digital asthma self-management interventions: a systematic review. J Med Internet Res. 2014;16(2):e51.

    Article  Google Scholar 

  13. 13.

    National Institute for Health and Care Excellence (2020). Behaviour change: digital and mobile health interventions. Retrieved from

  14. 14.

    O'brien O, McCarthy M, Gibney E, McAuliffe F. Technology-supported dietary and lifestyle interventions in healthy pregnant women: a systematic review. Eur J Clin Nutr. 2014;68(7):760.

    CAS  Article  Google Scholar 

  15. 15.

    Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng H-Y, Corbett MS, Eldridge SM, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JPT. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.

    Article  Google Scholar 

  16. 16.

    Sterne JAC, Higgins JPT, Elbers RG, Reeves BC and the development group for ROBINS- I. Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I): detailed guidance, updated 12 October 2016. Available from [accessed April 2020].

  17. 17.

    Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from

  18. 18.

    Duval S, Tweedie R. Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–63 Retrieved from

    CAS  Article  Google Scholar 

  19. 19.

    Centre for Disease Control and Prevention (CDC). Influenza Vaccination among Pregnant Women--Massachusetts, 2009-2010. MMWR Morb Mortal Wkly Rep. 2013;62(43):854–7.

    Google Scholar 

  20. 20.

    Frew PM, Zhang S, Saint-Victor DS, Schade AC, Benedict S, Banan M, Ren X, Omer SB. Influenza Vaccination Acceptance among Diverse Pregnant Women and its Impact on Infant Immunization. Human Vaccines & Immunotherapeutics. 2013;9(12):2591–602.

    Article  Google Scholar 

  21. 21.

    Laenen J, Roelants M, Devlieger R, Vandermeulen C. Influenza and Pertussis Vaccination Coverage in Pregnant Women. Vaccine. 2015;33(18):2125–31.

    Article  Google Scholar 

  22. 22.

    Miller DP Jr, Denizard-Thompson N, Weaver KE, Case LD, Troyer JL, Spangler JG, Lawler D, Pignone MP. Effect of a digital health intervention on receipt of colorectal cancer screening in vulnerable patients. Ann Intern Med. 2018;168(8):550.

    Article  Google Scholar 

Download references


The authors would like to thank Samantha Johnson (Warwick Medical School library service) for help refining the search terms for the review.


There is no specific funding for completing this review; however, Open Access costs have been funded by the Open Access Fund at the University of Warwick. Researchers working on this review are funded by the Unit of Academic Primary Care with Warwick Medical School.

Author information




JP developed the plan for the systematic review and wrote the initial draft of the protocol.

HA commented on and revised the draft of the protocol, and approved the final version.

JP is the guarantor for the review. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Joanne Parsons.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Parsons, J., Atherton, H. How effective are digital interventions in increasing flu vaccination amongst pregnant women? A systematic review protocol. Syst Rev 9, 117 (2020).

Download citation


  • Systematic Review
  • Flu vaccination
  • Pregnancy
  • Digital interventions