Influenza vaccines licensed in the United States in healthy children: a systematic review and network meta-analysis (Protocol)
Systematic Reviews volume 1, Article number: 65 (2012)
Influenza is an acute respiratory illness caused by influenza viruses, which occurs in epidemics worldwide every year. Children are an important target for prevention methods, including vaccination. While evidence about the decision on whether to vaccinate healthy children is robust, evidence supporting the decision of which of available vaccines to use remains unclear.
This review will summarize the evidence about the efficacy and safety of the available vaccines for seasonal influenza licensed in the United States for use in healthy children.
An umbrella systematic review (SR) and network meta-analysis will be conducted of randomized controlled trials (RCTs). We will search for SRs to identify parallel RCTs evaluating inactive and/or live attenuated influenza vaccines licensed in the United States for use in healthy children to prevent influenza. Subsequently, we will update the literature search of the selected SRs to the present time to capture recent controlled studies. To complement the work focused on harms, we will also select observational studies focusing on post marketing retrospective studies. Inclusion will not be limited by language, publication date or publication status. To identify additional candidate studies, we will review the reference lists of the eligible primary studies and narrative reviews; we will query the expert members of the Advisory Committee on Immunization Practices and review references from their previous statement. Additionally, we will review the reports from the Institute of Medicine on the adverse effects of vaccines. Two reviewers will independently determine study eligibility and will extract descriptive, methodological (using the Cochrane risk of bias tool for RCTs and the Newcastle–Ottawa scale for observational studies) and efficacy data. When possible, we will conduct meta-analyses and network meta-analyses by combining indirect and direct comparisons.
We will evaluate heterogeneity using the I2 statistic and the agreement of indirect comparisons and direct evidence. We will report the Cochrane Q test to determine the statistical significance of heterogeneity.
The overall quality of evidence will be assessed following the GRADE (Grading of Recommendation, Assessment, Development and Evaluation) approach.
Our systematic review will allow patients, clinicians, guideline developers and policy makers to make evidence-based choices between the two available vaccine options, by providing information regarding benefits and harms of these types of vaccines.
Influenza is an acute respiratory illness caused by influenza A or B viruses, which occurs in seasonal outbreaks worldwide each year, causing substantial morbidity and mortality—between 3,000 and 49,000 deaths  and over 200,000 hospitalizations  annually. While the majority of influenza-related deaths occur among adults aged 65 years and older , children are also at risk of severe disease, particularly those under 5 years of age .
Children are the major source for the diffusion of influenza viruses in the community [4, 5]. Some studies showed socioeconomic benefits from influenza vaccination in healthy children, both for them and their adult household contacts . For these reasons, much emphasis has been placed on preventing influenza in children. The adoption of universal annual influenza vaccination of children could reduce the burden of influenza illness among vaccinated children, unvaccinated household and school contacts, and reduce community transmission, thereby reducing severe illness among high risk adults .
Currently available seasonal influenza vaccines may be divided into two groups:
Inactivated influenza vaccines (IIV): These include subunit vaccines (containing only surface antigens); split-virion vaccines (in which virus is inactivated with disrupting agents, yielding surface and internal antigens); and whole virion inactivated vaccines (containing whole inactivated viruses). Among these, only split-virion vaccines are currently licensed in the United States. Most are administered intramuscularly; one intradermal preparation is available. IIV is available from a number of different manufacturers; age indications for the different preparations differ. IIV preparations are available for children as young as six months of age.
Live-attenuated (cold-adapted) influenza vaccines (LAIV): These intranasally-administered vaccines contain virus which can multiply only in the nasal passages. LAIV is licensed for persons 2 through 49 years old and is recommended for healthy, non-pregnant individuals who do not have chronic medical conditions, due to relative lack of data in the other populations .
The Centers for Disease Control and Prevention (CDC) Advisory Committee on Immunization Practices (ACIP)  and the American Academy of Pediatrics (AAP)  recommend annual influenza vaccination for all children aged ≥6 months. While the evidence supporting the benefits of vaccinating healthy children to prevent influenza is robust, the evidence to support the decision on which vaccine form to use with whom remains unclear; ACIP currently expresses no preference for one vaccine over the other .
This systematic review (SR) aims to summarize the evidence concerning the efficacy and safety of US-licensed seasonal influenza vaccines for healthy children 6-months through 18-years old. Evidence for the comparative benefits of LAIV and IIV among children 2- through 18-years old will also be examined.
This evidence will help guideline developers, clinicians and patients in choosing the most suitable vaccine for each age group based on tradeoffs between harms and benefits.
Considering the availability of multiple well-conducted SRs addressing this question, we will start by conducting an umbrella systematic review [10, 11] to identify existing SRs and select the most recent and comprehensive ones. We will evaluate each systematic review using the Assessment of Multiple Systematic Reviews (AMSTAR) criteria  particularly emphasizing the quality and comprehensiveness of the search strategy. We will also consider their inclusion and exclusion criteria. Subsequently, we will update and, if needed, modify the literature search of the selected SRs to the present time to capture recent controlled studies (randomized controlled trials (RCTs) and non-randomized studies). We will search electronic databases (Ovid Medline, OVID EMBASE, OVID Cochrane Library, Web of Science, Scopus and PsycInfo) from the last search date stated in the included SRs through the present time. With input from study investigators with expertise in conducting SRs (MHM, VMM), a reference librarian (LP) and the first author (GP) will design and execute these electronic search strategies using controlled vocabulary and text words. To identify additional candidate studies, we will review the reference lists of the eligible primary studies, narrative reviews and systematic reviews; and we will query the expert members of the ACIP Influenza Work Group and review the references from the previous ACIP Influenza Statement. Additionally, we will review the Institute of Medicine (IOM) report on vaccine adverse effects [13–15] to capture other possible harm evidence. The two search strategies (for SRs and individual studies) are given in the Additional file 1.
We will include parallel-design RCTs that compared LAIV and IIV to each other or to placebo. Eligible trials must evaluate the efficacy of seasonal influenza vaccines (not pandemic vaccines) licensed in the United States and administered as recommended by the ACIP, and must enroll healthy children (<18 years). We will exclude vaccinations not administered according to ACIP recommendations (for example, LAIV given to children under 2 years of age). Evidence concerning adjuvanted vaccines, subunit vaccines, and whole-virus vaccines will be excluded.
For evidence of harm, we will select observational studies focusing on post marketing retrospective studies. These studies typically have a larger sample size which is essential to detect rare adverse events caused by the vaccine, such as Guillain-Barre syndrome or anaphylaxis, with incidence less than 10 cases per million .
Search output will be uploaded into an online reference management system (DistillerSR, Ottawa, Canada) to allow fast and transparent processing with better tracking and real-time evaluation of inter-reviewer agreement and progress of reviewers. Two reviewers working independently will consider the potential eligibility of each of the abstracts and titles that will result from executing the search strategy. Reviewers will request the full text versions of all potentially eligible studies. The full text of papers on which there is disagreement will also be retrieved for evaluation. Two reviewers working independently will consider the full text reports (all available versions of each study) for eligibility. The reviewers will calibrate their judgments using a smaller set of reports. Subsequently, disagreements will be resolved by consensus; if not possible, by arbitration by a third reviewer. Agreement will be measured using the kappa or phi statistic, as appropriate (the latter is appropriate when the distribution of the feature under evaluation is extremely common or rare). This process is represented in Figure 1.
Data collection and extraction
We will collect data in predefined electronic forms designed using the online reference management system. Two reviewers separately and independently will extract full descriptions of participants enrolled, the interventions they received (vaccine description, doses, route), and the measure of outcome.
Outcomes of interest
Effectiveness outcomes measures: a. Laboratory-confirmed influenza infection, confirmed by polymerase chain reaction or viral culture; b. Influenza-like illness (ILI) ; c. Medically attended respiratory illness (MARI); d. Hospitalization; e. Mortality; f. Medically attended wheezing.
Safety outcome measures: a. Fever due to vaccination; b. Febrile seizures; c. Immediate hypersensitivity/anaphylaxis; d. Guillain-Barré syndrome; e. Serious adverse effects.
To assess the methodological quality of RCTs we will use the Cochrane risk-of-bias tool assessment  to determine: how the randomization sequence was generated; how allocation was concealed; whether there were important imbalances at baseline; which groups were blinded (patients, care givers, data collectors, outcome assessors, data analysts); loss to follow-up; whether participants were analyzed as randomized; and how missing outcome data were addressed. We will also evaluate the adequacy of the outcome measurement process. For observational studies the quality will be assessed using the Newcastle–Ottawa scale. No scoring system will be derived.
Relative comparisons will be quantified by means of the relative risk (RR) of outcomes with one vaccine form versus the other. We choose to report relative risk instead of odds ratio since RR is more intuitive to clinicians and patients [1–18]. To pool head-to-head comparisons, we will conduct random effects models using the DerSimonian & Laird method and report point estimates and 95% confidence interval . We will evaluate heterogeneity using the I2 statistic, a measure of the total variations of study effect sizes due to true heterogeneity between studies. Considering that thresholds for the interpretation of I2 can be misleading we will analyze its importance based on magnitude and direction of effects and strength of evidence for heterogeneity (for example, P value from the Cochran’s Q test and confidence intervals for I2) . We will report the Cochrane Q test to determine the statistical significance of heterogeneity.
When possible, we will conduct network meta-analyses by combining indirect and direct evidence. The network of comparisons is presented graphically in Figure 2. Due to the small number of comparisons in this study, we are quite confident that the network will have at least one pair of treatments being compared both directly and indirectly. Thus, Lumley’s generalized linear mixed models will be used . The agreement of indirect comparisons and direct evidence, that is, the degree of coherence of the network, will be measured and incorporated in the calculation of the confidence interval of the pooled estimate. If the incoherence is large and network meta-analysis is deemed inappropriate, only estimates from direct evidence will be pooled and reported. In rare cases in which the network of comparisons does not have at least one pair of treatments being compared both directly and indirectly, we will use adjusted indirect comparison (AIC) models instead of generalized linear mixed models .
For studies with loss to follow-up, we will apply the intention-to-treat principle and analyze the initial number of patients in the group to which they were randomized. The decision is to preserve randomization benefits in balancing prognosis of trial arms even if it leads to underestimated effect sizes .
Statistical analyses will be conducted using STATA version 12.0 (StataCorp, College Station, TX, USA) and R version 2.15.0 (R Foundation for Statistical Computing, Vienna, Austria).
If sufficient data were available to explore causes of inconsistency and subgroup-treatment interactions, we will construct the following subgroup analyses defined by: a. Method used for the diagnosis of laboratory confirmed influenza (PCR versus viral culture versus other); and b. Age groups (6 to 23 months, 2 to 8 years, 9 to 18 years).
Publication bias will be assessed using the Peters linear regression of intervention effects on inverse of study sample size and visual inspection of funnel plots .
Data extractors will record the author’s contact information (name and email address) from RCTs and observational studies that fulfilled all the eligibility criteria. We will contact the authors to obtain sufficient details for quality assessment or missing data. If we do not receive the information from the authors we will try to obtain additional information about the methodological characteristics of each study by getting access to each study’s protocol. For industry-sponsored trials we will seek additional information from the current license holder. For studies sponsored by the National Institute of Health we will ask them for the protocol.
This study will be reported in accordance with the recommendations set forth by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) workgroup  and the recommendations developed by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) . We will present evidence tables for each trial including description of the population characteristics, interventions, methodological quality and main findings.
Evaluating the quality of evidence
The quality of evidence will be assessed following the GRADE approach. Evidence profiles will be developed using the software GRADE profiler (GRADEpro) as shown in Table 1. This evaluation will include factors of methodological limitations of the studies, such as: overall quality assessment of the included studies, imprecision, indirectness, inconsistency, reporting and publication biases. This will show our confidence in the pooled estimate according to the quality and will be classified as follow: high, moderate, low or very low . Evidence from observational studies could be upgraded if studies showed a large effect size, dose response effect or if the plausible residual confounding is considered to strengthen the association . Unique to judging the quality of evidence in vaccination studies, we will consider additional factors suggested by the Strategic Group of Advisory Experts (SAGE) committee of the World Health Organization which are upgrading the evidence for a population effect or evidence of herd immunity .
This network meta-analysis aims at synthesizing the available direct and indirect evidence on the effectiveness and safety of the different vaccines for seasonal influenza in healthy children currently available in the United States. This meta-analysis will allow patients, physician, guideline developers and policy makers to make evidence based choices between the two available vaccine options.
Limitations and strengths of this study
We will depend on existing SRs to identify individual studies published prior to the date of our search update. Therefore, the comprehensiveness of this SR will depend in part on that of the published SRs. The existing reviews we identified through a preliminary search of the literature seem quite comprehensive (adequate according to the AMSTAR assessment ) and include some published by the Cochrane Collaboration . However, we are also supplementing the search strategy by querying experts and reviewing previous ACIP statements and IOM reports to reduce the risk of missing relevant studies.
It is plausible that the variation of influenza strain and virulence from year to year or the matching between the vaccine and the circulating strain will vary across studies. Therefore, the network method of pooling direct and indirect estimates may be associated with a higher level of inconsistency/heterogeneity. In this case, we will abandon the network meta-analysis approach and only use head-to-head RCTs pooled in a traditional random effects model meta-analysis.
These criteria were adopted to provide direct evidence for the comparative effectiveness of contemporary vaccines administered in the United States. The evidence will be less directly applicable to other geographic locations or settings in which the vaccines are administered in a way inconsistent with the ACIP statement.
Systematic review status
The systematic review is currently in the data extraction and preliminary analysis phase. We expect completion by October 2012.
This study was supported by the Center for Disease Control and Prevention (CDC).
Advisory Committee on Immunization Practices
Assessment of Multiple Systematic Reviews
Grading of Recommendations Assessment, Development and Evaluation
inactivated influenza vaccine
live attenuated influenza vaccine
randomized controlled trials
Centers for Disease Control and Prevention (CDC): Estimates of deaths associated with seasonal influenza, United States, 1976–2007. MMWR Morb Mortal Wkly Rep. 2010, 59: 1057-1062.
Thompson WW, Shay DK, Weintraub E, Brammer L, Bridges CB, Cox NJ, Fukuda K: Influenza-associated hospitalizations in the United States. JAMA. 2004, 292: 1333-1340. 10.1001/jama.292.11.1333.
Poehling KA, Edwards KM, Weinberg GA, Szilagyi P, Staat MA, Iwane MK, Bridges CB, Grijalva CG, Zhu Y, Bernstein DI, Herrera G, Erdman D, Hall CB, Seither R, Griffin MR, New Vaccine Surveillance Network: The underrecognized burden of influenza in young children. N Engl J Med. 2006, 355: 31-40. 10.1056/NEJMoa054869.
Munoz FM: Influenza virus infection in infancy and early childhood. Paediatr Respir Rev. 2003, 4: 99-104. 10.1016/S1526-0542(03)00027-7.
Fiore AE, Uyeki TM, Broder K, Finelli L, Euler GL, Singleton JA, Iskander JK, Wortley PM, Shay DK, Bresee JS, Cox NJ: Prevention and control of influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2010. MMWR Recomm Rep. 2010, 59: 1-62.
Principi N, Esposito S, Marchisio P, Gasparini R, Crovari P: Socioeconomic impact of influenza on healthy children and their families. Pediatr Infect Dis J. 2003, 22: S207-S210.
Fiore AE, Epperson S, Perrotta D, Bernstein H, Neuzil K: Expanding the recommendations for annual influenza vaccination to school-age children in the United States. Pediatrics. 2012, 129 (Suppl 2): S54-S62.
Centers for Disease Control and Prevention (CDC): Prevention and control of influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2011. MMWR Morb Mortal Wkly Rep. 2011, 60: 1128-1132.
American Academy of Pediatrics Committee on Infectious Diseases: Recommendations for prevention and control of influenza in children, 2011–2012. Pediatrics. 2011, 128: 813-825.
Smith V, Devane D, Begley CM, Clarke M: Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011, 11: 15-10.1186/1471-2288-11-15.
Thomson DRK, Becker L, Klassen T, Hartling L: The evolution of a new publication type: Steps and challenges of producing overviews of reviews. Research Synthesis Methods. 2011, 1: 198-211.
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM: Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007, 7: 10-10.1186/1471-2288-7-10.
Stratton KAD, Wizemann T, McCormick MC: Immunization safety review: influenza vaccine and neurologic complications. Immunization Safety Review: Influenza Vaccine and Neurologic Complications. 2004, National Academy of Sciences
Program CotRotNIPsRPaDS: Vaccine safety research, data access, and public trust. Vaccine Safety Research, Data Access, and Public Trust. 2005, National Academies Press
Stratton KFA, Rusch E, Wright Clayton E: Adverse effects of vaccines: evidence and causality. Adverse Effects of Vaccines: Evidence and Causality. 2011, The National Academies Press
Greene SK, Kulldorff M, Lewis EM, Li R, Yin R, Weintraub ES, Fireman BH, Lieu TA, Nordin JD, Glanz JM, Baxter R, Jacobsen SJ, Broder KR, Lee GM: Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink Project. Am J Epidemiol. 2010, 171: 177-188. 10.1093/aje/kwp345.
Higgins J, Green S: Cochrane Handbook for Systematic Reviews of terventions Version 5.1.0 [updated March 2011]. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011, Oxford: The Cochrane Collaboration
Jaeschke R: Understanding the results: more about odds ratios. User’s Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Edited by: Guyatt GH. 2008, The McGraw-Hill Companies, Inc, 221-230.
Whitehead A: Combining Estimates of a Treatment Difference Across Trials. Meta-analysis of controlled clinical trials, Volume 23. 2002, Wiley, 57-98.
Higgins JP, Thompson SG, Deeks JJ, Altman DG: Measuring inconsistency in meta-analyses. BMJ. 2003, 327: 557-560. 10.1136/bmj.327.7414.557.
Lumley T: Network meta-analysis for indirect treatment comparisons. Stat Med. 2002, 21: 2313-2324. 10.1002/sim.1201.
Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D’Amico R, Bradburn M, Eastwood AJ: Indirect comparisons of competing interventions. Health Technol Assess. 2005, 9: 1-134. iii-iv
Montori VM, Guyatt GH: Intention-to-treat principle. CMAJ. 2001, 165: 1339-1341.
Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L: Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006, 295: 676-680. 10.1001/jama.295.6.676.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009, 62: e1-e34. 10.1016/j.jclinepi.2009.06.006.
Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, Boersma C, Thompson D, Larholt KM, Diaz M, Barrett A: Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health. 2011, 14: 429-437. 10.1016/j.jval.2011.01.011.
Canfield SE, Dahm P: Rating the quality of evidence and the strength of recommendations using GRADE. World J Urol. 2011, 29: 311-317. 10.1007/s00345-011-0667-2.
Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, Atkins D, Kunz R, Brozek J, Montori V, Jaeschke R, Rind D, Dahm P, Meerpohl J, Vist G, Berliner E, Norris S, Falck-Ytter Y, Murad MH, Schünemann HJ, GRADE Working Group: GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol. 2011, 64: 1311-1316. 10.1016/j.jclinepi.2011.06.004.
Duclos P, Durrheim DN, Reingold AL, Bhutta ZA, Vannice K, Rees H: Developing evidence-based immunization recommendations and GRADE. Vaccine. 2012, 31: 12-19. 10.1016/j.vaccine.2012.02.041.
Jefferson T, Rivetti A, Harnden A, Di Pietrantonj C, Demicheli V: Vaccines for preventing influenza in healthy children. Cochrane Database Syst Rev. 2008, CD004879-2
The authors declare they have no competing interests.
All listed authors contributed substantially to the design of this protocol. All authors read and approved the final manuscript.
Electronic supplementary material
About this article
Cite this article
Prutsky, G.J., Domecq, J.P., Elraiyah, T. et al. Influenza vaccines licensed in the United States in healthy children: a systematic review and network meta-analysis (Protocol). Syst Rev 1, 65 (2012). https://doi.org/10.1186/2046-4053-1-65
- Influenza vaccine
- Systematic review