A logic model will be developed to help with scoping the review, defining and conducting the review and making the review relevant to policy and practice , using templates developed as part of the EU-funded INTEGRATE-HTA project (Anke Rohwer, personal communication).
We will also attempt to capture the complexity of the different interventions included in the review by assessing the following domains proposed as part of a new tool developed within the Methodological Investigation of Cochrane Reviews of Complex Interventions (MICCI) project (Simon Lewin, personal communication): (1) number of discrete, active components included in the intervention compared with the control (or usual care); (2) number of behaviors or actions of intervention recipients or participants to which the intervention is directed; (3) number of organizational levels targeted by the intervention; (4) degree of flexibility or tailoring permitted across sites or individual intervention implementation/application; (5) the level of skill required by those delivering the intervention; and (6) the level of skill required for the targeted behavior when entering the study by those receiving the intervention (consumers, professionals, planners) in order to meet the intervention’s objectives
Types of studies to be included
RCTs, cluster randomized controlled trials (cRCTs), NRCTs, interrupted time series studies (ITSs) and controlled before-and-after studies (CBAs) will all be included.
Uncontrolled before-and-after studies, cohort studies, case-control studies and cross-sectional studies will qualify for inclusion.
Types of studies to be excluded
Supplementary immunization activities such as mass campaigns and school-based immunization services will be excluded.
Types of participants
Participants will include: children under 10 years of age receiving WHO recommended vaccines through ‘routine immunization services’, pregnant women receiving tetanus toxoid (TT) vaccination according to the national immunization schedule, caregivers of children or pregnant women who are receiving the vaccines, healthcare workers administering the vaccines and health facilities or health programs providing immunization services.
Types of interventions
Interventions for improving routine immunization coverage will be categorized in the following four groups, as implemented in a previous systematic review : (1) patient-oriented or community-oriented interventions, (2) provider-oriented interventions, (3) health-system interventions, and (4) multifaceted interventions (unique combinations of any of the above).
The primary outcome will be the proportion of children who have been fully immunized by the recommended age according to the national immunization schedule, or an appropriate proxy measure such as DTP3 coverage.
Secondary outcomes will be identified from included studies. These may include, but are not limited to: (1) other measures of immunization program performance as reported by the authors, for example, coverage with a specific vaccine, vaccine dropout rate, and adverse events following immunization (AEFI); (2) occurrence of vaccine preventable diseases (VPDs); (3) attitude and care-seeking behavior of caregivers towards immunization; (4) attitude and skills of healthcare workers; (5) characteristics of health facilities or health programs providing immunization services; (6) implementation of intervention; and (7) cost of intervention.
Search strategy and sources
The following electronic databases will be searched for peer-reviewed literature: Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE published by 31 August 2013.
A comprehensive search strategy will be developed that includes terms for immunization, coverage and immunization programs; as well as terms for children. The search strategy will be adapted to suit each individual database using applicable controlled vocabulary (see Additional file 1 for the proposed MEDLINE search strategy).
Data collection and analysis
Two authors (SM and CSW) will screen titles and abstracts of studies for potential eligibility. Following this, full texts of potentially eligible studies will be retrieved. Two authors (SM and CSW) will independently apply inclusion criteria to identify relevant studies to be included in the review. Any disagreements between the two authors regarding study eligibility will be resolved by discussion and consensus, failing which a third author (ER, RvK or GDH) will arbitrate. We will provide a table with characteristics of included studies, and another of excluded studies with reasons for their exclusion, in the final review.
A data extraction form will be developed by consultation and consensus among all authors. Two authors (SM and CSW) will independently extract data and assess risk of bias in included studies, compare their results, and resolve any discrepancies by discussion and consensus. We plan to analyze the data using Review Manager (RevMan).
Assessment of risk of bias
The quality of studies will be assessed using the modified GATE tool for experimental studies and the modified GATE tool for observational studies. Details of both of these tools are provided in the updated NICE Public Health Guidance manual for 2013 .
Two authors (SM and CSW) will apply the inclusion criteria, and any disagreements will be resolved by discussion and consensus; failing which we will consult a third author (ER, RvK or GDH).
Measures of treatment effect and data synthesis
We will express the result of each study as a risk ratio with its corresponding 95% confidence intervals for dichotomous data, or mean difference with its standard deviation for continuous data. We will conduct meta-analysis for the same type of participants, interventions, study designs, and outcome measures where homogeneity of data allows. We will use the random-effects model as the default procedure for meta-analyses due to anticipated heterogeneity, even if the latter is not statistically significant. If meta-analysis is not feasible due to significant statistical heterogeneity, we will explore the use of harvest plots. Harvest plots are a novel method for synthesizing evidence about the differential effects of heterogeneous and complex interventions, allowing review authors to maximize the learning potential derived from the studies included, to tailor the characteristics of studies that are most relevant within a particular body of evidence, and to aid in visualizing the results .
Assessment of heterogeneity
We will assess statistical heterogeneity using the χ2 test of homogeneity and quantify it using the I2 test statistic. We will describe heterogeneity as high if the I2 test statistic is greater than 50%, and will consider it statistically significant if the P value for the χ2 test is ≤0.1. If studies are found to be homogeneous (that is P value for the χ2 test is >0.1), results will be pooled by random effects meta-analysis, stratified by study design. Subgroup analyses will be conducted where possible, with subgroups defined by continent (Africa, Asia, Europe, Latin America and the Caribbean), setting (for example, urban vs rural), and vaccine delivery strategies. We will also tabulate heterogeneity in context and implementation between studies, using a new tool developed as part of the EU-funded INTEGRATE-HTA project (Lisa Pfadenhauer, personal communication).
Assessment of reporting bias
A funnel plot will be used to investigate the risk of publication bias by intervention type. The funnel plot will be visually examined for asymmetry. Provided 10 or more studies are included in the analysis for each intervention type, we will use the Beggs-Egger test to assess for funnel plot asymmetry.
If there is sufficient data, we will conduct sensitivity analyses to assess the effects of missing data, study design, and risk of bias on our primary meta-analyses. When we find a study with missing data, we will first perform available cases analysis; followed by sensitivity analyses according to imputations (that is, from assuming that all missing data are failures to assuming that all missing data are successes). We will also conduct sensitivity analyses to investigate the robustness of the results to study design (intervention vs correlation), method of meta-analysis (that is, random effects vs fixed effect), and risk of bias (that is, excluding studies with high risk of bias).
This protocol has not been registered with PROSPERO.