The association between adverse pregnancy outcomes and maternal human papillomavirus infection: a systematic review protocol
© The Author(s). 2017
Received: 5 November 2016
Accepted: 24 February 2017
Published: 11 March 2017
Human papillomavirus (HPV) is the most prevalent genital infection, especially in young women of reproductive age. In vitro and animal model experiments provide compelling evidence of the harmful effect of HPV on pregnancy outcomes, but results from epidemiologic studies are inconclusive. We aim to determine the strength of the relationship between adverse pregnancy outcomes (APO) and HPV infection and assess its consistency across studies, by systematically reviewing the literature.
The search strategy has been developed on the basis of the PICOS framework: Population (pregnant women); Exposure (HVP infection confirmed by HPV testing); Comparator (pregnant women without HPV infection); Outcomes (miscarriage, spontaneous preterm birth, low birth weight, preterm premature rupture of membranes, pregnancy-induced hypertensive disorders and intrauterine growth restriction) and Study design (observational studies). We will search three information sources: (1) electronic databases (MEDLINE, EMBASE, and EBM Reviews databases); (2) Grey literature (Google Scholar and Web of Science conference proceedings); and (3) citing and cited articles of included studies. Two reviewers (JN, NZ) will independently and in duplicate screen identified articles, select eligible studies, and extract data. Discrepancies will be resolved by consensus and otherwise by discussion with the other authors (MHM, HT). Quality of included studies will be assessed using the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies. We will narratively synthesize extracted data whether meta-analysis is conducted or not. Meta-analysis of each outcome will be performed, and where appropriate, an average measure of association will be computed. We will use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess and grade the strength of confidence in cumulative estimate.
Comprehensive and high-quality evidence of a negative effect of HPV on pregnancy outcomes might be an additional motivation for HPV vaccination. Absence of such relationship could dispel anxiety and reassure HPV-infected pregnant women and clinicians. Findings of a poor level of confidence will allow identification of current knowledge gaps on HPV-pregnancy outcome relationship that need further research.
Systematic review registration
KeywordsPregnant women Human papillomavirus Adverse pregnancy outcomes Systematic review Protocol
Human papillomavirus (HPV) is the most common sexually transmitted infection in adults [1, 2]. The lifetime probability of genital HPV acquisition is estimated to be more than 80% in both women and men by the age of 45 years . The highest incidence rates occur in young adults, just after the onset of sexual activity . Longitudinal studies in young women 17–24 years of age-reported incidence rates ranging from 15.7 to 29.4 HPV infections of all types per 1000 women-months . Cohort studies of women in their thirties showed a lower incidence rate, between 5.2 and 13.4 HPV infections any type per 1000 women-months .
Similarly, the prevalence is higher in younger than in older age groups. A multi-country meta-analysis estimated the global HPV prevalence at 11.7% (confidence interval 95% 11.6%-11.7%) in women with normal cytology with an important variation within and between geographic regions . The prevalence peaks at around 25 years and decreases thereafter. In some populations, a smaller second peak is observed at 45 years [5, 6].
In men, the prevalence estimates of genital HPV infection vary widely across studies ranging from 1 to 84% . Unlike in women, the HPV prevalence curve in men peaks at older ages and remains steady high or decreases slightly with increasing age .
Fortunately, almost 80% of HPV infections resolve spontaneously within 1–2 years . However, accumulating evidence suggests that HPV infection is more likely to persist during pregnancy  and regress after delivery [10–13]. Indeed, the increase of steroid hormones during pregnancy could modify the maternal immune system and contribute to the “tolerance” of the fetus but decrease the ability to clear infections, including HPV .
Maternal infections, inflammation , and changes in vaginal bacterial microbiota  have been recognized as an underlying cause of major adverse pregnancy outcomes (APO) such as miscarriage , spontaneous preterm birth , and pregnancy-related hypertension disorders . In particular, a recent large multicenter study carried out across eight clinical sites in the USA reported that infections/inflammation are involved in 38% of cases of spontaneous premature delivery . This proportion could even be underestimated, as some of pathogens are under detected because of the lack of sensitivity of conventional detection techniques and difficulties in taking informative samples when the infection is intrauterine . In addition, some genital infections remain asymptomatic and, as such, are not diagnosed .
Especially, HPV infections possess those peculiarities in terms of long-term latency, episodic detectability, and genital as well as intrauterine localization. Indeed, genital HPV infection may have a long-term clinical latency, which either reflects the viral persistence (defined as two or more HPV-positive tests over a certain period ) or viral latency (HPV is not cleared but remains undetectable by conventional molecular tests and can eventually reactivate) . Clinical latency may encompass episodic states of viral clearance and recurrence, which can lead to misinterpretation of HPV infection status based only on one or few HPV tests . Furthermore, HPV has been localized in amniotic fluid , in placental trophoblastic cells [25, 26] and umbilical cord blood [26, 27]. Sometimes, intrauterine infection does not necessary coincide with cervical infection .
In vitro studies [28–30] and animal models [31–33] demonstrated biological plausibility of detrimental effect of HPV on pregnancy outcomes. These experimental studies have shown that HPV can replicate in trophoblasts  leading to (1) inhibition of blastocyst formation ; (2) failed or suboptimal endometrial implantation of trophoblastic cells [29, 31]; and (3) apoptosis of embryonic cells . Besides these direct effects on placenta cells, it has been hypothesized that HPV-trophoblast interaction may trigger immune hypersensitivity to bacteria leading to pregnancy complications, such as preeclampsia or preterm labor .
Along with these experimental evidences, observational studies have investigated the effect of HPV on APO, but their results are equivocal. Some authors have concluded to the association between HPV and APO [35–39], whereas others have not found any association [40–43] or have found that HPV is not an independent risk factor of APO but could potentiate the harmful effect of other genital infections on pregnancy [42, 44].
HPV infections can be a source of anxiety for pregnant women and their family. Indeed, health professionals have the task to answer questions and reassure HPV-positive women attending antenatal clinics on the risk of APO related to HPV infection. With this kind of inconsistent data, systematic reviews are recommended for identification, appraisal, and synthesis of evidence in order to inform policy makers and clinicians .
Two reviews [46, 47] both published in 2014 have focused on the question of HPV and pregnancy complications. Bonde et al. reviewed the association between HPV infection and infertility, APO, and the risk of vertical transmission of HPV . This review did not follow any systematic approach to document and synthesize the existing literature. After a narrative assessment of the literature, the authors concluded that more research was necessary to reach firm conclusions on the association between pregnancy outcomes and HPV . Conversely, Huang et al. conducted a meta-analysis and found a twofold increased risk of preterm birth in pregnant women infected by HPV (pooled OR 2.12, 95% CI 1.51–2.98) . Although several subgroup analyses were conducted to explore the substantial heterogeneity (I 2 of 61%), this review still contained several limitations such as important exposure measurement bias. Indeed, in three of eight included studies, HPV detection was prior to pregnancy [39, 48] or postnatal . Given the high clearance rate of HPV  and considering that pregnancy represents the risk period, measuring exposure out of the pregnancy period might lead to exposure measurement bias. However, the effect of this bias on the pooled estimate has not been assessed.
In addition to the time point of HPV detection, other factors, such as exposure measurement methods (cytology vs HPV test) and characteristics of participants (e.g., other genital infections) acting as confounding factors, could explain the substantial heterogeneity in this review and could have led to a biased estimate. Also, the assessment of confidence in cumulative evidence was limited to publication bias analysis. Finally, there could be residual confounding that may explain the large magnitude effect reported by this meta-analysis.
Clinical trials have shown HPV vaccines to be highly efficacious in preventing infections with the HPV types targeted by the vaccines . There is a growing literature on the promising benefits of HPV vaccination to prevent HPV-related APO [50, 51]. In fact, widespread implementation of HPV vaccination programs is expected to reduce HPV prevalence and diseases related to HPV [49, 52]. Especially, HPV vaccination is expected to reduce incidence of cervical cancer and pre cancers, which will reduce the need for cervical excision and therefore the number of preterm birth [46, 50, 53]. However, there is insufficient evidence on the potential harmful effect of HPV on pregnancy outcomes to support this potential benefit of HPV vaccine on pregnancy outcomes.
Given the rarity and limited quality of evidence of previous reviews, there is a need to conduct a systematic review using a clear and reproducible method taking into account limitations of previous reviews (exposure window, measurement of exposure, and uncontrolled confounding). We will follow recommended guidelines to conduct an extensive and systematic search, critically appraise and summarize the search results and thoroughly assess the quality of the overall evidence. We will also use sensitivity analyses to assess the impact of potential limitations. Thus, we will provide a better understanding of contradictory previous studies and recent literature on the effect of HPV infection on pregnancy outcomes. Ultimately, the current review could shed light on the level of confidence in overall evidence on the association between HPV infection and APO.
Our main objective is to assess whether HPV infection in pregnant women is associated with APO.
This protocol will follow the preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P 2015) . In interest of transparency and completeness, a completed PRISMA-P 2015 checklist is provided [see Additional file 1].
The search strategy has been developed on the basis of the PICOS framework: Population, Intervention (or Exposure), Comparator, Outcomes and Study design.
Types of studies
We will include all types of observational studies: cohort (retrospective and prospective), case-control, and cross-sectional studies. We will also consider data from placebo groups of HPV vaccination randomized controlled trials, which might have documented HPV-associated APO.
Population of interest
We will include studies investigating APO in women with and without HPV infection.
The exposure of interest is the presence compared to the absence of HPV infection in pregnant women. We will consider HPV infection in all genital sites (vulva, vagina, or cervix), in placenta, or other products of conception (POC). The presence or absence of HPV infection will need to have been proven by a sensitive molecular method. We will include studies reporting either on overall HPV infection or on HPV type-specific infection.
Findings from in vitro studies and animal models proved that HPV replication in the trophoblasts is the principal cause leading to various HPV-related APO [28–34]. Practically, it is difficult to distinguish the exact exposure time window, which reflects the period during which the exposure (HPV infection) is having its effects relevant to the outcomes of interest (APO) . We assume that the pregnancy period represents the exposure time window, which is the time period during which HPV infection exerts its deleterious effect on pregnancy outcome. Thus, we will include studies that reported HPV infection during pregnancy. However, the median duration of HPV infection has been estimated at approximately 12 months in a recent multi-country study of healthy women aged over 25 years . Consequently, HPV infection detected prior to pregnancy may persist during pregnancy and even after. Therefore, we will also consider studies that reported HPV detection within 12 months prior to pregnancy and after delivery. The effect of these exposure window assumptions and inclusion of studies that used cytology as exposure measurement will be assessed in sensitivity analyses.
The comparison groups will include pregnant women who are not infected by HPV, matched or not for known possible confounders (age, smoking, alcohol, or other genital infections). For the studies on HPV infection in POC, comparators of concern will be POC in women without HPV infection.
Outcomes and prioritization
Prioritization and definitions of the outcomes of interest, with sources of definitions in brackets, are described below:
Pregnancy loss (or spontaneous abortion or miscarriage), defined as a nonviable, intrauterine pregnancy with either an empty gestational sac, or a gestational sac containing an embryo or fetus without fetal heart activity within the first 12 weeks of gestation (early pregnancy loss) or within the first 20 weeks (late or second trimester pregnancy loss) (ACOG: American College of Obstetricians and Gynecologists) 
Preterm birth (PTB), defined as birth between 200/7 and 366/7 weeks of gestation (ACOG) . Studies without distinction between spontaneous and elective preterm birth will be included. Where feasible, this aspect will be assessed in subgroup or sensitivity analysis.
Secondary outcomes consist of APO that have been associated with maternal genital viral infections in general  and with HPV in particular  or have been linked to primary outcomes . Thus, we will include APO that may be associated with PTB (such as pregnancy-induced hypertensive disorders (PIHD) [59, 60]) or that may be on the causal pathway of PTB (such as preterm pre-labor rupture of membranes (PPROM) . In addition, secondary outcomes will include low birth weight (LBW) as a proxy of PTB  and intrauterine growth restriction (IUGR) as a constituent of LBW . We will include studies that reported on birth defects as almost 50% of early miscarriages are due to chromosomal abnormalities . Likewise, we will consider stillbirths, which are related to infection in general  and to most of previous APO such as PIHD, IUGR, and PPROM .
LBW, defined as a weight of less than 2500 g irrespective of the gestational age (WHO-ICD-10: World Health Organization-International statistical Classification of Diseases-tenth revision) .
PIHD, including preeclampsia/eclampsia and gestational hypertension. Preeclampsia is a syndrome defined by hypertension (blood pressure of 140 mmHg systolic or higher or 90 mmHg diastolic or higher that occurs after 20 weeks of gestation in a woman with previously normal blood pressure) and proteinuria (urinary excretion of 0.3 g protein or higher in a 24-h urine specimen). Eclampsia is defined as the presence of new-onset grand mal seizures in a woman with preeclampsia. The term gestational hypertension refers to a woman who develops an elevated blood pressure without proteinuria after 20 weeks of gestation and blood pressure levels return to normal postpartum (ACOG) .
PPROM, defined as membrane rupture before labor and before 37 weeks of gestation (ACOG) 
IUGR, defined as a fetus with an estimated fetal weight <10th percentile at birth (SOGC: Society of Obstetricians and Gynecologists of Canada) .
Birth defects. For practical reasons, we will focus on birth defects that may be noticed at birth and for which there are screening and diagnostic tests during pregnancy .
Stillbirth, defined as fetal death that occurs during pregnancy at 20 weeks of gestation or greater .
There will be no period or geographical restriction.
There will be no language restriction provided that there is an English or French abstract. Excluded English or French relevant titles without English or French abstract will be provided as an appendix.
Our systematic search will target three information sources. We will initially target MEDLINE, EMBASE, and EBM Reviews databases (Cochrane Database of Systematic Reviews, American College of Physicians- ACP- Journal Club, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Health Technology Assessment and National Health Service –NHS- Economic Evaluation Database). We will search MEDLINE (PubMed and Ovid interfaces) from 1946, EMBASE from 1981, and EBM (Ovid interface) from 1991 onwards.
To ensure a complete search coverage, we will hand-search reference lists of included titles and of any previous review related to our research question. Moreover, we will search for additional relevant titles by screening citations of included articles using Google Scholar or Web of Science.
Finally, in order to identify unpublished studies, we will search grey literature through Google Scholar and Web of Science. Especially, we will search relevant conference proceedings published on websites of international conferences on HPV, such as IPV (International Papillomavirus) and EUROGIN (European Research Organization on Genital Infection and Neoplasia) conferences.
We will combine three systematic search strategies to identify relevant literature. Initially, for each of electronic databases, a specific search strategy was developed. We have first developed a search strategy for MEDLINE based on Medical Subject Headings (MeSH) and free text words. Our research team developed the terms for MEDLINE on the basis of the eligibility criteria. The search strategy was developed by the principal author (JN) and a health sciences librarian with expertise in systematic literature searching. We combined terms (or exploded terms where possible) using Boolean operators. Terms developed in MEDLINE were adapted for correct use in the EMBASE and EBM Reviews databases. Preliminary search strategies piloted for PubMed, EMBASE, and EBM Reviews databases are provided [see Additional file 2].
The second search strategy will target grey literature to identify additional unpublished articles in journals indexed in electronic databases. To maximize identification of more relevant studies, we will combine three complementary grey literature online sources, namely Google Scholar, Web of Science conference proceedings citation index, and International conferences specific websites. Thus, we will search Google Scholar using the following free text words: “pregnancy outcomes” and “papillomavirus”. We will retain relevant unpublished titles, which will be screened according to the eligibility criteria. The same search terms used for MEDLINE will be adapted for Web of Science. We will apply filters within Web of Science search strategy to restrict results to conference proceedings. In addition, we will hand-search for relevant conference proceedings of not formally published studies through online abstract books of IPV and Eurogin conferences.
The third search strategy will target additional relevant titles by searching citations of included articles and of any previous review. We will screen reference lists of included studies and previous reviews to find relevant cited articles. We will use Google Scholar (or Web of Science) to find citing publications of included articles. Citing and cited articles will be screened using the same inclusion and exclusion criteria as for the two previous search strategies.
All titles identified from electronic databases (MEDLINE, EMBASE, and EBM Reviews databases) will be uploaded and combined into an EndNote file (EndNote X7.3). Titles retrieved from Grey literature will be manually entered into EndNote file or uploaded individually from Google Scholar. Using EndNote’s auto-deduplication function, we will first automatically find and remove duplicate publications. To avoid any residual duplicate, we will sort the remaining records by author’s names and later by titles. For the sake of clarity, all duplicates will be kept into duplicate EndNote subfolder. After duplicates are removed, all remaining records will be saved into an EndNote subfolder.
Two authors JN and NZ will independently and in duplicate screen the remaining records in two phases.
At the first phase (screening), a standardized checklist created on the basis of eligibility criteria will be used to screen titles and abstracts [see Additional file 3]. The screening phase will differentiate potentially eligible from non-eligible studies. Thus, study’s title or abstract that clearly does not pertain to any of PICOS criteria will be excluded. Studies that cannot be clearly excluded on the basis of title or abstract (uncertain and eligible) will be retained for further assessment.
At the second phase (eligibility), full text reports of all potentially eligible records will be retrieved and independently analyzed by the same two reviewers. A standardized form with explicit and detailed eligibility criteria on the basis of PICOS will be used to screen full text reports [see Additional file 4].
With the aim of reducing errors, a pilot test of the standardized tools will be conducted prior to the complete selection process. At each selection step, a random sample of 10% of the records will be used to test screening and eligibility tools. Disagreements will be resolved by discussion between the two reviewers. Any persisting discrepancy will be discussed with the senior researchers (HT and MHM).
In the interest of transparency, a list of excluded studies after full text assessment and detailed reasons of exclusion will be provided in review additional files. A PRISMA flow diagram will summarize the entire selection process. For each selection step, the flow diagram will indicate the number of retained and excluded records with a short description of the reasons for exclusion.
Data collection process
Data will be extracted from selected studies using a standardized form adapted from the Effective Practice and Organization of Care (EPOC) data collection form  (Additional file 4). To minimize errors in data collection, a pilot test of standardized form will be conducted on a random sample of 10% of included studies. The form will be revised as needed to insure that relevant information is collected. The two first reviewers will independently and in duplicate extract data from full text reports of included studies. Discrepancies will be resolved through consensus and if necessary by discussion with a third senior author. In the absence of important information (such as the number of APO and total sample size), we plan to contact (with a maximum of three email attempts) the primary author of included studies to obtain missing data.
Extracted data will include, but not be limited to, study characteristics (first author name, country, and year of publication), methods (aim of study, design, unit of observation, definition of APO), participants’ characteristics (source population description, inclusion and exclusion criteria, methods of recruitment, sample size, average age, HPV detection procedures, time of HPV detection), and number of each APO within HPV-exposed and HPV-unexposed groups. HPV genotypes will be extracted when possible. Some studies may include a composite measure of APO. We will extract, if possible, frequency of each APO from the composite outcome. If the information on each component of composite outcome is not available, we plan to contact the primary author to obtain missing data (with a maximum of three email attempts); otherwise, we will report the composite outcome with its definition provided in individual studies.
Risk of bias in individual studies
We will use the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies to evaluate methodological quality and risk of bias of included studies. The EPHPP is a standardized tool relevant to evaluate quality of quantitative observational studies . This quality assessment tool mainly encompasses the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist. The EPHPP has acceptable content and construct validity  as well as excellent inter-rater reliability compared to the Cochrane Collaboration Risk of Bias Tool (CCRBT) . The EPHPP tool evaluates six domains of study validity: selection bias, study design, confounders, blinding, data collection method, and withdrawals/dropouts . Each of six domains can be rated as strong, moderate, or weak. Based on individual domain ratings, the quality of each included study will be globally scored as either (1) strong (no weak individual domain rating); (2) moderate (one weak individual domain rating), or (3) weak (two or more weak individual domains ratings) . This bias and quality assessment will be independently and in duplicate carried out by the two first reviewers. We will compute agreement in individual domain ratings between reviewers for each included primary study based on the Cohen kappa coefficient. Discrepancies and study component ratings with kappa values below 0.80 will be resolved through discussion. Unresolved disagreement will be discussed with a third author. Differences among studies revealed by quality assessment of primary studies will guide, where appropriate, subsequent subgroup, or sensitivity analyses .
The purpose of our data synthesis will be twofold: (1) a narrative synthesis through which we will summarize and discuss findings of included studies and (2) a statistical analysis whereby we will investigate relationship between APO and HPV infection.
We will narratively synthesize included studies whether meta-analysis is appropriate or not. For the purpose of transparency and reproducibility, we will adopt, where applicable, the narrative synthesis guidance proposed by Popay et al. . The guidance describes a framework of four elements on which the narrative synthesis may be based: (1) developing a theory of how the intervention works, why, and for whom, (2) developing a preliminary synthesis of findings of included studies, (3) exploring relationships within and between studies, and (4) assessing the robustness of the synthesis. It is not required to proceed linearly or to apply all four elements in narrative synthesis. However, one chooses the elements to be employed depending on the nature of research question being reviewed . According to the objective of this review, we plan to use only the 2nd and 3rd elements of this guideline.
We will initially present findings first by primary outcomes and then by secondary outcomes. We will use tabular summary to synthesize individual studies characteristics and results (direction and magnitude of effects). In addition, we will explore factors, which might explain differences in direction and size of effect within the included studies [45, 74]. With particular consideration to potential confounders, we will assess results in light of participant’s characteristics and exposure measurement differences. Without computing a pooled estimate, we will visually assess the dispersion of effect estimates from primary studies using forest plots. We will finally conclude the narrative synthesis by an assessment of publication bias and strength of overall evidence.
Assessment of the extent of inconsistency
We will use forest plots to represent the dispersion of observed RRs. Along with the forest plot, we will compute I 2 statistic (with its 95% CI) to formally quantify the proportion of variance in observed RRs that reflects the true heterogeneity between studies rather than chance . According to criteria of Higgins et al., we will consider I 2 of more than 50% as substantial heterogeneity .
Assessment of the amount of dispersion
In addition to I 2 , we will report T 2 statistic as the absolute amount of dispersion of true effects, especially in case of meta-analyses with few studies . We will also compute the 95% prediction interval to estimate the range within which a hypothetical new true measure of association is expected to be found in 95 of 100 cases 
- 3.Investigating the causes of heterogeneity and limitations of previous reviews:
Subgroup analyses and meta-regression
We will investigate through subgroup analyses how clinical and methodological variability influence the pooled measure of association. Although both clinical and methodological diversity lead to statistical heterogeneity , we assume that clinical factors may vary more across included studies than methodological factors. Thus, clinical diversity could be more prone to affect clinical meaningful of an average summary estimate and heterogeneity than methodological diversity. Subgroup analyses will therefore be focused on the assessment of the effect of key characteristics of participants identified from literature, which may be associated with the primary outcomes of interest and uncontrolled in previous reviews. Thus, we will conduct subgroup analyses based, but not limited to, maternal age, parity, race, smoking, drug use, genital infections, and prior APO. We will also group and compare studies that have matched/controlled and not matched/not controlled data on some of these key variables (age, parity, and smoking). Where appropriate, we will conduct a meta-regression on the basis of study average or proportion values of key participant’s characteristics.
We will assess robustness of our findings by restricting the analysis to a subset of studies with high or moderate quality based on EPHPP score. Sensitivity analysis will be undertaken to take into account quality relative to confounding bias by stratification of most adjusted and least adjusted estimates from each study. Similarly, we plan to assess the effect of time point detection of HPV by restricting analysis to different alternative exposure windows (12 months prior to pregnancy, pregnancy period, or 12 months postpartum). Finally, we will investigate the extent to which inclusion of studies that used cytology as exposure measurement affects the results.
If there are at least 10 studies for each outcome, we will use funnel plot to explore the potential of publication bias and small studies effect . Additionally, Egger’s test for funnel plot asymmetry will be conducted . However, in case of less than 10 studies, we will assess qualitatively the small study effect. For this purpose, we will perform a cumulative meta-analysis sorting studies from largest to smallest study to examine the effect of study size on the pooled estimate . We will assess selective reporting within included studies by comparing outcomes reported in methods and results sections .
Confidence in cumulative evidence
We will use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess and grade the confidence of evidence for each outcome across included studies . Quality rating of overall evidence will be downgraded according to five factors: (1) limitations in the design (risk of bias across studies); (2) indirectness (exposure or outcomes are different from those defined in inclusion criteria of this protocol); (3) unexplained heterogeneity (after subgroup analyses); (4) imprecision of effect estimates (few participants or few events); and (5) risk of publication bias  . In addition and where appropriate, quality rating will be upgraded according to the following three factors: (1) a large magnitude of effect, (2) a dose response gradient (e.g., the more types of HPV are identified, the high is the risk of pregnancy outcome), and (3) plausible residual confounding that would reduce a demonstrated effect or suggest a spurious effect when results show no effect .We will integrate downgrading and upgrading factors to obtain an overall quality of evidence for each outcome of interest. Overall quality of evidence will be then ranked as high, moderate, low, or very low as specified by GRADE approach . However, in order to be more explicit about overall level of confidence of our findings, we will translate GRADE levels into categories suggested by Young et al.  as high (further research is very unlikely to change our confidence in the estimate of effect), moderate (further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate), low (further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate), or very low (very uncertain about the estimate of effect). Finally, key information concerning the quality of evidence of all outcomes will be concisely combined in “summary of findings” table.
This review will rely only on observational studies, which are prone to confounding biases. Moreover, we expect that there will be much heterogeneity in included studies in terms of study design, exposure measurement, characteristics of participants, and outcomes. Nevertheless, to our knowledge, this will be the first review that will use a validated systematic and transparent methodological process to appraise and summarize the existing literature on HPV-associated pregnancy outcomes. Thus, whatever conclusion our study reaches (harmful effect, absence of effect, or uncertain effect), we are confident that this review will provide rigorous and comprehensive high-quality evidence related to HPV-associated pregnancy outcomes. Indeed, we expect that our findings will ultimately fall into one of the following three scenarios. Firstly, if our findings are in favor of a harmful effect of HPV infection on pregnancy outcomes, this might explain a part of APO so far classified as idiopathic. Therefore, the fact that HPV infection might explain some APO could strengthen primary prevention through vaccination. Secondarily, in case of absence of relationship, this might dispel anxiety and reassure HPV-infected pregnant women and health professionals. Thirdly, if ever we end up being uncertain of the effect, we will provide an overview of gaps in current knowledge and propose further research directions on HPV-pregnancy outcome relationship.
Dissemination of findings
We will report this review in accordance to the Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines  and the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) . This systematic review will be part of JN’s PhD research thesis co-supervised by HT and MHM. This systematic review will also be submitted for publication in a relevant peer-reviewed journal, and findings will be presented in scientific conferences.
American College of Obstetricians and Gynecologists
Adverse pregnancy outcomes
Cochrane Collaboration Risk of Bias Tool
Canadian Institutes of Health Research
Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies
Effective Practice and Organization of Care
European Research Organization on Genital Infection and Neoplasia conference
Fonds de la Recherche du Québec en Santé
Grading of Recommendations Assessment, Development and Evaluation
International Papillomavirus conference
Intrauterine growth restriction
Low birth weight
Medical Subject Headings
Meta-analysis of Observational Studies in Epidemiology guidelines
Population, Intervention-Exposure, Comparator, Outcomes and Study design
Pregnancy-induced hypertensive disorders
Products of conception
Preterm pre-labor rupture of membranes
Preferred Reporting Items for Systematic Reviews and Meta-analyses
Preferred Reporting Items for Systematic review and Meta-Analysis Protocols
Quebec Training Network in Perinatal Research
Society of Obstetricians and Gynecologists of Canada
Strengthening the Reporting of Observational Studies in Epidemiology
World Health Organization-International statistical Classification of Diseases-tenth revision
We would like to thank Philippe Dodin, the health specialist librarian at Sainte-Justine Hospital (Montreal, Canada), for his valuable contribution to search strategy development.
No funding has been received for the review protocol or the review. JN holds a PhD scholarship from Quebec Training Network in Perinatal Research (QTNPR). Funding for PhD award (to JN) was also provided in part by a grant from the Canadian Institutes of Health Research (CIHR) (MOP-136833) to HT. HT holds a salary award (chercheur-boursier) from the FRQ-S and from CIHR (New investigator salary award). MHM holds a salary award (chercheur-boursier clinicien) from the FRQ-S.
Availability of data and materials
JN, HT, and MHM designed the study. JN wrote the first manuscript of the protocol. NZ, HT, and MHM critically revised the protocol. All authors read and approved the final manuscript.
JN is a MD, MPH, and PhD candidate in Epidemiology. NZ is a MSc. HT is a PhD in Epidemiology. MHM is a MD in Obstetrics-Gynecology and PhD in Epidemiology.
JN and NZ declare no competing interests. HT has received occasional lecture or consultation fees from GlaxoSmithKline Biological and Merck. MHM was a site principal investigator for a Merck clinical trial on HPV vaccination.
Consent for publication
Ethics approval and consent to participate
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Satterwhite CL, Torrone E, Meites E, Dunne EF, Mahajan R, Ocfemia MC, Su J, Xu F, Weinstock H. Sexually transmitted infections among US women and men: prevalence and incidence estimates, 2008. Sex Transm Dis. 2013;40(3):187–93.View ArticlePubMedGoogle Scholar
- Chesson HW, Dunne EF, Hariri S, Markowitz LE. The estimated lifetime probability of acquiring human papillomavirus in the United States. Sex Transm Dis. 2014;41(11):660–4.View ArticlePubMedGoogle Scholar
- Bosch FX, Burchell AN, Schiffman M, Giuliano AR, de Sanjose S, Bruni L, Tortolero-Luna G, Kjaer SK, Munoz N. Epidemiology and natural history of human papillomavirus infections and type-specific implications in cervical neoplasia. Vaccine 2008, 26 Suppl 10:K1–16.
- Trottier H, Burchell AN. Epidemiology of mucosal human papillomavirus infection and associated diseases. Public Health Genomics. 2009;12(5-6):291–307.View ArticlePubMedGoogle Scholar
- Bruni L, Diaz M, Castellsague X, Ferrer E, Bosch FX, de Sanjose S. Cervical human papillomavirus prevalence in 5 continents: meta-analysis of 1 million women with normal cytological findings. J Infect Dis. 2010;202(12):1789–99.View ArticlePubMedGoogle Scholar
- Trottier H, Ferreira S, Thomann P, Costa MC, Sobrinho JS, Prado JC, Rohan TE, Villa LL, Franco EL. Human papillomavirus infection and reinfection in adult women: the role of sexual activity and natural immunity. Cancer Res. 2010;70(21):8569–77.View ArticlePubMedPubMed CentralGoogle Scholar
- Smith JS, Gilbert PA, Melendy A, Rana RK, Pimenta JM. Age-specific prevalence of human papillomavirus infection in males: a global review. J Adolesc Health. 2011;48(6):540–52.View ArticlePubMedGoogle Scholar
- Rachel Skinner S, Wheeler CM, Romanowski B, Castellsague X, Lazcano-Ponce E, Rowena Del Rosario-Raymundo M, Vallejos C, Minkina G, Pereira Da Silva D, McNeil S, et al. Progression of HPV infection to detectable cervical lesions or clearance in adult women: Analysis of the control arm of the VIVIANE study. Int J Cancer. 2016;138(10):2428–38.View ArticlePubMedPubMed CentralGoogle Scholar
- Liu P, Xu L, Sun Y, Wang Z. The prevalence and risk of human papillomavirus infection in pregnant women. Epidemiol Infect. 2014;142(8):1567–78.View ArticlePubMedGoogle Scholar
- Jalil EM, Bastos FI, Melli PPS, Duarte G, Simoes RT, Yamamoto AY, Morais RAA, Quintana SM. HPV clearance in postpartum period of HIV-positive and negative women: a prospective follow-up study. BMC Infect Dis. 2013;13:564.View ArticlePubMedPubMed CentralGoogle Scholar
- Nobbenhuis MAE, Helmerhorst TJM, van den Brule AJC, Rozendaal L, Bezemer PD, Voorhorst FJ, Meijer CJLM. High-risk human papillomavirus clearance in pregnant women: trends for lower clearance during pregnancy with a catch-up postpartum. Br J Cancer. 2002;87(1):75–80.View ArticlePubMedPubMed CentralGoogle Scholar
- Fife KH, Katz BP, Brizendine EJ, Brown DR. Cervical human papillomavirus deoxyribonucleic acid persists throughout pregnancy and decreases in the postpartum period. Am J Obstet Gynecol. 1999;180(5):1110–4.View ArticlePubMedGoogle Scholar
- Castellsague X, Drudis T, Canadas MP, Gonce A, Ros R, Perez JM, Quintana MJ, Munoz J, Albero G, de Sanjose S, et al. Human papillomavirus (HPV) infection in pregnant women and mother-to-child transmission of genital HPV genotypes: a prospective study in Spain. BMC Infect Dis. 2009;9:74.View ArticlePubMedPubMed CentralGoogle Scholar
- Druckmann R, Druckmann MA. Progesterone and the immunology of pregnancy. J Steroid Biochem Mol Biol. 2005;97(5):389–96.View ArticlePubMedGoogle Scholar
- Romero R, Espinoza J, Kusanovic JP, Gotsch F, Hassan S, Erez O, Chaiworapongsa T, Mazor M. The preterm parturition syndrome. Bjog. 2006;113 Suppl 3:17–42.View ArticlePubMedGoogle Scholar
- Donati L, Di Vico A, Nucci M, Quagliozzi L, Spagnuolo T, Labianca A, Bracaglia M, Ianniello F, Caruso A, Paradisi G. Vaginal microbial flora and outcome of pregnancy. Arch Gynecol Obstet. 2010;281(4):589–600.View ArticlePubMedGoogle Scholar
- Giakoumelou S, Wheelhouse N, Cuschieri K, Entrican G, Howie SE, Horne AW. The role of infection in miscarriage. Hum Reprod Update. 2016;22(1):116–33.View ArticlePubMedGoogle Scholar
- Manuck TA, Esplin MS, Biggio J, Bukowski R, Parry S, Zhang H, Varner MW, Andrews W, Saade G, Sadovsky Y, et al. The phenotype of spontaneous preterm birth: application of a clinical phenotyping tool. Am J Obstet Gynecol. 2015;212(4): 487.e1-487.e11.
- Rustveld LO, Kelsey SF, Sharma R. Association between maternal infections and preeclampsia: a systematic review of epidemiologic studies. Matern Child Health J. 2008;12(2):223–42.View ArticlePubMedGoogle Scholar
- Payne MS, Bayatibojakhi S. Exploring preterm birth as a polymicrobial disease: an overview of the uterine microbiome. Front Immunol. 2014;5:595.View ArticlePubMedPubMed CentralGoogle Scholar
- Rositch AF, Koshiol J, Hudgens MG, Razzaghi H, Backes DM, Pimenta JM, Franco EL, Poole C, Smith JS. Patterns of persistent genital human papillomavirus infection among women worldwide: a literature review and meta-analysis. Int J Cancer. 2013;133(6):1271–85.View ArticlePubMedGoogle Scholar
- Schiffman M, Kjaer SK. Chapter 2: Natural history of anogenital human papillomavirus infection and neoplasia. J Natl Cancer Inst Monogr. 2003;31:14–9.View ArticleGoogle Scholar
- Gravitt PE. The known unknowns of HPV natural history. J Clin Invest. 2011;121(12):4593–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Armbruster-Moraes E, Ioshimoto LM, Leao E, Zugaib M. Detection of human papillomavirus deoxyribonucleic acid sequences in amniotic fluid during different periods of pregnancy. Am J Obstet Gynecol. 1993;169(4):1074.View ArticlePubMedGoogle Scholar
- Weyn C, Thomas D, Jani J, Guizani M, Donner C, Van Rysselberge M, Hans C, Bossens M, Englert Y, Fontaine V. Evidence of human papillomavirus in the placenta. J Infect Dis. 2011;203(3):341–3.View ArticlePubMedPubMed CentralGoogle Scholar
- Sarkola ME, Grenman SE, Rintala MA, Syrjanen KJ, Syrjanen SM. Human papillomavirus in the placenta and umbilical cord blood. Acta Obstet Gynecol Scand. 2008;87(11):1181–8.View ArticlePubMedGoogle Scholar
- Lee SM, Park JS, Norwitz ER, Koo JN, Oh IH, Park JW, Kim SM, Kim YH, Park CW, Song YS. Risk of vertical transmission of human papillomavirus throughout pregnancy: a prospective study. PLoS One. 2013;8(6):e66368.View ArticlePubMedPubMed CentralGoogle Scholar
- Boulenouar S, Weyn C, Van Noppen M, Moussa Ali M, Favre M, Delvenne PO, Bex F, Noel A, Englert Y, Fontaine V. Effects of HPV-16 E5, E6 and E7 proteins on survival, adhesion, migration and invasion of trophoblastic cells. Carcinogenesis. 2010;31(3):473–80.View ArticlePubMedGoogle Scholar
- Gomez LM, Ma Y, Ho C, McGrath CM, Nelson DB, Parry S. Placental infection with human papillomavirus is associated with spontaneous preterm delivery. Hum Reprod. 2008;23(3):709–15.View ArticlePubMedGoogle Scholar
- You H, Liu Y, Agrawal N, Prasad CK, Edwards JL, Osborne AF, Korourian S, Lowery CL, Hermonat PL. Multiple human papillomavirus types replicate in 3A trophoblasts. Placenta. 2008;29(1):30–8.View ArticlePubMedGoogle Scholar
- Hong LJ, Oshiro BT, Chan PJ. HPV-16 exposed mouse embryos: a potential model for pregnancy wastage. Arch Gynecol Obstet. 2013;287(6):1093–7.View ArticlePubMedGoogle Scholar
- Henneberg AA, Patton WC, Jacobson JD, Chan PJ. Human papilloma virus DNA exposure and embryo survival is stage-specific. J Assist Reprod Genet. 2006;23(6):255–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Calinisan JH, Chan SR, King A, Chan PJ. Human papillomavirus and blastocyst apoptosis. J Assist Reprod Genet. 2002;19(3):132–6.View ArticlePubMedPubMed CentralGoogle Scholar
- Kwon JY, Romero R, Mor G. New insights into the relationship between viral infection and pregnancy complications. Am J Reprod Immunol. 2014;71(5):387–90.View ArticlePubMedGoogle Scholar
- Hermonat PL, Han L, Wendel PJ, Quirk JG, Stern S, Lowery CL, Rechtin TM. Human papillomavirus is more prevalent in first trimester spontaneously aborted products of conception compared to elective specimens. Virus Genes. 1997;14(1):13–7.View ArticlePubMedGoogle Scholar
- McDonnold M, Dunn H, Hester A, Pacheco LD, Hankins GD, Saade GR, Costantine MM. High risk human papillomavirus at entry to prenatal care and risk of preeclampsia. Am J Obstet Gynecol. 2014;210(2):138. e131-135.View ArticlePubMedGoogle Scholar
- Cho G, Min KJ, Hong HR, Kim S, Hong JH, Lee JK, Oh MJ, Kim H. High-risk human papillomavirus infection is associated with premature rupture of membranes. BMC Pregnancy Childbirth. 2013;13:173.View ArticlePubMedPubMed CentralGoogle Scholar
- Zuo Z, Goel S, Carter JE. Association of cervical cytology and HPV DNA status during pregnancy with placental abnormalities and preterm birth. Am J Clin Pathol. 2011;136(2):260–5.View ArticlePubMedGoogle Scholar
- Hong JN, Berggren EK, Campbell SL, Smith JS, Rahangdale L. Abnormal cervical cancer screening in pregnancy and preterm delivery. Paediatr Perinat Epidemiol. 2014;28(4):297–301.View ArticlePubMedGoogle Scholar
- Conde-Ferraez L, Chan May Ade A, Carrillo-Martinez JR, Ayora-Talavera G, Gonzalez-Losa Mdel R. Human papillomavirus infection and spontaneous abortion: a case-control study performed in Mexico. Eur J Obstet Gynecol Reprod Biol. 2013;170(2):468–73.View ArticlePubMedGoogle Scholar
- Skoczynski M, Gozdzicka-Jozefiak A, Kwasniewska A. Prevalence of human papillomavirus in spontaneously aborted products of conception. Acta Obstet Gynecol Scand. 2011;90(12):1402–5.View ArticlePubMedGoogle Scholar
- Subramaniam A, Lees BF, Becker DA, Tang Y, Khan MJ, Edwards RK. Evaluation of human papillomavirus as a risk factor for preterm birth or pregnancy-related hypertension. Obstet Gynecol. 2016;127(2):233–40.View ArticlePubMedGoogle Scholar
- Ticconi C, Pietropolli A, Fabbri G, Capogna MV, Perno CF, Piccione E. Recurrent miscarriage and cervical human papillomavirus infection. Am J Reprod Immunol. 2013;70(5):343–6.PubMedGoogle Scholar
- Slatter TL, Hung NG, Clow WM, Royds JA, Devenish CJ, Hung NA. A clinicopathological study of episomal papillomavirus infection of the human placenta and pregnancy complications. Mod Pathol. 2015;28(10):1369–82.View ArticlePubMedGoogle Scholar
- Higgins JPT, Green S. (editors): Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011] The Cochrane Collaboration; 2011. Available from www.handbook.cochrane.org
- Bonde U, Joergensen JS, Mogensen O, Lamont RF. The potential role of HPV vaccination in the prevention of infectious complications of pregnancy. Expert Rev Vaccines. 2014;13(11):1307–16.View ArticlePubMedGoogle Scholar
- Huang QT, Zhong M, Gao YF, Huang LP, Huang Q, Wang W, Wang ZJ, Yu YH. Can HPV vaccine have other health benefits more than cancer prevention? A systematic review of association between cervical HPV infection and preterm birth. J Clin Virol. 2014;61(3):321–8.View ArticlePubMedGoogle Scholar
- Yang R, Wang Y, Qiao J, Liu P, Geng L, Guo YL. Does human papillomavirus infection do harm to in-vitro fertilization outcomes and subsequent pregnancy outcomes? Chin Med J. 2013;126(4):683–7.PubMedGoogle Scholar
- Schiller JT, Castellsague X, Garland SM. A review of clinical trials of human papillomavirus prophylactic vaccines. Vaccine. 2012;30 Suppl 5:F123–38.View ArticlePubMedPubMed CentralGoogle Scholar
- Sjoborg KD, Eskild A. Vaccination against human papillomavirus—an impact on preterm delivery? Estimations based on literature review. Acta Obstet Gynecol Scand. 2009;88(3):255–60.View ArticlePubMedGoogle Scholar
- Soergel P, Makowski L, Schippert C, Staboulidou I, Hille U, Hillemanns P. The cost efficiency of HPV vaccines is significantly underestimated due to omission of conisation-associated prematurity with neonatal mortality and morbidity. Hum Vaccin Immunother. 2012;8(2):243–51.View ArticlePubMedGoogle Scholar
- Castle PE, Maza M. Prophylactic HPV vaccination: past, present, and future. Epidemiol Infect. 2016;144(3):449–68.View ArticlePubMedGoogle Scholar
- Berenson AB, Patel PR, Barrett AD. Is administration of the HPV vaccine during pregnancy feasible in the future? Expert Rev Vaccines. 2014;13(2):213–9.View ArticlePubMedGoogle Scholar
- Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.View ArticlePubMedPubMed CentralGoogle Scholar
- Rothman KJ. Induction and latent periods. Am J Epidemiol. 1981;114(2):253–9.View ArticlePubMedGoogle Scholar
- ACOG. The American College of Obstetricians and Gynecologists Practice Bulletin No 150: Early pregnancy loss. Obstetrics and gynecology. 2015;125(5):1258-67.
- ACOG. The American College of Obstetricians and Gynecologists Practice Bulletin No. 159 Summary: management of preterm labor. Obstetrics Gynecology. 2016;127(1):190–1.
- Gibson CS, Goldwater PN, MacLennan AH, Haan EA, Priest K, Dekker GA. Fetal exposure to herpesviruses may be associated with pregnancy-induced hypertensive disorders and preterm birth in a Caucasian population. BJOG. 2008;115(4):492–500.View ArticlePubMedGoogle Scholar
- Kramer MS. The epidemiology of adverse pregnancy outcomes: an overview. J Nutr. 2003;133(5 Suppl 2):1592S–6S.PubMedGoogle Scholar
- Auger N, Le TU, Park AL, Luo ZC. Association between maternal comorbidity and preterm birth by severity and clinical subtype: retrospective cohort study. BMC Pregnancy Childbirth. 2011;11:67.View ArticlePubMedPubMed CentralGoogle Scholar
- Ananth CV, Vintzileos AM. Epidemiology of preterm birth and its clinical subtypes. J Maternal-Fetal Neonatal Med. 2006;19(12):773–82.View ArticleGoogle Scholar
- Goldenberg RL, McClure EM, Saleem S, Reddy UM. Infection-related stillbirths. Lancet. 2010;375(9724):1482–90.View ArticlePubMedPubMed CentralGoogle Scholar
- ACOG. ACOG Practice Bulletin No. 102: management of stillbirth. Obstetrics Gynecol. 2009;113(3):748–61.View ArticleGoogle Scholar
- World Health Organization. International statistical classification of diseases and related health problems, tenth revision, version 2015. Geneva: World Health Organization; 2015.Google Scholar
- ACOG. Practice bulletin. Diagnosis and management of preeclampsia and eclampsia. Number 33, January 2002. Obstetrics Gynecol. 2002;99(1):159–67.Google Scholar
- ACOG. Practice Bulletin No. 160: premature rupture of membranes. Obstetrics Gynecol. 2016;127(1):e39-51.Google Scholar
- Lausman A, Kingdom J, Maternal Fetal Medicine C, Gagnon R, Basso M, Bos H, Crane J, Davies G, Delisle MF, Hudon L, et al. Intrauterine growth restriction: screening, diagnosis, and management. J Obstet Gynaecol Can. 2013;35(8):741–57.View ArticlePubMedGoogle Scholar
- ACOG. Screening tests for birth defects. http://www.acog.org/Patients/FAQs/Screening-Tests-for-Birth-Defects-what. Accessed 17 Jan 2017.
- Data extraction and management. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services. 2013. http://epoc.cochrane.org/epoc-specific-resources-review-authors. Accessed 25 June 2016.
- Armijo-Olivo S, Stiles CR, Hagen NA, Biondo PD, Cummings GG: Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract. 2012, 18(1):12-18.
- Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews Evid Based Nurs. 2004;1(3):176–84.View ArticlePubMedGoogle Scholar
- Quality Assessment Tool for Quantitative Studies. http://www.ephpp.ca/PDF/Quality%20Assessment%20Tool_2010_2.pdf. Accessed 25 June 2016.
- Meade MO, Richardson WS. Selecting and appraising studies for a systematic review. Ann Intern Med. 1997;127(7):531–7.View ArticlePubMedGoogle Scholar
- Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M. Guidance on the conduct of narrative synthesis in systematic reviews. Lancaster: ESRC Research Methods Programme; 2006.Google Scholar
- Centre for Reviews and Dissemination. Guidance for undertaking reviews in health care. UK: CRD, University of York; 2009.Google Scholar
- Borenstein MH, Hedges LV, Higgins JPT, Rothstein HR. Random-effects model. In: Introduction to Meta-Analysis. United Kingdom: John Wiley and Sons, Ltd; 2009: 70-72.
- Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009;172(1):137–59.View ArticlePubMedPubMed CentralGoogle Scholar
- Borenstein MH, Hedges LV, Higgins JPT, Rothstein HR. Publication bias. In: Introduction to meta-Analysis. United Kingdom: John Wiley and Sons, Ltd; 2009. p. 277–92.
- Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.View ArticlePubMedGoogle Scholar
- Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.View ArticlePubMedGoogle Scholar
- Young MM, Stevens A, Porath-Waller A, Pirie T, Garritty C, Skidmore B, Turner L, Arratoon C, Haley N, Leslie K, et al. Effectiveness of brief interventions as part of the screening, brief intervention and referral to treatment (SBIRT) model for reducing the non-medical use of psychoactive substances: a systematic review protocol. Syst Rev. 2012;1:22.View ArticlePubMedPubMed CentralGoogle Scholar
- Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283(15):2008–12.View ArticlePubMedGoogle Scholar
- Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–e34.View ArticlePubMedGoogle Scholar