Benefits and harms of the human papillomavirus (HPV) vaccines: systematic review with meta-analyses of trial data from clinical study reports

Abstract

Objective

To assess the benefits and harms of the human papillomavirus (HPV) vaccines.

Data sources

Clinical study reports obtained from the European Medicines Agency and GlaxoSmithKline from 2014 to 2017.

Eligibility criteria

Randomised trials that compared an HPV vaccine with a placebo or active comparator in healthy participants of all ages.

Appraisal and synthesis

Two researchers extracted data and judged risk of bias with the Cochrane tool (version 2011). Risk ratio (RR) estimates were pooled using random-effects meta-analysis.

Outcomes

Clinically relevant outcomes in intention to treat populations—including HPV-related cancer precursors irrespective of involved HPV types, treatment procedures and serious and general harms.

Results

Twenty-four of 50 eligible clinical study reports were obtained with 58,412 pages of 22 trials and 2 follow-up studies including 95,670 participants: 79,102 females and 16,568 males age 8–72; 393,194 person-years; and 49 months mean weighted follow-up. We judged all 24 studies to be at high risk of bias. Serious harms were incompletely reported for 72% of participants (68,610/95,670). Nearly all control participants received active comparators (48,289/48,595, 99%). No clinical study report included complete case report forms. At 4 years follow-up, the HPV vaccines reduced HPV-related carcinoma in situ (367 in the HPV vaccine group vs. 490 in the comparator group, RR 0.73 [95% confidence interval, CI, 0.53 to 1.00], number needed to vaccinate [NNV] 387, P = 0.05, I² = 67%) and HPV-related treatment procedures (1018 vs. 1416, RR 0.71 [95% CI 0.63 to 0.80], NNV 75, P < 0.00001, I² = 45%). The HPV vaccines increased serious nervous system disorders (exploratory analysis: 72 vs. 46, RR 1.49 [1.02 to 2.16], number needed to harm [NNH] 1325, P = 0.040, I² = 0%) and general harms (13,248 vs. 12,394, RR 1.07 [95% CI 1.03 to 1.11], NNH 51, P = 0.0002, I² = 77%) but did not significantly increase fatal harms (45 vs. 38, RR 1.19 [95% CI 0.65 to 2.19], P = 0.58, I² = 30%) or serious harms (1404 vs. 1357, RR 1.01 [95% CI 0.94 to 1.08], P = 0.79, I² = 0%).

Conclusion

At 4 years follow-up, the HPV vaccines decreased HPV-related cancer precursors and treatment procedures but increased serious nervous system disorders (exploratory analysis) and general harms. As the included trials were primarily designed to assess benefits and were not adequately designed to assess harms, the extent to which the HPV vaccines’ benefits outweigh their harms is unclear. Limited access to clinical study reports and trial data with case report forms prevented a thorough assessment.

Systematic review registration

CRD42017056093. Our systematic review protocol was registered on PROSPERO in January 2017: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20170030.pdf. Two protocol amendments were registered on PROSPERO on November 2017: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20171116.pdf. Our index of the HPV vaccine studies was published in Systematic Reviews in January 2018: https://doi.org/10.1186/s13643-018-0675-z. A description of the challenges obtaining the data was published in September 2018: https://doi.org/10.1136/bmj.k3694.

Introduction

The approved human papillomavirus (HPV) vaccines—GlaxoSmithKline’s Cervarix™ and Merck Sharp and Dohme’s Gardasil™ and Gardasil 9™—are considered safe and effective [1,2,3]. Recent evidence suggests that the vaccines have significant and long-lasting effects (> 12 years) on cervical cancer [4, 5], better effectiveness when vaccinated below the age of 17 [6], and are possibly able to substantially reduce the global incidence of cervical cancer [7]. However, there are important uncertainties regarding both the benefits and harms of the vaccines.

Uncertainties of the benefits of the HPV vaccines

The HPV vaccines’ regulatory approvals were mainly based on per-protocol populations and surrogate outcomes of HPV-related lesions, e.g. ‘cervical intraepithelial neoplasia or worse’ (CIN2⁺) infected with an HPV vaccine-specific HPV type, such as HPV types 16 and 18 that are associated with the majority of HPV-related cancers [8,9,10]. It was considered unfeasible and unethical to use HPV-related cancer as the primary outcome [11, 12], since it takes many years for cancer to develop after an HPV infection and also because cervical screening is an established secondary prevention method that leads to removal of precancerous lesions before they become cancerous. Up to 15% of HPV-related cervical cancers may not contain HPV [13], but HPV may be identified in more cases with newer and more sensitive analysis methods [14]. HPV-related lesions are often infected with more than one HPV type, some of which may not be targeted by the vaccines [15]. This makes it impossible to assess which HPV type caused the lesion. The regulatory vaccine approvals were not based on HPV-related lesions irrespective of HPV type in intention to treat populations, and factors such as antigenic changes and herd immunity may be important in the long-term perspective, as the approved HPV vaccines only target up to 9 of the 25 HPV types considered oncogenic [1].

Uncertainties of the harms of the HPV vaccines

A Cochrane review from 2018 [3] and most large epidemiological studies [16,17,18,19,20] did not find serious or general harms associated with the HPV vaccines. The Cochrane review was mainly based on journal publications that often are influenced by reporting bias [21,22,23,24], and epidemiological studies are influenced by confounding [25].

Acknowledged rare serious harms include anaphylaxis and syncope [8,9,10]. Some case studies have reported rare neurological harms such as postural orthostatic tachycardia syndrome (POTS) [26, 27] and complex regional pain syndrome (CRPS) [28]. Cluster analyses of individual case safety reports from the World Health Organisation’s (WHOs) VigiBase® revealed additional harms—often serious in nature—that overlapped with the symptomatology of POTS and CRPS [29]. Although the European Medicines Agency’s (EMA) investigation of POTS and CRPS did not find an association with the HPV vaccines [2], EMA’s investigation was based on the HPV vaccine manufacturers’ own assessments [30], and about 30 cases of POTS and CRPS were not recognised in the HPV vaccine manufacturers’ trials [31, 32]. Other reported rare harms have included chronic fatigue syndrome (CFS), Guillain–Barré syndrome (GBS) and premature ovarian failure (POF) [33,34,35].

Addressing the uncertainties of the HPV vaccines

To address the uncertainties of the benefits and harms of the HPV vaccines, we conducted a systematic review with meta-analyses of trial data from clinical study reports. As of July 2017, about one third of the HPV vaccine studies had not been published and study results were not posted for about half of the completed studies on ClinicalTrials.gov [36]. Therefore, we based our review on study programmes in order to identify all trials [36] and on clinical study reports [37], as these reports provide vastly more information about a study than a corresponding journal publication [21,22,23,24].

Methods

Search strategy and study eligibility

Using a six-step process, we constructed and published an index of the HPV vaccine study programmes [36] that included 206 comparative prospective studies (see Fig. 1). Two researchers (LJ and TJ) conducted the six steps that included searches of trial registers, journal publication databases and correspondence with regulators and HPV vaccine manufacturers. It was not feasible to account for duplicate entries, as we indexed studies and searched databases that used different IDs for a unique study (e.g. register ID, study programme ID, manufacturer ID and publication ID) [36].

Fig. 1

Benefits and harms of the HPV vaccines: flowchart of the inclusion of clinical study reports. For details on the correspondence and searches conducted in steps 1 to 6, see Jørgensen et al. ([36]: Appendices 1 and 2). Two hundred six studies were identified according to our inclusion and classification criteria, see Jørgensen et al. ([36]: Methods). N = the number of studies/entries evaluated

In May 2014, we requested the study programmes’ corresponding clinical study reports from the European Medicines Agency (EMA; via its policy 0043) and obtained those reports that were freely available on GlaxoSmithKline’s online trial register. We did not request clinical study reports from the manufacturers, as this would limit our ability to use and share the data [38]. In January 2017, we registered our systematic review protocol in PROSPERO (International prospective register of systematic reviews): CRD42017056093 [37].

We included those trials and their follow-up studies of the 206 comparative studies from our index that were randomised clinical phase II, III or IV trials. We aimed to include studies for which we obtained industry clinical study reports or similar non-industry reports. In the event of no clinical study report being available (for an otherwise eligible trial), we did not include data from the trial publication. We also aimed to include periodical safety update reports. PICO criteria (participants, interventions, comparisons and outcomes) were used to select trials that compared an HPV vaccine with a placebo (normal saline) or active comparator (adjuvant or non-HPV vaccine such as a hepatitis vaccine) in healthy participants (see Additional file 1 for our PRISMA checklist).

Data extraction and risk of bias assessment

One researcher performed trial selection and data extraction (LJ); a second researcher (TJ) checked the selection and extraction; a third researcher (PCG) arbitrated. Cochrane’s tool (version 2011) was used for risk of bias assessments [25].

Outcome assessment

We assessed the following primary outcomes: all-cause mortality, deaths from and incidence of HPV-related cancers, incidence of histologically confirmed carcinoma in situ and moderate intraepithelial neoplasia, fatal harms, serious harms and harms of special interest (anaphylaxis, chronic fatigue syndrome [CFS], complex regional pain syndrome [CRPS], Guillain-Barré syndrome [GBS], postural orthostatic tachycardia syndrome [POTS], premature ovarian failure [POF] and syncope). Histological outcomes were assessed irrespective of which HPV types were involved.

Secondary outcomes included HPV-related external genital lesions and referral procedures, new onset diseases (reported in the included clinical study reports as ‘medically significant conditions’ and ‘new medical history’) and general harms (reported as 'solicited', ‘unsolicited’ and ‘systemic adverse events’). We did not consider cytological, serological or virological outcomes or local harms due to their lower clinical importance.

The clinical study reports included over 3000 different types of harms that were classified with MedDRA (Medical Dictionary for Regulatory Activities) preferred terms. The harms were often incompletely and heterogeneously reported (see Table 1). We extracted and assessed all individual harms classified with MedDRA-preferred terms. We performed meta-analyses for the five most commonly occurring fatal and serious harms, the five fatal and serious harms that the HPV vaccines increased the most and the five fatal and serious harms that the HPV vaccines decreased the most. For new onset diseases and general harms, we performed meta-analyses for the three most common, increased and decreased harms for each category (‘medically significant conditions’ and ‘new medical history’; and ‘solicited’, ‘unsolicited’ and ‘systemic adverse events’). MedDRA-preferred terms and total harms were reported as the number of participants with one or more harms over the total number of participants.

Table 1 Benefits and harms of the HPV vaccines: reporting of harms in included HPV vaccine studies

To check for possible harm clustering on an organ system level, we meta-analysed the MedDRA-preferred terms in their respective system organ classes (for example, the MedDRA-preferred terms ‘dizziness’, ‘pain’ and ‘syncope’ were part of and therefore included in the MedDRA system organ class ‘nervous system disorders’). Only Merck clinical study reports included aggregate numbers for participants with MedDRA system organ class harms, and only for new onset diseases (‘new medical history’) and general harms (‘systemic adverse events’). For all GlaxoSmithKline clinical study reports and for serious harms for Merck clinical study reports, we pooled MedDRA-preferred terms in their respective system organ classes. A participant could potentially be included more than once in a separate analysis (e.g. if a participant experienced a serious ‘headache’ and serious ‘dizziness’, the participant would be counted twice in the MedDRA system organ class analysis of serious nervous system disorders); we therefore consider these MedDRA system organ class analyses exploratory.

Post hoc exploratory outcome assessment

As we did not obtain complete case report forms or individual participant data for any trial, and as the trials’ harm assessments had low internal and external validity (see Table 1 and the “Discussion” section), we performed post hoc exploratory outcome analyses where we (1) compared the clinical study report data with pharmacovigilance data; and (2) assessed signs and symptoms of POTS and CRPS (see protocol amendment on PROSPERO [39]).

• 1) We compared the three largest harm clusters reported from pharmacovigilance up to 1 January 2015 to the World Health Organisation’s (WHO) VigiBase® [29] with the clinical study report data (for example, VigiBase’s largest HPV vaccine harm cluster—‘expected systemic reactions’—consists of the MedDRA-preferred terms headache, nausea, pyrexia, dizziness and vomiting). This was done to assess if the pharmacovigilance data were comparable to the clinical study report data. We used the individual harm cluster terms and found the corresponding MedDRA-preferred terms in the clinical study report data. The data were synthesised or those MedDRA-preferred terms included in each harm cluster.

• 2) POTS and CRPS are rare syndromes that are difficult to identify; as mentioned, about 30 cases of POTS and CRPS were not recognised in the HPV vaccine manufacturers’ trials [31, 32], and there were no reports of POTS and CRPS in the clinical study reports (see Table 9 and the “Results” section). To assess whether signs and symptoms consistent with POTS and CRPS were present in the data, we asked a physician (Louise Brinth) with clinical expertise in POTS and CRPS to assess the reported MedDRA-preferred terms as ‘definitely’, ‘probably’, ‘probably not’ or ‘definitely not’ associated with the syndromes. As an example, the physician judged the MedDRA-preferred terms ‘dizziness postural’ and ‘pain in extremity’ to be ‘definitely’ associated with POTS and CRPS, respectively. The physician was blinded to the allocation groups and outcome data. The data was synthesised for those MedDRA-preferred terms that the physician judged ‘definitely’ associated with POTS or CRPS. (Note that the synthesis of two or more different MedDRA-preferred term categories may include a participant more than once in an analysis.)

Data synthesis and analysis

Risk ratios were meta-analysed with the random-effects inverse variance method. As small trials carry more weight with this method, we compared random-effects to a fixed-effect risk ratio for all outcomes. Absolute risk estimates were calculated as the number needed to vaccinate (NNV) or harm (NNH). Review Manager 5 was used for data synthesis and the intention to treat principle to calculate effect estimates. Sensitivity and subgroup analyses were conducted to investigate potential sources of heterogeneity by taking account of age, gender, risk of bias [25] and type of HPV vaccine and comparator.

Results

Characteristics of included trials

We identified 50 eligible studies: 43 industry trials, 5 industry follow-up studies and 2 non-industry trials (see Fig. 1). We obtained 24 clinical study reports of 58,412 pages from EMA and GlaxoSmithKline for 22 industry trials and 2 industry follow-up studies (17 Cervarix™, 5 Gardasil™, 1 Gardasil 9™ and 1 monovalent Merck HPV type 16 vaccine) with a total of 95,670 participants (79,102 females and 16,568 males age 8–72) and 393,194 person-years (see Tables 2 and 3 and Additional file 2). The 24 clinical study reports included 79% (95,670/121,441) of the total eligible sample of the 50 identified eligible studies. It is possible that for some of these eligible studies clinical study reports were never written but journal articles were published. The mean follow-up time was 49 months (weighted by sample size). About two fifths of the participants in the control groups received the aluminium-based adjuvants that were used in the HPV vaccines (18,192/48,595), three fifths received hepatitis vaccines that also contained the aluminium-based adjuvants that were used in the HPV vaccines—except for the hepatitis vaccine Aimmugen™—(29,500), and less than a thousand participants received carrier solution (597) or saline placebo (306).

Table 2 Benefits and harms of the HPV vaccines: number of pages obtained of clinical study reports from the European Medicines Agency and GlaxoSmithKline

Table 3 Benefits and harms of the HPV vaccines: characteristics of included participants

Characteristics of potentially eligible studies

For the 26 remaining and potentially eligible studies (23 trials and three follow-up studies) for which no clinical study reports were obtained (or similar reports for the two non-industry trials), numbers of participants were identified for 20 of the 23 industry and 1 of the 2 non-industry trials. The trials included 25,632 and 139 participants, respectively, which were equal to 21% of the total eligible sample (25,771/121,441). These studies were not included in the review or analyses (see Additional file 3).

Risk of bias of included trials

All 22 trials and the 2 follow-up studies were at low risk of bias for ‘sequence generation’ and ‘allocation concealment’, and the majority were at low risk of bias for ‘blinding of outcome assessors’ (19/24) and ‘blinding of participants and personnel’ (16/24; see Figs. 2 and 3 and Additional file 2). However, due to the following reasons, we judged all studies to be at high risk of bias. Nearly all control participants (48,289/48,595, 99%) received an active comparator such as HPV vaccine aluminium-containing adjuvants or hepatitis vaccines. This distorted—to an unknown extent—the assessment of harms, as the trials tested an HPV vaccine vs. an active part of the same HPV vaccine (see reference [38] for additional clarification). Furthermore, serious harms were incompletely reported for 72% of the participants (68,610/95,670; see Table 1 and Additional file 2). All 24 clinical study reports contained redactions—especially of harms—and lacked significant parts such as serious harm narratives and case report forms (except for two reports: HPV-001 and HPV-008, which, however, included less than half of the participants’ case report forms) [38]. These situations are not covered by Cochrane’s risk of bias tool version 2011. Although not related to participant attrition, we judged the lack of serious harm narratives and case report forms as high risk of ‘incomplete outcome data’. In addition, while not related to the availability of study protocols, we judged the redactions of the clinical study reports as high risk of ‘selective outcome reporting’. We decided to conduct meta-analyses, since the high risk of bias mainly constituted situations that to our knowledge are not related to empirically verified bias mechanisms.

Fig. 2

Benefits and harms of the HPV vaccines: risk of bias graph

Fig. 3

Benefits and harms of the HPV vaccines: risk of bias summary (each study is noted as “manufacturer ID: type of HPV vaccine vs. type of comparator (included gender, age group; months of follow-up)”, e.g. “HPV-001: Cervarix vs. Al(OH)₃ (f, 15-26; 27)”)

Benefits

Seven clinical study reports assessed histological outcomes of which four reported HPV-related cancer outcomes irrespective of involved HPV types. At 4 years follow-up, the HPV vaccines did not decrease HPV-related cancer (7 in the HPV vaccine groups vs. 3 in the comparator groups, risk ratio [RR] 1.68 [95% confidence interval, CI, 0.51 to 5.49], P = 0.39, I² = 0%) or deaths hereof (2 vs. 1, RR 1.44 [95% CI 0.23 to 9.12], P = 0.70, I² = 0%); whereas they decreased HPV-related carcinoma in situ (367 vs. 490, RR 0.73 [95% CI 0.53 to 1.00], number needed to vaccinate [NNV] 387, P = 0.05, I² = 67%) and the composite surrogate outcome of HPV-related moderate intraepithelial neoplasia or worse (952 vs. 1239, RR 0.78 [95% CI 0.66 to 0.91], NNV 190, P = 0.002, I² = 53%). The HPV vaccines also decreased HPV-related external genital lesions (289 vs. 582, RR 0.56 [95% CI 0.39 to 0.82], NNV 47, P = 0.003, I² = 83%) and HPV-related treatment procedures such as cervical conisations (1018 vs. 1416, RR 0.71 [95% CI 0.63 to 0.80], NNV 75, P < 0.00001, I² = 45%) (see Table 4 and Additional file 4).

Table 4 Benefits and harms of the HPV vaccines: summary of HPV-related outcomes

Harms

Serious harms

The HPV vaccines did not significantly increase fatal harms (45 vs. 38, RR 1.19 [95% CI 0.65 to 2.19], P = 0.58, I² = 30%) or serious harms (1404 vs. 1357, RR 1.01 [95% CI 0.94 to 1.08], P = 0.79, I² = 0%), and no individual fatal or serious harm classified with a MedDRA-preferred term was significantly increased or decreased by the HPV vaccines (see Table 5 and Additional file 4).

Table 5 Benefits and harms of the HPV vaccines: summary of fatal and serious harms

New onset diseases

The HPV vaccines increased new onset back pain (397 vs. 336, RR 1.15 [95% CI 1.00 to 1.33], NNH 589, P = 0.05, I² = 0%) but decreased new onset gynaecological chlamydia infection (1409 vs. 1512, RR 0.93 [95% CI 0.87 to 1.00], NNV 176, P = 0.05, I² = 0%) and vaginal infection (369 vs. 420, 0.87 [95% CI 0.76 to 1.00], NNV 150, P = 0.05, I² = 0%) (see Table 6 and Additional file 4).

Table 6 Benefits and harms of the HPV vaccines: summary of new onset diseases

General harms

The HPV vaccines increased general harms (13,248 vs. 12,394, RR 1.07 [95% CI 1.03 to 1.11], NNH 51, P = 0.0002, I² = 77%)—especially myalgia (3989 vs. 3047, RR 1.41 [95% CI 1.24 to 1.60], NNH 26, P < 0.00001, I² = 80%), fatigue (4933 vs. 4489, RR 1.13 [95% CI 1.08 to 1.18], NNH 67, P < 0.00001, I² = 22%) and headache (5561 vs. 5246, RR 1.06 [95% CI 1.02 to 1.11], NNH 83, P = 0.009, I² = 40%) (see Table 7 and Additional file 4).

Table 7 Benefits and harms of the HPV vaccines: summary of general harms

Exploratory harm analyses

The HPV vaccines increased serious nervous system disorders grouped in the MedDRA system organ class (72 vs. 46, RR 1.49 [95% CI 1.02 to 2.16], number needed to harm [NNH] 1325, P = 0.04, I² = 0%) but decreased new onset vascular disorders grouped in the MedDRA system organ class (234 vs. 294, RR 0.80 [95% CI 0.67 to 0.94], NNV 439, P = 0.009, I² = 0%) (see Table 8 and Additional file 4).

Table 8 Benefits and harms of the HPV vaccines: summary of exploratory harm analyses by MedDRA system organ class

Harms of special interest

Cases of anaphylaxis and syncope were evenly distributed. There were no cases of chronic fatigue syndrome (CFS), complex regional pain syndrome (CRPS), Guillain-Barré syndrome (GBS) or postural orthostatic tachycardia syndrome (POTS), but there was one case of premature ovarian failure (POF) in the HPV vaccine group (see Table 9 and Additional file 4).

Table 9 Benefits and harms of the HPV vaccines: summary of harms of special interest and post hoc exploratory harm analyses

Post hoc exploratory harm analyses of special interest

The data from the included clinical study reports that corresponded to the three largest harm clusters reported from pharmacovigilance were associated with general harms, but not serious harms or new onset diseases. The serious harms that were judged ‘definitely associated’ with POTS or CRPS by the blinded physician were increased by the HPV vaccines, both for POTS (56 vs. 26, RR 1.92 [95% CI 1.21 to 3.07], NNH 1073, P = 0.006, I² = 0%) and CRPS (95 vs. 57, RR 1.54 [95% CI 1.11 to 2.14], NNH 906, P = 0.010, I² = 0%). The new onset diseases that were judged ‘definitely associated’ with POTS were also increased by the HPV vaccines (3675 vs. 3352, RR 1.08 [95% CI 1.01 to 1.15], NNH 144, P = 0.03, I² = 29%) (see Table 9 and Additional file 4).

Subgroup analyses

Younger HPV vaccinated participants were more protected against moderate HPV-related intraepithelial neoplasia or worse than older participants (age 15 to 29: 784 vs. 1079, RR 0.71 [95% CI 0.61 to 0.83]; age 21 to 72: 168 vs. 160, RR 1.04 [95% CI 0.84 to 1.29]; ratio of relative risk [RRR] 1.46 [1.12 to 1.91]) and also experienced fewer fatal harms than older participants (age 15 to 27: 24 vs. 32, RR 0.77 [95% CI 0.45, 1.33]; age 21 to 72: 21 vs. 6, RR 3.13 [95% CI 1.29 to 7.61]; RRR 0.25 [95% CI 0.09 to 0.70]), but there were no differences for serious nervous system disorders (age 10 to 35: 53 vs. 35, RR 1.46 [95% CI 0.95 to 2.25]; age 21 to 72: 19 vs. 11, RR 1.56 [95% CI 0.75 to 3.25]; RRR 0.93 [95% CI 0.40 to 2.19]), serious harms that were judged ‘definitely associated’ with of CRPS (age 9 to 35: 76 vs. 48, RR 1.48 [95% CI 1.03 to 2.12]; age 21 to 72: 19 vs. 9, RR 2.11 [95% CI 0.67 to 6.69]; RRR 0.70 [95% CI 0.21 to 2.34]) or serious harms that were judged ‘definitely associated’ with POTS (age 12 to 35: 43 vs. 21, RR 1.86 [95% CI 1.10, 3.15]; age 21 to 72: 13 vs. 5, RR 2.22 [95 CI 0.76 to 6.47]; RRR 0.84 [95% CI 0.25 to 2.76]) (see Additional file 4; note that the subgroup analyses used overlapping age groups due to the different age groups included in the trials). No significant subgroup differences were identified for subgroup analyses based on gender and control treatment.

Random-effects vs. fixed-effect

We found similar results with the fixed-effect model but with narrower confidence intervals, as the between-trial variance is not included in this model.

Discussion

Our systematic review of 24 clinical study reports with 95,670 participants showed that the HPV vaccines within 4 years of follow-up decreased HPV-related carcinoma in situ, which have a high likelihood of progressing to cancer [1], and HPV-related treatment procedures, but the vaccines also increased serious nervous system disorders (exploratory analysis) and general harms. Younger participants who are those primarily intended to receive HPV vaccination [1] were more protected against HPV-related neoplasia and had fewer fatal harms.

Strengths

Our review was based on study programmes, randomised trials reported in clinical study reports, clinically important pre-specified outcomes, intention to treat analyses, absolute risk estimates and a conservative statistical method based on the random-effects model. There was no heterogeneity for serious nervous system disorders or for the post hoc exploratory harm analyses of serious signs and symptoms judged ‘definitely associated’ with POTS and CRPS by a blinded physician with clinical expertise.

Limitations

Insufficient trial data access, incomplete reporting, data fragmentation and limited trial follow-up periods were major limitations. It took 3 years to obtain an incomplete subset of the eligible clinical study reports; a process which we have documented in detail elsewhere [38]. Our review is therefore limited by reporting bias—the bias that we aimed to reduce [37]. We did not obtain any periodical safety update reports before our data lock. The inclusion of the remaining participants from the 26 studies with no available clinical study reports included a fifth of the total eligible participants, which could have influenced our review, as some of our results had P values around our cut-off of 0.05 and confidence intervals that were wide.

We performed multiple comparisons: 166 meta-analyses of which 31 (19%) showed statistical significance for the total risk ratio estimate. With our P value cut-off of 0.05, about eight (166*0.05) or a fourth (8/31) of the significant results are likely to have occurred by chance. We did not use Bonferroni (or similar) corrections [40], as one of our primary outcomes was serious harms, which were affected by incomplete reporting (see Table 1) and lack of saline placebo controls (see Additional file 2).

The 24 included clinical study reports only included one Gardasil 9 trial (V503–006) that was small and did not investigate histological outcomes. Many countries are currently implementing Gardasil 9 as a two-dose regimen in their vaccination programme instead of Cervarix or Gardasil [1]. Two doses of Gardasil 9 may induce fewer harms than three doses, but Gardasil 9 may induce more harms than Gardasil. For example, in the clinical study report that we obtained of phase 3 multicentre trial V503-001/NCT00543543 (not eligible for our systematic review) of 7106 and 7109 healthy females age 16–26 randomised to receive three doses Gardasil 9 or Gardasil, there were more serious harms (233 vs. 183, RR 1.27 [95% CI 1.05 to 1.54], NNH 151, P = 0.010; reported from day 0 to 390) and general harms (‘systemic adverse events’: 2086 vs. 1929, RR 1.08 [95% CI 1.03 to 1.14], NNH 75, P = 0.003; reported 0–14 days post-vaccination) in the Gardasil 9 group. A 0.5-ml dose of Gardasil 9 contains more virus-like particles (270 μg vs. 100 μg) and aluminium-containing adjuvant (500 μg vs. 225 μg) compared to a 0.5-ml dose of Gardasil, which could explain the harm differences. Although Gardasil 9 targets five more HPV types than Gardasil, Gardasil 9 did not decrease CIN2⁺ more than Gardasil during trial V503-001’s 42-month follow-up (325 vs. 326, RR 1.00 [95% CI 0.86 to 1.16], P = 0.97).

A substantial part of our results should be interpreted carefully due to high heterogeneity. We expected the high heterogeneity for several results (e.g. for HPV-related carcinoma in situ), as the included trials comprised 16 different subgroups—based on the type of HPV vaccine, comparator, age and gender. All meta-analyses were divided according to the 16 subgroups to provide heterogeneity measures (see Additional file 4), but the nationality of the participants and regional practices of HPV-related screening and treatment procedures may also have contributed to the heterogeneity.

Limitations of benefit assessment

Only 10 HPV-related cancers occurred in the follow-up periods. Extended follow-up was not possible for 75% of the comparator participants (36,344/48,595), as they were offered HPV vaccination at trial completion.

We only included benefit results of intention to treat analyses, which also included participants that were enrolled after they had been infected with HPV. The HPV vaccines have no documented effect on HPV-related neoplasia caused by previous infections [1]. Our benefit results may therefore be skewed toward the null compared to real-life settings where mainly 12-year-old adolescents—that are expected to not be previously HPV-infected—are HPV vaccinated. Getting vaccinated before sexual debut is likely to improve the HPV vaccines’ benefits, but no included trial investigated histological outcomes for participants that were vaccinated under the age of 15.

Three trials—HPV-008, V501-013 and V501-015 that contained 38% (36,266/95,670) of the analysed participants—were stopped early when HPV type 16/18-related cervical intraepithelial neoplasia or worse (CIN2⁺) was significantly reduced for their HPV vaccine per-protocol populations. On average, trials stopped early for benefits exaggerate effects by 29% compared to completed trials of the same intervention [41]. When the three trials were excluded from our CIN2⁺ meta-analysis, CIN2⁺ was not significantly decreased (184 vs. 200, RR 0.85 [95% CI 0.54 to 1.33], P = 0.47, I² = 77%; see Additional file 4).

One clinical study report (HPV-015) only reported CIN2⁺, although there were three cases of HPV-related cancers in the HPV vaccine group and one in the comparator group (see Additional file 4). These cancers were listed as serious harms and were not mentioned elsewhere in the clinical study report. For transparency, it would have been more appropriate to report each histological outcome (cancer, carcinoma in situ, moderate intraepithelial neoplasia, etc.) than only a composite surrogate outcome such as CIN2⁺.

No trial tested the HPV vaccines’ protection against cervical cancer without using cervical screening. This may reduce external validity, as some studies show that HPV-vaccinated women may tend to avoid cervical screening [42]; although other studies have not shown a clear tendency [43]. The trial personnel often performed cervical screening together with colposcopy every 6 months, and the included participants were often women aged 15–26. In clinical practice, cervical screening is usually performed every 3 to 5 years and recommended after age 25 [44], as most CIN2⁺ lesions in women under age 30 regress spontaneously, which may justify active surveillance rather than immediate intervention [45].

No trial used mandatory biopsies, which may reduce internal validity. For example, the precursor lesion of cervical adenocarcinoma is difficult to detect on colposcopy, but easier to recognise on a biopsy [46]. The incidence of cervical adenocarcinoma is increasing and may more often be HPV negative compared to cervical squamous carcinoma [46], but only 5% (40/857) of the reported cervical carcinoma in situ cases in the included studies were adenocarcinoma in situ (see Table 4).

We did not pre-specify genital warts as an outcome, but the HPV vaccines reduced external genital lesions and there is strong evidence that the HPV vaccines—especially Gardasil and Gardasil 9 that target the HPV types 6 and 11—decrease the incidence of genital warts [47].

Limitations of harm assessment

Only Merck clinical study reports reported aggregate numbers for participants with MedDRA system organ classified harms, and only for new onset diseases and general harms. The synthesis of MedDRA system organ classes for all GlaxoSmithKline clinical study reports and for serious harms for Merck clinical study reports may therefore include a participant more than once. As a result, we consider these analyses exploratory.

Serious harms were incompletely reported for 72% of the participants (68,610/95,670; see Table 1 and Additional file 2). There were 2.8 times more serious harms reported in the clinical study reports that reported serious harms for the whole trial period (1838/27,493 vs. 923/38,356). As an example, trial HPV-008 of Cervarix that had reported all serious harms during its 48 months follow-up reported 10 times more participants with serious harms compared to V501-015 of Gardasil that only reported serious harms 14 days post-vaccination (1664/18,644 vs. 102/12,167). In the cluster-randomised trial, HPV-040, 88% (28,473 of 32,176) of the participants were not included for serious harms reporting (see Table 1 and Additional file 2).

The use of active comparators may have underestimated harms related to the HPV vaccines [38]. The aluminium-containing comparators were used, as they provided a similar appearance to that of the HPV vaccines, which enhanced blinding and decreased the risk of performance and detection bias. A single trial—V503-006, of Gardasil 9—used a saline placebo in 306 participants who had previously been vaccinated with Gardasil. It is unlikely that those who had experienced harms following previous Gardasil vaccination would have participated in the Gardasil 9 trial, so the trial’s harm results are not reliable. The trial’s blinding procedure was adequate to ensure low risk of performance and detection bias and could have been used in other trials (see Additional file 2).

Although the manufacturers consider the aluminium-containing comparators to be safe, 52% of the participants (49,301/95,670) were only included in the trials if they had never received the aluminium-containing comparators before. GlaxoSmithKline state that their aluminium-containing comparator induces myalgia (“higher incidences of myalgia might namely be attributable to the higher content of aluminium in the HPV vaccine [450 micrograms Al(OH)₃] than the content of aluminium in the HAV [hepatitis A] vaccine [225 micrograms Al(OH)₃]” [48]), which we found was increased by the HPV vaccines (see Table 7).

The clinical study reports, their informed consent forms and corresponding journal publications (for example, V501-013 [49] and V501-015 [50]) often used the term placebo (which is a substance with no active effect) to describe the active aluminium-based comparators.

Two thirds of the participants (63,468/95,670) were only included in the trials if they had no history of immunological or nervous system disorders (see Additional file 2). Such disorders are not listed as warnings or contraindications on the package inserts of the approved HPV vaccines [8,9,10]. The degree of harms might therefore be higher in clinical practice than in the trials. The HPV vaccines did not increase the three largest HPV vaccine-related VigiBase® harms clusters for serious harms and new onset diseases (see the “Methods” section, Table 9 and Additional file 4), which may reflect the differences between real-life and the trials’ settings and entry criteria.

The exploratory analyses of MedDRA system organ classes may have included a participant more than once. For serious nervous system disorders, this is unlikely, as there were only 118 participants with such disorders (reported as individual MedDRA-preferred terms) for 61,331 participants (see Additional file 4). We note, however, that the serious nervous system disorders consisted of very heterogenous harms, for example, ‘anoxic encephalopathy’, ‘moyamoya disease’ and ‘vertebral artery dissection’.

The serious harm analyses of MedDRA-preferred terms associated with POTS and CRPS may also have included a participant more than once, although this is unlikely as there only were 82 participants with a POTS sign/symptom for 60,058 participants and 152 participants with a CRPS sign/symptom for 60,915 participants. The selection of MedDRA-preferred terms associated with POTS and CRPS was subjective, not verified by other assessors and included some signs/symptoms that do not align well with the diagnostic criteria of POTS or CRPS [51, 52], for example, ‘constipation’, ‘vision blurred’ and ‘vomiting’. Other blinded assessors would possibly assign MedDRA-preferred terms differently, as there were over 3000 different included MedDRA-preferred terms. The post hoc exploratory POTS and CRPS analyses were based on randomised trial data where serious harms were underreported and likely underestimated, but since no complete serious harm narratives or complete case report forms were available, the analyses could not take symptom duration, symptom clustering or the diagnostic criteria into account. Therefore, the analyses do not prove that the HPV vaccines cause POTS and CRPS, but they do provide a signal, which makes it important to carry out independent analyses of POTS and CRPS based on the complete data set with individual participant data.

Similar studies

In May 2018, a Cochrane review of the HPV vaccines that included 26 trials with 73,428 female participants concluded that the HPV vaccines decrease precursors to cervical cancer and do not increase serious or general harms [3]. The Cochrane review had similar inclusion criteria to our review, but it was mainly based on journal publications and only included phase II and III trials. In comparison, we identified 50 possibly eligible studies for which we obtained clinical study reports for 22 trials and two follow-up studies and included 30% more participants (95,670) than the Cochrane review. We found that the HPV vaccines decrease precursors to HPV-related cancer and treatment procedures but increase serious nervous system disorders (exploratory analyses) and general harms. Another recent review on males [53] and most large epidemiological studies have found no serious harms associated with the HPV vaccines [16,17,18,19,20].

Conclusion

At 4 years follow-up, the HPV vaccines decreased HPV-related precursors to cervical cancer and treatment procedures but increased serious nervous system disorders (exploratory analysis) and general harms. As the included trials were primarily designed to assess benefits and not adequately designed to assess harms, the extent to which the benefits outweigh the harms is unclear. Limited access to clinical study reports and trial data with case report forms prevented a thorough assessment. An independent assessment of the complete individual participant data is needed. If granted access to the complete data set with individual participant data, we will update this systematic review. A large industry-independent multicentre trial of two doses of Gardasil 9 vs. saline placebo would likely be informative in identifying a more accurate benefit-harm balance, but we recognise that such a trial will be considered unethical in most settings.