Our systematic review of 24 clinical study reports with 95,670 participants showed that the HPV vaccines within 4 years of follow-up decreased HPV-related carcinoma in situ, which have a high likelihood of progressing to cancer , and HPV-related treatment procedures, but the vaccines also increased serious nervous system disorders (exploratory analysis) and general harms. Younger participants who are those primarily intended to receive HPV vaccination  were more protected against HPV-related neoplasia and had fewer fatal harms.
Our review was based on study programmes, randomised trials reported in clinical study reports, clinically important pre-specified outcomes, intention to treat analyses, absolute risk estimates and a conservative statistical method based on the random-effects model. There was no heterogeneity for serious nervous system disorders or for the post hoc exploratory harm analyses of serious signs and symptoms judged ‘definitely associated’ with POTS and CRPS by a blinded physician with clinical expertise.
Insufficient trial data access, incomplete reporting, data fragmentation and limited trial follow-up periods were major limitations. It took 3 years to obtain an incomplete subset of the eligible clinical study reports; a process which we have documented in detail elsewhere . Our review is therefore limited by reporting bias—the bias that we aimed to reduce . We did not obtain any periodical safety update reports before our data lock. The inclusion of the remaining participants from the 26 studies with no available clinical study reports included a fifth of the total eligible participants, which could have influenced our review, as some of our results had P values around our cut-off of 0.05 and confidence intervals that were wide.
We performed multiple comparisons: 166 meta-analyses of which 31 (19%) showed statistical significance for the total risk ratio estimate. With our P value cut-off of 0.05, about eight (166*0.05) or a fourth (8/31) of the significant results are likely to have occurred by chance. We did not use Bonferroni (or similar) corrections , as one of our primary outcomes was serious harms, which were affected by incomplete reporting (see Table 1) and lack of saline placebo controls (see Additional file 2).
The 24 included clinical study reports only included one Gardasil 9 trial (V503–006) that was small and did not investigate histological outcomes. Many countries are currently implementing Gardasil 9 as a two-dose regimen in their vaccination programme instead of Cervarix or Gardasil . Two doses of Gardasil 9 may induce fewer harms than three doses, but Gardasil 9 may induce more harms than Gardasil. For example, in the clinical study report that we obtained of phase 3 multicentre trial V503-001/NCT00543543 (not eligible for our systematic review) of 7106 and 7109 healthy females age 16–26 randomised to receive three doses Gardasil 9 or Gardasil, there were more serious harms (233 vs. 183, RR 1.27 [95% CI 1.05 to 1.54], NNH 151, P = 0.010; reported from day 0 to 390) and general harms (‘systemic adverse events’: 2086 vs. 1929, RR 1.08 [95% CI 1.03 to 1.14], NNH 75, P = 0.003; reported 0–14 days post-vaccination) in the Gardasil 9 group. A 0.5-ml dose of Gardasil 9 contains more virus-like particles (270 μg vs. 100 μg) and aluminium-containing adjuvant (500 μg vs. 225 μg) compared to a 0.5-ml dose of Gardasil, which could explain the harm differences. Although Gardasil 9 targets five more HPV types than Gardasil, Gardasil 9 did not decrease CIN2+ more than Gardasil during trial V503-001’s 42-month follow-up (325 vs. 326, RR 1.00 [95% CI 0.86 to 1.16], P = 0.97).
A substantial part of our results should be interpreted carefully due to high heterogeneity. We expected the high heterogeneity for several results (e.g. for HPV-related carcinoma in situ), as the included trials comprised 16 different subgroups—based on the type of HPV vaccine, comparator, age and gender. All meta-analyses were divided according to the 16 subgroups to provide heterogeneity measures (see Additional file 4), but the nationality of the participants and regional practices of HPV-related screening and treatment procedures may also have contributed to the heterogeneity.
Limitations of benefit assessment
Only 10 HPV-related cancers occurred in the follow-up periods. Extended follow-up was not possible for 75% of the comparator participants (36,344/48,595), as they were offered HPV vaccination at trial completion.
We only included benefit results of intention to treat analyses, which also included participants that were enrolled after they had been infected with HPV. The HPV vaccines have no documented effect on HPV-related neoplasia caused by previous infections . Our benefit results may therefore be skewed toward the null compared to real-life settings where mainly 12-year-old adolescents—that are expected to not be previously HPV-infected—are HPV vaccinated. Getting vaccinated before sexual debut is likely to improve the HPV vaccines’ benefits, but no included trial investigated histological outcomes for participants that were vaccinated under the age of 15.
Three trials—HPV-008, V501-013 and V501-015 that contained 38% (36,266/95,670) of the analysed participants—were stopped early when HPV type 16/18-related cervical intraepithelial neoplasia or worse (CIN2+) was significantly reduced for their HPV vaccine per-protocol populations. On average, trials stopped early for benefits exaggerate effects by 29% compared to completed trials of the same intervention . When the three trials were excluded from our CIN2+ meta-analysis, CIN2+ was not significantly decreased (184 vs. 200, RR 0.85 [95% CI 0.54 to 1.33], P = 0.47, I2 = 77%; see Additional file 4).
One clinical study report (HPV-015) only reported CIN2+, although there were three cases of HPV-related cancers in the HPV vaccine group and one in the comparator group (see Additional file 4). These cancers were listed as serious harms and were not mentioned elsewhere in the clinical study report. For transparency, it would have been more appropriate to report each histological outcome (cancer, carcinoma in situ, moderate intraepithelial neoplasia, etc.) than only a composite surrogate outcome such as CIN2+.
No trial tested the HPV vaccines’ protection against cervical cancer without using cervical screening. This may reduce external validity, as some studies show that HPV-vaccinated women may tend to avoid cervical screening ; although other studies have not shown a clear tendency . The trial personnel often performed cervical screening together with colposcopy every 6 months, and the included participants were often women aged 15–26. In clinical practice, cervical screening is usually performed every 3 to 5 years and recommended after age 25 , as most CIN2+ lesions in women under age 30 regress spontaneously, which may justify active surveillance rather than immediate intervention .
No trial used mandatory biopsies, which may reduce internal validity. For example, the precursor lesion of cervical adenocarcinoma is difficult to detect on colposcopy, but easier to recognise on a biopsy . The incidence of cervical adenocarcinoma is increasing and may more often be HPV negative compared to cervical squamous carcinoma , but only 5% (40/857) of the reported cervical carcinoma in situ cases in the included studies were adenocarcinoma in situ (see Table 4).
We did not pre-specify genital warts as an outcome, but the HPV vaccines reduced external genital lesions and there is strong evidence that the HPV vaccines—especially Gardasil and Gardasil 9 that target the HPV types 6 and 11—decrease the incidence of genital warts .
Limitations of harm assessment
Only Merck clinical study reports reported aggregate numbers for participants with MedDRA system organ classified harms, and only for new onset diseases and general harms. The synthesis of MedDRA system organ classes for all GlaxoSmithKline clinical study reports and for serious harms for Merck clinical study reports may therefore include a participant more than once. As a result, we consider these analyses exploratory.
Serious harms were incompletely reported for 72% of the participants (68,610/95,670; see Table 1 and Additional file 2). There were 2.8 times more serious harms reported in the clinical study reports that reported serious harms for the whole trial period (1838/27,493 vs. 923/38,356). As an example, trial HPV-008 of Cervarix that had reported all serious harms during its 48 months follow-up reported 10 times more participants with serious harms compared to V501-015 of Gardasil that only reported serious harms 14 days post-vaccination (1664/18,644 vs. 102/12,167). In the cluster-randomised trial, HPV-040, 88% (28,473 of 32,176) of the participants were not included for serious harms reporting (see Table 1 and Additional file 2).
The use of active comparators may have underestimated harms related to the HPV vaccines . The aluminium-containing comparators were used, as they provided a similar appearance to that of the HPV vaccines, which enhanced blinding and decreased the risk of performance and detection bias. A single trial—V503-006, of Gardasil 9—used a saline placebo in 306 participants who had previously been vaccinated with Gardasil. It is unlikely that those who had experienced harms following previous Gardasil vaccination would have participated in the Gardasil 9 trial, so the trial’s harm results are not reliable. The trial’s blinding procedure was adequate to ensure low risk of performance and detection bias and could have been used in other trials (see Additional file 2).
Although the manufacturers consider the aluminium-containing comparators to be safe, 52% of the participants (49,301/95,670) were only included in the trials if they had never received the aluminium-containing comparators before. GlaxoSmithKline state that their aluminium-containing comparator induces myalgia (“higher incidences of myalgia might namely be attributable to the higher content of aluminium in the HPV vaccine [450 micrograms Al(OH)3] than the content of aluminium in the HAV [hepatitis A] vaccine [225 micrograms Al(OH)3]” ), which we found was increased by the HPV vaccines (see Table 7).
The clinical study reports, their informed consent forms and corresponding journal publications (for example, V501-013  and V501-015 ) often used the term placebo (which is a substance with no active effect) to describe the active aluminium-based comparators.
Two thirds of the participants (63,468/95,670) were only included in the trials if they had no history of immunological or nervous system disorders (see Additional file 2). Such disorders are not listed as warnings or contraindications on the package inserts of the approved HPV vaccines [8,9,10]. The degree of harms might therefore be higher in clinical practice than in the trials. The HPV vaccines did not increase the three largest HPV vaccine-related VigiBase® harms clusters for serious harms and new onset diseases (see the “Methods” section, Table 9 and Additional file 4), which may reflect the differences between real-life and the trials’ settings and entry criteria.
The exploratory analyses of MedDRA system organ classes may have included a participant more than once. For serious nervous system disorders, this is unlikely, as there were only 118 participants with such disorders (reported as individual MedDRA-preferred terms) for 61,331 participants (see Additional file 4). We note, however, that the serious nervous system disorders consisted of very heterogenous harms, for example, ‘anoxic encephalopathy’, ‘moyamoya disease’ and ‘vertebral artery dissection’.
The serious harm analyses of MedDRA-preferred terms associated with POTS and CRPS may also have included a participant more than once, although this is unlikely as there only were 82 participants with a POTS sign/symptom for 60,058 participants and 152 participants with a CRPS sign/symptom for 60,915 participants. The selection of MedDRA-preferred terms associated with POTS and CRPS was subjective, not verified by other assessors and included some signs/symptoms that do not align well with the diagnostic criteria of POTS or CRPS [51, 52], for example, ‘constipation’, ‘vision blurred’ and ‘vomiting’. Other blinded assessors would possibly assign MedDRA-preferred terms differently, as there were over 3000 different included MedDRA-preferred terms. The post hoc exploratory POTS and CRPS analyses were based on randomised trial data where serious harms were underreported and likely underestimated, but since no complete serious harm narratives or complete case report forms were available, the analyses could not take symptom duration, symptom clustering or the diagnostic criteria into account. Therefore, the analyses do not prove that the HPV vaccines cause POTS and CRPS, but they do provide a signal, which makes it important to carry out independent analyses of POTS and CRPS based on the complete data set with individual participant data.
In May 2018, a Cochrane review of the HPV vaccines that included 26 trials with 73,428 female participants concluded that the HPV vaccines decrease precursors to cervical cancer and do not increase serious or general harms . The Cochrane review had similar inclusion criteria to our review, but it was mainly based on journal publications and only included phase II and III trials. In comparison, we identified 50 possibly eligible studies for which we obtained clinical study reports for 22 trials and two follow-up studies and included 30% more participants (95,670) than the Cochrane review. We found that the HPV vaccines decrease precursors to HPV-related cancer and treatment procedures but increase serious nervous system disorders (exploratory analyses) and general harms. Another recent review on males  and most large epidemiological studies have found no serious harms associated with the HPV vaccines [16,17,18,19,20].