Standards
The methods described by the Office of Health Assessment and Translation (OHAT) will be followed to conduct this review [41]. Our reporting adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis protocols (PRISMA-P) statement [43] and Meta-analysis of Observational Studies in Epidemiology (MOOSE) statements [44] (see Additional file 1).
Protocol registration number
This protocol is registered in the International Prospective Register of Systematic Reviews (PROSPERO CRD42016050861).
Eligibility criteria
As we will summarize the evidence and assess its certainty separately for bodies of evidence from observational, experimental, and mechanistic studies, we decided to present the eligibility criteria according to the type of evidence. The following criteria will be applied for observational studies:
-
Type of study: Case–control, cohort, and cross-sectional studies.
-
Participants: Any adult (≥18 years) identified as occupationally and/or environmentally exposed to paraquat, regardless gender.
-
Exposure: Any period, frequency and amount of exposure to paraquat.
-
Controls: Participants that were not exposed to paraquat.
-
Primary outcomes: PD diagnosed by experts through clinical assessments and/or differential diagnosis.
-
Secondary outcomes: Signs and symptoms of parkinsonism, such as tremor, bradykinesia, rigidity, postural instability, mobility, activities of daily living, emotional well-being, and cognition functions reported by the participants in interviews and/or questionnaires.
The following criteria will be applied for experimental animal studies:
-
Type of study: Any study using laboratory rodents designed to assess endpoints of paraquat neurotoxicity. Studies evaluating the neuroprotective effect of a given substance in response to paraquat will not be considered.
-
Animals: Rats and mice with no restrictions as to strain, gender, age, and life stage at paraquat exposure or outcome assessments.
-
Exposure: Only multiple-dose non-acute studies will be considered. There will be no restrictions regarding dose or exposure route. No mixtures will be allowed.
-
Controls: Animals not exposed to paraquat or any other chemical. The doses used in each study will also be considered mutual controls (dose–response).
-
Primary outcomes: Morphological alterations (reduction in the average volume of SNpc, dead dopaminergic neurons in the nigrostriatal pathway and/or intracytoplasmic presence of Lewy bodies) and biochemical alterations (reduction in the levels of dopamine and its metabolites in the striatum).
-
Secondary outcomes: Neurobehavioral alterations (low motor activity, low sensorimotor reflexes, effects as to cognitive function) and detection of reactive oxygen species—ROS—in dopaminergic neurons.
The following criteria will be applied for mechanistic studies:
-
Type of study: In vivo studies with rodents and in vitro studies designed to assess cellular, biochemical, and/or molecular mechanisms of paraquat neurotoxicity.
-
Animals (for in vivo studies): Rats and mice with no restrictions as to strain, gender, age, and life stage at exposure or outcome assessments.
-
Cell line (for in vitro studies): Cell lines such as SK-N-SH, SH-SY5Y, PC12, RBE, astrocytes, dopaminergic neurons, or other cell lines used in in vitro models for PD.
-
Controls (for in vitro and in vivo studies): Systems not exposed to paraquat. The doses used in each study will also be considered mutual controls (dose–response).
-
Exposure (for in vitro and in vivo studies): For in vivo studies, criteria adopted for experimental rodent studies will be applied. For in vitro studies, the criteria adopted for conventional in vitro studies will be applied, for example, doses should not reach cytotoxicity levels. No mixtures will be allowed.
-
Outcomes (for in vitro and in vivo studies): Cellular, biochemical, and/or molecular pathways to death of dopaminergic neurons, such as oxidative stress, mitochondrial dysfunction, intracytoplasmic presence of Lewy bodies in dopaminergic neurons, and/or other key molecular initiating events related to parkinsonism.
Search methods for primary studies
Electronic searches
Peer-reviewed original studies published in the following electronic databases will be searched: PubMed (1966 to present), EMBASE (1980 to present), Web of Science (1990 to present), Toxnet (2007 to present), and LILACS (1982 to present), without language, year, and status of publication restrictions.
Search strategy
MeSH terms and free terms related to “Parkinson’s Disease,” “herbicides,” and “paraquat” will be combined. The search strategy will be adapted for each database (Table 1).
References listed in the selected studies will be analyzed for additional citations. Studies’ main authors will eventually be contacted for potentially missing data.
Searching other resources
Screening of the “grey literature” will also be performed to identify publications not commercially published or not readily available to the public. Examples of “grey literature” include technical reports from scientific research groups or government agencies, such as US Environmental Protection Agency (EPA), Food and Drug Administration (FDA), Agency for Toxic Substances and Disease Registry (ATSDR), and National Institute for Occupational Safety and Health (NIOSH).
Eligibility determination
Two reviewers (CV and JLVC) will independently screen all titles and abstracts identified by the literature search, obtain and read full-text articles of all potentially eligible studies, and evaluate them for eligibility. Disagreements will be resolved by consensus and, eventually, through consultation with technical advisors to improve accuracy and consistency among screeners.
Study flow diagram
A PRISMA flow diagram will be produced to indicate the number of included and excluded studies and the corresponding reasons for exclusion.
Data extraction
Reviewers will undergo calibration exercises and work in pairs to independently extract data from included studies. A standard form will be used to extract information from the included observational studies, such as study design, study characteristics (e.g., sample sizes, number of cases, variables for which controls were matched to cases, study population, and confounding factors), type of exposure (i.e., occupational, environmental, or both), details on paraquat exposure (e.g., source, duration, frequency, intensity), methods of exposure measurement (e.g., questionnaires/interviews or fluid analysis such as blood and urine for characterization of possible internal exposure), criteria used for PD diagnosis, effect estimate value or description of qualitative results, etc.
Data extracted from experimental animal studies will include experimental design, guideline compliance (e.g., use of EPA (Environmental Protection Agency), OECD (Organization for Economic Co-operation and Development), NTP (National Toxicology Program), or another guideline for study design), animal characteristics (specie, strain, gender, genetic background, number of groups, number of animals per group, age, or life stage at start of dosing and at health outcome assessment), paraquat characteristics (supplier, catalog number, and purity), treatment (doses, frequency, eventual information on internal dosimetry), vehicle used, routes of administration, protocol (randomization procedure, evaluated outcome(s), methods for assessing outcomes(s), blinding during outcome assessment, statistical methods), results (measures of effect at each dose or concentration level (e.g., mean, median, frequency, and measures of precision or variance) or description of qualitative results, no observed effect level (NOEL), lowest observed effect level (LOEL), benchmark dose (BMD) analysis), etc.
For in vitro studies, the following information will be extracted: cell/tissue model (cell line, cell type or tissue, source of cells/tissue,sex of human/animal of origin, specie, strain), name and source of assay kits, treatment (concentration levels, as presented and converted to micromolar when possible, duration, and frequency of dosing), protocol description (number of replicates per group, randomization procedure), outcome (outcome(s) assessed, methods for assessment, blinding during outcome assessment), results (NOEC (no observed effect concentration), LOEC (lowest observed effect concentration), IC50 (inhibitory concentration 50), statistical methods, etc.
Assessment of internal validity of individual studies
Reviewers, working in pairs, will independently assess the risk of bias of each human or animal study; a specific protocol to address the risk of bias of mechanistic studies is still in development. Possible sources of bias will be assessed through 11 pre-defined questions considering (1) potential confounders, (2) confidence in exposure characterization, and (3) confidence in the outcome assessment [45]. The answer to each question is assigned to one of the four following categories of risk of bias: definitely low, probably low, probably high, or definitely high. After assigning each response to one of the categories, we will also use a three-tier system to classify the studies regarding their overall methodological quality. Studies considered at definitely or probably low risk of bias will be grouped as tier 1 studies. Tier 3 studies refer to the ones considered to be at definitely or probably high risk of bias; tier 2 will encompass studies that did not meet the criteria for tier 1 and tier 3 and therefore fall into an in-between category [41].
Data synthesis and statistical analysis
Meta-analysis of results will be considered by investigating heterogeneity among animal and human studies, separately. It is common, though, that environmental health studies have some differences regarding outcome assessments and exposure definitions that could be an obstacle to formal statistical meta-analysis [26]. The heterogeneity associated with pooled effect estimates will be assessed with the use of a χ
2 test and the I
2 statistic (I
2 = [(Q − df)/Q] × 100%) [46, 47]. Heterogeneity will be classified as follows: 0 to 40% (no important heterogeneity); 30 to 60% (moderate heterogeneity); 50 to 90% (substantial heterogeneity); and 75 to 100% (considerable heterogeneity).
Statistical analysis will be conducted using the Comprehensive Meta-Analysis STATA software (version 10.1). If considerable heterogeneity is detected among studies, meta-analysis will not be indicated and results will only be presented in tables or in a narrative synthesis. However, if heterogeneity does not exceed 75%, we will use random effects meta-analysis [48], which is a more conservative approach of pooling the results. In this case, the measure of association will be presented as odds ratios and mean difference, both with a 95% confidence interval (95% CI), for studies with dichotomous and continuous data, respectively [49].
Subgroup and sensitivity analysis
Subgroup analysis will be conducted to investigate possible heterogeneity causes and other risk factors that could be potential cofounders. Studies will be allocated into groups according to common characteristics, as described below. The outcomes are assessed to determine if exposure caused a significant effect due to a specific feature. For human studies, subgroup analyses will be conducted for different exposure periods (<5 years vs. ≥5 years), co-exposures to other pesticides also related to PD (e.g., paraquat plus rotenone vs. paraquat plus maneb vs. only paraquat), different comorbidities (e.g., smoking vs. alcohol vs. diabetes), different age groups (e.g., <60 years of age vs. ≥60 years), and presence or absence of family history of PD.
For experimental animal studies, subgroup analyses will be conducted for dose (e.g., higher vs. lower), duration of paraquat exposure in multidose studies, and routes of administration (e.g., oral vs. dermal vs. inhalation vs. intraperitoneal vs. intravenous).
If there are enough studies, we will develop sensitivity analyses to explore robustness of the results for each type of study design “e.g., observational studies (cohort, case–control, cross-sectional) vs. experimental animal studies, cohort vs. case–control vs. cross-sectional” and studies with low risk of bias vs. those with high risk of bias.
Certainty of evidence
The certainty of the evidence for each outcome will also be rated at high, moderate, low, or very low, using an adapted version of the Grading of Recommendations Assessment, Development and Evaluation system (GRADE) [41]. To date, this adjusted protocol was only applied to human and animal studies that evaluate toxicological outcomes and is not yet used to evaluate mechanistic studies.
Available studies on a particular outcome will be initially grouped by key study design features that guarantee that the paraquat exposure preceded and was associated with the development of the signs and symptoms of parkinsonism [41]. Each grouping of studies will be given an initial certainty rating depending on the presence of those features.
In this adaptation of GRADE, experimental studies begin as “high certainty” of the evidence, cohort studies as “moderate certainty,” case–control studies as “moderate or low certainty,” and cross-sectional studies as “low certainty” of the evidence. This initial rating could be downgraded by the presence of risk of bias, imprecision, unexplained inconsistency, indirectness, and publication bias [41]. If there is a minimum of ten studies included in the systematic review, funnel plots might be used to assess the possibility of publication biases [50]. Egger’s regression test and “trim and fill” techniques might also be used to visualize asymmetrical or symmetrical patterns of study results [51].
On the other hand, the presence of a large magnitude of effect, a dose–response gradient, recognition of confounding factors that could underestimate or overestimate the effect and consistency among studies with different designs could upgrade the certainty of the evidence [41]. After the evaluation of each study group for each outcome, only the ones with the highest level of certainty will be translated into the respective level of evidence, as indicated below.
Translating certainty ratings into levels of evidence for parkinsonian-like effects
Five descriptors will be used to rate the level of evidence: “high,” “moderate,” “low,” “inadequate evidence,” and “evidence of no health effect.” The first three descriptors (“high,” “moderate,” and “low” level of confidence) used in the previous step to indicate the certainty of the evidence will be directly converted into levels of evidence. However, if the level of certainty is “very low” or no evidence is identified, the level of evidence will be considered “inadequate” [41].
The descriptor “evidence of no health effect” that indicates that paraquat is not related to PD in humans or in rodent models will be considered only when the level of certainty is high [41].
Integrated evidence for paraquat hazard identification
For proper categorization of paraquat in one of the five categories of hazard (known, suspected, presumed, not classifiable, or not identified to be hazardous to humans) [41], evidence coming from human and animal studies will be integrated with mechanistic data that may be relevant to support biological plausibility and increase or decrease the hazard classification.
Among the factors that can support biological plausibility and increase the hazard classification, the magnitude of the effect, the dose–response gradient, and the direct and indirect consistencies between outcomes from studies with different biological levels (human, rodents, and in vitro) will be considered. On the contrary, hazard classification may be reduced by the identification of risk of bias, unexplained inconsistencies between studies with related outcomes, non-relevance of the paraquat mechanism of toxicity to humans and dose levels not relevant to real human exposure.