Skip to main content

Sex/gender reporting and analysis in Campbell and Cochrane systematic reviews: a cross-sectional methods study

Abstract

Background

The importance of sex and gender considerations in research is being increasingly recognized. Evidence indicates that sex and gender can influence intervention effectiveness. We assessed the extent to which sex/gender is reported and analyzed in Campbell and Cochrane systematic reviews.

Methods

We screened all the systematic reviews in the Campbell Library (n = 137) and a sample of systematic reviews from 2016 to 2017 in the Cochrane Library (n = 674). We documented the frequency of sex/gender terms used in each section of the reviews.

Results

We excluded 5 Cochrane reviews because they were withdrawn or published and updated within the same time period as well as 4 Campbell reviews and 114 Cochrane reviews which only included studies focused on a single sex. Our analysis includes 133 Campbell reviews and 555 Cochrane reviews. We assessed reporting of sex/gender considerations for each section of the systematic review (Abstract, Background, Methods, Results, Discussion). In the methods section, 83% of Cochrane reviews (95% CI 80–86%) and 51% of Campbell reviews (95% CI 42–59%) reported on sex/gender. In the results section, less than 30% of reviews reported on sex/gender. Of these, 37% (95% CI 29–45%) of Campbell and 75% (95% CI 68–82%) of Cochrane reviews provided a descriptive report of sex/gender and 63% (95% CI 55–71%) of Campbell reviews and 25% (95% CI 18–32%) of Cochrane reviews reported analytic approaches for exploring sex/gender, such as subgroup analyses, exploring heterogeneity, or presenting disaggregated data by sex/gender.

Conclusion

Our study indicates that sex/gender reporting in Campbell and Cochrane reviews is inadequate.

Peer Review reports

Background

Integration of sex/gender in systematic reviews is important for understanding the applicability of evidence. An increasing number of health researchers are accounting for sex/gender in their studies in recognition of this importance [1, 2]. This shift is due, in part, to an increasing number of research funders and scientific journals requiring researchers to account for sex and gender in their research [3].

Sex refers to the biological attributes, such as physiological characteristics, that generally distinguish males and females. Gender refers to the socially constructed roles, behaviors, and identities of girls, women, boys, men, and gender-diverse individuals. Gender influences the way people act and interact with each other, how they view themselves and others, and the distribution of power within a society. Sex and gender can interact with other determinants of health to influence health status and the effectiveness of interventions.

The risk due to underrepresentation of population subgroups, including women, is well known in primary research. For example, the safety of interventions may differ depending on sex/gender; between 1997 and 2000, eight out of 10 drugs withdrawn by the Food and Drug Administration (FDA) in the USA were the result of greater risks of adverse events for women [4]. In 2013, new information about the adverse effects for a drug to treat insomnia, particularly for women, required the FDA to recommend lower starting doses because of the increased risk of next-morning impairment [5]. Additionally, the benefit of an intervention may vary depending on sex/gender. A systematic review of carotid endarterectomy found that the benefits of surgery were greater for men than women and when stenosis was 50–69% there was no evidence of benefit for women [6]. This information is important as it could lead to separate recommendations, preventing women from undergoing surgery for little to no benefit.

The proportion of female participants in randomized controlled trials (RCTs) has been increasing, in part due to increasing FDA requirements [7]. However, this has not led to an increase in analyses of outcomes by sex/gender. Recently, a study assessing gender in a sample of RCTs published in leading medical journals indexed in PubMed found that while sex/gender data were available to the authors of the studies, they were not often analyzed or discussed. The authors reported that none of the included studies described a priori methods to assess sex/gender differences, only 8 of the 57 included studies presented sex-disaggregated data, and none of these discussed these results or provide reasons for differences in outcomes [8]. Similarly, a review of 100 Canadian RCTs found that only 6% of studies conducted subgroup analyses across sex/gender of which only one discussed the challenges with such analyses and the implications of the findings for clinical practice [9].

Cochrane is an international network of researchers, patients, and health professionals who aim to produce and maintain systematic reviews of healthcare interventions. A related organization, the Campbell Collaboration, produces systematic reviews of interventions in crime and justice, disability, education, international development, and social welfare. The Campbell and Cochrane Equity Methods Group was established in 2007 to encourage authors of systematic reviews to consider whether the effectiveness of the intervention will differ by population subgroup. The Equity Methods Group uses the acronym PROGRESS (Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital) to help remind systematic review authors to consider whether the intervention may have differential effects across these characteristics [10]. Sex/gender is a key focus of the Campbell and Cochrane Equity Methods Group; (http://methods.cochrane.org/equity/sex-and-gender-analysis); they have developed guidance for systematic review authors, such as briefing notes [11] and a planning tool [12].

The use of systematic reviews in policy and practice decision making is increasing [13, 14]. However, policymakers often cite the lack of context and equity considerations (e.g., including sex/gender information) as barriers to using systematic reviews in decision making [12, 14,15,16]. A major reason is that despite an increasing desire to incorporate sex/gender considerations in research, sex/gender differences remain underreported [17]. Systematic reviews are limited in their ability to report on sex/gender if the studies included in the review have not reported sex/gender considerations. Thus, it is unsurprising that an assessment of the Canadian clinical practice guidelines found inconsistencies in the degree to which sex/gender considerations are incorporated, with only 35% of guidelines providing specific screening, diagnosis, or management considerations based on sex/gender [18].

Within systematic reviews, considering sex/gender implies reporting not only the population characteristics of the included studies but also providing some insight into the possible sex/gender differences in the prevalence of the condition, the benefit of the intervention, or safety concerns. Often, systematic reviews that have considered sex/gender have done so using a subgroup analysis, but considerations of the applicability of the evidence are also important. For example, if the authors of the systematic review note that the included studies are gender imbalanced, then the results of the review may not be universally applicable and this should be reported. If the authors fail to mention this, it presents a missed opportunity to address potentially important issues related to sex/gender and our knowledge of benefits/harms for all those who may receive the intervention.

This project began as a partnership with the CIHR Institute of Gender and Health (IGH) with the aim of developing a Cochrane Corner to highlight Cochrane systematic reviews which have considered or assessed sex/gender. The goal of this Cochrane Corner was to increase awareness of the Cochrane Library among those involved in sex/gender and health research while also increasing awareness of sex/gender-based analyses among the Cochrane community. The Cochrane Corner is now available on the Campbell and Cochrane Equity Methods Group’s website: https://methods.cochrane.org/equity/igh-cochrane-corner.

We reviewed systematic reviews published in the Campbell and Cochrane Libraries to assess the extent to which sex/gender is reported and analyzed. For the purposes of this study, we use the term sex/gender to refer to all concepts of sex and/or gender.

Methods

We screened the full text of every systematic review published in the Campbell Library (n = 137 as of September 2017). For the Cochrane Library, we used the advanced search option within the Archie database to select only reviews which used one of our sex/gender search terms (“sex,” “gender,” “male,” “women,” “boys,” “girls”) in at least one of the following review sections: title/abstract, introduction, methods, results, or discussion (n = 674) (total number of reviews published between August 1, 2016, and July 31, 2017, was 1373). We included only the previous year of Cochrane reviews since this resulted in a large sample of reviews representing all of the Cochrane Review Groups. However, since the Campbell Library has a smaller number of published reviews, we decided to include all reviews.

We developed and pre-tested a data extraction form in Excel using the European Association of Science Editors (EASE) Sex and Gender Equity in Research (SAGER) guidelines and our previous work assessing reporting of sex/gender in a sample of randomized controlled trials [19, 9]. We assessed the full text of each review for reporting of sex/gender. One author (one of JT, OD, and MY) independently extracted the text for each section in which sex/gender was reported for each systematic review. A second author (JP) randomly selected 10%, using a random number generator, for independent data quality verification. We classified each mention of sex/gender in each section of the review as descriptive assessment of reporting and analysis (e.g., mention of sex/gender), analytic approaches (e.g., subgroup analyses), and judging applicability (e.g., explicit statement about the applicability of the results).

For those studies which did not report on sex/gender in the abstract, we assessed whether they reported on sex/gender in other review sections. Similarly, we assessed whether reviews that mentioned sex/gender in the methods section also reported sex/gender in the results or discussion. We have summarized the data using descriptive statistics and 95% confidence intervals.

We categorized reviews which stated that recruitment was limited to one sex as “single sex” and excluded these from our analysis. We made this distinction based on how the systematic review authors described eligible studies, particularly in the “types of participants” section of the review. When eligibility was open to more than one sex/gender, we considered the review to be “mixed sex.” For example, we classified a review of interventions for female breast cancer survivors as “single sex” and excluded it. However, in some reviews, although a condition may be sex-specific, the authors included interventions in which multiple sex/genders were eligible. For example, a review of interventions to reduce female genital mutilation included studies in which any members of communities practicing female genital mutilation or cutting were recruited (e.g., men and women and community leaders). In these cases, we classified the reviews as “mixed sex” and included them in our analysis. Finally, to compare sex/gender reporting in Campbell and Cochrane reviews, we assessed the reviews published in 2016–2017 since reporting guidelines have changed considerably over the years and we felt it was inappropriate to compare Campbell reviews from 2004 with Cochrane reviews from 2016 [20,21,22,23,24]. Since the number of Campbell reviews published in 2016–2017 is small, we compared this sample to Cochrane reviews published by review groups in which the interventions assessed may be more comparable to the reviews published by Campbell. These groups included the Consumers and Communication (CCG), Effective Practice and Organization of Care (EPOC), and Cochrane Public Health (CPHG) review groups. The Campbell Collaboration publishes reviews of interventions addressing broad social issues, such as education, crime and justice, and international development. As such, the questions are often broader and may require the inclusion of a range of study designs. The Cochrane CCG publish reviews of interventions to assess the way people interact with healthcare professionals, services, and researchers; EPOC publish reviews of interventions assessing professional practice and the delivery of health services; and CPHG publish systematic reviews of interventions at the population level to assess health, wellbeing, learning, and social outcomes. For these reasons, we decided that Campbell reviews were most comparable to these groups than other Cochrane review groups which assess interventions to prevent, treat, or manage specific health conditions.

Results

We screened 137 Campbell reviews published between May 2004 and July 2017 and 674 new and updated Cochrane reviews published between August 2016 and July 2017. Five Cochrane reviews were excluded because they were withdrawn or published and updated (or published with a correction) within the same time frame.

We classified 114 Cochrane reviews and 4 Campbell reviews as single sex because they restricted eligibility to studies that included women only (n = 114) or men only (n = 4). Of the 114 Cochrane reviews, 27 (male n = 1; female n = 26) were considered sex-specific (e.g., interventions for vasectomy), 3 restricted eligibility to a single sex but mentioned the opposite sex/gender (e.g., interventions for assisted reproductive technologies that recruited only females but mentioned male factor infertility), and 9 restricted eligibility to a single sex with no mention of the opposite sex/gender even though the condition is not sex-specific (e.g., breast cancer reviews that only included women). There were 75 reviews that we considered to be maternal health focused (e.g., breastfeeding, pregnancy) where only 4% of applicable reviews mentioned the sex of the fetus or infant and 60% of applicable reviews mentioned partner, family, couple, or father involvement. The four Campbell single-sex reviews focused on interventions for female victims of intimate partner abuse, interventions for men who abuse their female partner, interventions for women’s empowerment, and treatment for male sexual offenders.

Therefore, the total number of Cochrane reviews included in our analysis is 555 and we have included 133 Campbell reviews. The study flow diagram is described in Fig. 1. The characteristics of the systematic reviews included in this study are presented in Table 1, and the sex/gender reporting by review section is presented in Table 2.

Fig. 1
figure 1

Study flow diagram

Table 1 Characteristics of the included systematic reviews
Table 2 Reporting of sex/gender by review section

Only 14% (95% confidence interval (CI) 8–19%) of Campbell reviews and 10% (95% CI 7–12%) of Cochrane reviews mentioned sex/gender in the abstract of the review. Of these, 16% (95% CI 10–22%) of Campbell reviews and 11% (95% CI 6–16%) of Cochrane reviews included analytic descriptions of sex/gender in the abstracts. Only one Campbell review reported on the applicability of the evidence with regard to sex/gender (5%, 95% CI 1–9%). The sex/gender reporting of Campbell reviews is presented in Fig. 2 and Cochrane reviews in Fig. 3.

Fig. 2
figure 2

Categories of sex/gender reporting in Campbell systematic reviews

Fig. 3
figure 3

Categories of sex/gender reporting in Cochrane systematic reviews

In the background section, 20% (95% CI 13–26%) of Campbell reviews and 28% (95% CI 24–31%) of Cochrane reviews mentioned sex/gender. These were all descriptive, such as reporting the prevalence of the condition or risk factors by sex/gender.

In the methods section, 51% (95% CI 42–59%) of Campbell and 83% (95% CI 80–86%) of Cochrane reviews reported sex/gender considerations. Of these, 87% (95% CI 81–93%) and 81% (95% CI 75–88%) of Campbell and Cochrane reviews respectively mentioned sex/gender in a descriptive way in the methods section. Most often, this was with respect to including sex/gender as data collection items or describing the populations of the included studies. Analytic approaches to sex/gender were reported in the methods section of 13% (95% CI 7–19%) of Campbell and 19% (95% CI 12–25%) of Cochrane reviews, such as planned subgroup analyses as well as investigating population characteristics, such as exploring sex/gender as a source of heterogeneity.

When reported in the methods section, 9.8% (95% CI 5–15%) of Campbell reviews and 20.5% (95% CI 17–24%) of Cochrane reviews also reported sex/gender in the results section and 6.8% (95% CI 2–11%) of Campbell reviews and 12.1% (95% CI 9–15%) of Cochrane reviews reported sex/gender in the discussion section.

In the results sections, 23% (95% CI 15–30%) of Campbell and 29% (95% CI 25–33%) of Cochrane reviews reported on sex/gender. Of these, 34% (95% CI 26–43%) of Campbell and 75% (95% CI 68–82%) of Cochrane reviews provided a descriptive report of sex/gender, such as the characteristics of the included populations, and 66% (95% CI 57–74%) of Campbell reviews and 25% (95% CI 18–32%) of Cochrane reviews reported analytic approaches for sex/gender. The analytic approaches included subgroup analyses (tests for interaction across sex/gender), assessment of heterogeneity (restricting to one sex/gender and assessing whether the results are consistent with the overall effects), or presentation of sex-disaggregated data (without testing for interactions). When reported in the results section of the review, 6.8% (95% CI 2–11%) of Campbell reviews and 9.2% (95% CI 7–12%) of Cochrane reviews also reported sex/gender in the discussion section.

For the discussion section, 14% (95% CI 8–19%) of Campbell and 15% (95% CI 12–18%) of Cochrane reviews reported on sex/gender. Of these, 37% (95% CI 29–45%) of Campbell reviews and 38% (95% CI 30–46%) of Cochrane reviews provided judgments about the applicability of the evidence for sex/gender in the discussion section of the reviews. Examples of the considerations of sex/gender as reported in each section of the review are provided in Table 3.

Table 3 Examples of reporting in each systematic review section

We assessed whether reporting of sex/gender considerations in the abstract of the systematic review is an indicator of its reporting in other review sections. For Campbell reviews, only 12% reported on sex/gender in the abstract. However, 62% (95% CI 54–71%) reported sex/gender considerations in other sections of the review without mentioning them in the abstract. In the results section, 47% of Campbell reviews (95% CI 39–56%) reported sex/gender without reporting it in the abstract.

For Cochrane reviews, while 10% reported sex/gender in the abstract, 86% (95% CI 83–89%) reported sex/gender in another section of the review without reporting it in the abstract. This includes 23% (95% CI 19–26%) that reported sex/gender in the results section but not in the abstract and 11% (95% CI 6–17%) of Campbell and 11% (95% CI 9–14%) of Cochrane reviews that reported sex/gender in the discussion but not the abstract.

Of the 51% (95% CI 42–59) of Campbell reviews reporting sex/gender in the methods, 81% (95% CI 75–88%) were descriptive and 19% (95% CI 12–25%) were analytic. However, in the results section, 37% (95% CI 29–45%) of the instances of reporting sex/gender were descriptive while 63% (95% CI 55–71%) were analytic. For the Cochrane reviews in our sample, 83% (95% CI 80–86%) reported sex/gender in the methods of which 81% (95% CI 75–88%) were descriptive and 19% (95% CI 12–25%) were analytic. In the results section, 75% (95% CI 18–32%) were descriptive and 25% (95% CI 18–32%) were analytic.

Finally, we compared the sex/gender reporting of the Campbell reviews published in 2016–2017 (n = 18) to the reviews published by the Cochrane CCG, EPOC, and CPHG (n = 16) (Fig. 4). Campbell reviews in this time period included more sex/gender considerations in the abstract (31.1% vs. 6.3%), background (25% vs. 12.5%), and results (37.5% vs 31.3%) compared to Cochrane while the Cochrane reviews reported more sex/gender considerations in the methods (81.3% vs 25%) and discussion (25% vs 12.5%) sections.

Fig. 4
figure 4

Reporting of sex/gender by review section, 2016–2017 Campbell and Cochrane CCG, EPOC, and CPGH reviews

Discussion

Overall, the methods sections included the most reporting of sex/gender in both Campbell (50.8%) and Cochrane (83.1%) reviews. The majority of these were descriptive considerations of sex/gender, such as the collection of sex/gender data. A previous study of systematic reviews indexed in MEDLINE in 1 month in 2004 found that almost half of systematic reviews described the sex/gender of the included populations but only 4% assessed sex/gender differences or included sex/gender when making judgements about the applicability or the implications of the evidence [25]. A study of sex/gender reporting in RCTs in 2013–2014 found that 98% of trials report the sex/gender characteristics of their included population which indicates that there is an opportunity for reporting within reviews to improve [9].

We found that 62% of Campbell reviews and 86% of Cochrane reviews did not report sex/gender in the abstract but included sex/gender considerations in a later section. This included 47% of Campbell reviews and 23% of Cochrane reviews which reported sex/gender considerations in the results section. This is an important finding since those who use systematic reviews in decision making often rely on the abstract to make a decision about whether to access the full text. Important sex/gender-related evidence, including findings of no differences in effectiveness based on sex/gender, may be missed.

This sample of systematic reviews presents some important evidence regarding sex/gender differences. For example, one review found statistically significantly higher risk of neonatal mortality among female infants for vitamin A supplementation at 12 months follow-up [26]. Another review reported that the risk of stroke or death following carotid endarterectomy for symptomatic carotid stenosis was higher among women and the number needed to treat to prevent one stroke is four times higher for women than men [6].

There has been debate about the impact of the lack of sex/gender information in systematic reviews. When the studies included in a systematic review have not considered sex/gender differences, it represents an applicability issue, but this could also be considered a risk of bias. The Cochrane Risk of Bias tool was developed to help systematic review authors consider the potential limitations of the studies included in their review [27]. It refers to systemic error because replication of the study would produce the same incorrect result. If trialists and the systematic review authors fail to consider potential differences in the effectiveness of an intervention for men and women, this may present a risk of bias, since users of the review are unable to accurately assess for whom the intervention is effective, or not [28]. At the very least, reporting bias could occur if only some included studies provide sex-disaggregated data and therefore any sex-based analysis is only based on that subset of data [11].

Two reviews included in our study did mention sex/gender as a potential source of bias. The first review assessed mobile health clinics but was focused on women and children’s health outcomes and therefore excluded studies in which services were offered exclusively to men [29]. The authors stated that this may have biased the estimate of the effectiveness of mobile health clinics in improving access to health care within underserved communities. The second reported that one included study which assessed postoperative pain was potentially biased because of an uneven distribution of male and female participants between groups. This was a small study with three arms assessing different timing of ibuprofen provided pre- or postoperatively for orthodontic pain. One study arm had more than twice as many female participants while the third had more than three times as many males. Since previous studies have shown evidence of a possible gender-based difference in pain, the review authors reported this as a potential source of bias [30, 31].

The impact of a lack of sex/gender consideration and potential bias resulting from a lack of consideration of sex/gender will depend on the review question and the characteristics of the studies included. For example, for some health conditions, a systematic review including studies with a majority of women may be acceptable, although the rationale should be described as well as any applicability implications. There are certain questions in which sex/gender could be particularly important, such as those about safety (e.g., pharmacokinetics, surgery) or implementation and programmatic effectiveness. Additionally, the analysis of sex/gender differences carries with it the risk of spurious findings if it is one of multiple analyses. We suggest that authors consider and justify all planned subgroup analyses on the basis of prior literature and advice from content experts and other stakeholders. The absence of evidence of a difference in effect is not justification to assume there is no difference. We propose that systematic review authors should consider population characteristics, such as sex/gender, as well as other potentially important factors, using the PROGRESS-Plus acronym and plan to present disaggregated data, when possible, or justify why these considerations are unnecessary.

Currently, Cochrane and the Campbell Collaboration have no specific policy on the reporting of sex/gender in systematic reviews. However, Cochrane has endorsed the SAGER guidelines developed by the European Association of Science Editors [32]. These guidelines aim to provide guidance on reporting of sex and gender information in study design, data analyses, results, and interpretation of the findings [19]. As of February 2017, 92 individuals/organizations have endorsed the SAGER guidelines.

A pilot study assessing Cochrane authors’ opinions of “briefing notes” for including sex/gender in systematic reviews found that authors were receptive to guidance on improving this aspect of their systematic review. However, they noted that depending on the review question, the extent of sex/gender analysis required and the appropriateness and feasibility will vary [11]. Respondents agreed that sex/gender considerations improve the usefulness of systematic review results for end users, such as policy makers and other decision makers.

Our study assessed sex/gender reporting in all Campbell reviews published as of July 2017 and all new and updated Cochrane reviews published between August 2016 and July 2017. This included a wide variety of systematic reviews published with 5 Campbell coordinating groups and 52 Cochrane review groups. This represents at least one review from each active group as of July 2017. However, our study has some limitations. First, we did not screen all Cochrane reviews published during this period and instead searched for key terms reported in each section of the review. This approach may have missed some reviews which only used terminology that implies sex/gender, such as father or mother, without stating it directly. However, we tested this and did not identify any additional reviews published within the time period sampled. Our preliminary work suggested that a sex/gender-specific term is typically used in at least one section of the review, if consideration of sex/gender is included in the review. For gender-diverse terms, we assumed that since the terminology is rapidly expanding, systematic review authors would have used the term “gender” within their review and therefore these reviews would have been included in our sample. We did not conduct data extraction in duplicate and instead performed a random 10% quality check. This could have resulted in missed instances of sex/gender reporting, but since the quality check had almost 100% agreement, we are confident that the risk is low. Additionally, we did not assess whether the terms “sex” or “gender” were used appropriately by the review authors. Instead, we captured any use of any term related to sex or gender as used by the systematic review author. Therefore, we are unable to report whether these terms are used correctly or consistently in systematic reviews.

We identified 117 systematic reviews that restricted eligibility to studies that focused on a single sex. We did not assess whether these reviews limited recruitment appropriately so we cannot comment on whether these reviews have missed the opportunity to assess potential sex/gender-related differences in the effectiveness of the intervention or to report on implications for the applicability of their findings.

Finally, some systematic review authors may consider sex/gender at the review outset but the team may make a decision that these issues are not relevant and therefore do not get report on them in the review. A limitation of our current study is that while these types of considerations may occur, we are limited by the reporting of the systematic reviews.

An ongoing challenge for systematic review authors is the inconsistent and evolving terminology used in primary studies (and elsewhere). The terms sex and gender are often used interchangeably or incorrectly which can make interpretation of data difficult for those using research results. In this study, we did not assess whether the terms were used correctly or consistently within Campbell and Cochrane reviews. However a recent review of sex/gender considerations in Canadian clinical practice guidelines found that only 25% of those that reported sex- or gender-specific considerations for screening, diagnosis, or management used the terms sex and gender appropriately [18]. Additionally, intersectionality, or the interrelationship of sex and gender and other personal characteristics, such as age, race, and socioeconomic status, has emerged as an important consideration [17]. These intersections require some consideration to understand how interventions will work in practice.

Conclusions

The absence of availability of sex/gender disaggregated data from primary studies and evidence syntheses affects our ability to make relevant policies and programs and affects our ability to reduce health inequities. Since the use of systematic reviews is encouraged for policymaking, an assessment of potential sex/gender-related differences in intervention effectiveness and safety is critically important. Ideally, the increased interest in and emphasis on the importance of sex/gender in research from funders and journal editors for all research, from biomedical research to clinical trials and systematic reviews, will result in improved reporting of these considerations. We hope that if we assess Campbell and Cochrane systematic reviews in 5 years the reporting of sex/gender similarities and differences will be improved.

Abbreviations

CCG:

Consumers and Communication Group (Cochrane)

CI:

Confidence interval

CIHR:

Canadian Institutes of Health Research

CPHG:

Cochrane Public Health Group

EASE:

European Association of Science Editors

EPOC:

Effective Practice and Organization of Care

FDA:

Food and Drug Administration

IGH:

Institute of Gender and Health

PROGRESS:

Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital

RCT:

Randomized controlled trial

SAGER:

Sex and Gender Equity in Research

References

  1. CIHR Institute of Gender and Health. Shaping science for a healthier world: Strategy 2017. Ottawa: Canadian Institutes of Health Research Institute of Gender and Health; 2014.

    Google Scholar 

  2. Johnson J, Sharman Z, Vissandjee B, Stewart DE. Does a change in health research funding policy related to the integration of sex and gender have an impact? PLoS One. 2014;9(6):e99900.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  3. Johnson JL, Beaudet A. Sex and gender reporting in health research: why Canada should be a leader. Can J Public Health. 2014;103(1):e80–1.

    Google Scholar 

  4. Sandberg K, Verbalis JG. Sex and the basic scientist: is it time to embrace title IX. Biol Sex Differ. 2013;4:13.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Kesselheim AS, Donneyong M, Dal Pan GJ, et al. Changes in prescribing and healthcare resource utilization after FDA Drug Safety Communications involving zolpidem-containing medications. Pharmacoepidemiol Drug Saf. 2017;26(6):712–21.

    Article  PubMed  CAS  Google Scholar 

  6. Orrapin S, Rerkasem K. Carotid endarterectomy for symptomatic carotid stenosis. Cochrane DB Syst Rev. 2017;6:CD001081.

    Google Scholar 

  7. Avery E, Clark J. Sex-related reporting in randomised controlled trials in medical journals. Lancet. 2016;388(10062):2839–40.

    Article  PubMed  Google Scholar 

  8. Phillips SP, Hamberg K. Doubly blind: a systematic review of gender in randomised controlled trials. Glob Health Action. 2016;9(1):29597.

    Article  PubMed  Google Scholar 

  9. Welch V, Doull M, Yoganathan M, et al. Reporting of sex and gender in randomized controlled trials in Canada: a cross-sectional methods study. Res Integr Peer Rev. 2017;2:15.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  10. O'Neill J, Tabish H, Welch V, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clinl Epidemiol. 2014;67(1):56–64.

    Article  Google Scholar 

  11. Doull M, Welch V, Puil L, et al. Development and evaluation of ‘briefing notes’ as a novel knowledge translation tool to aid the implementation of sex/gender analysis in systematic reviews: a pilot study. PLoS One. 2014;9(11):e110786.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  12. Doull M, Runnels V, Tudiver S, Boscoe M. Sex and gedner in systematic reviews: planning tool. 2011; http://methods.cochrane.org/sites/methods.cochrane.org.equity/files/public/uploads/SRTool_PlanningVersionSHORTFINAL.pdf. Accessed Aug 13 2017.

    Google Scholar 

  13. Bero LA, Jadad AR. How consumers and policymakers can use systematic reviews for decision making. Ann Intern Med. 1997;127(1):37–42.

    Article  PubMed  CAS  Google Scholar 

  14. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: what is evidence-informed policymaking? Health Res Policy Syst. 2009;7(Suppl 1):S1.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Chambers D, Wilson PM, Thompson CA, Hanbury A, Farley K, Light K. Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge-translation resources. Milbank Qy. 2011;89(1):131–56.

    Article  Google Scholar 

  16. Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Community Health. 2004;58(10):811–6.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Day S, Mason R, Lagosky S, Rochon PA. Integrating and evaluating sex and gender in health research. Health Res Policy Syst. 2016;14(1):75.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Tannenbaum C, Clow B, Haworth-Brockman M, Voss P. Sex and gender considerations in Canadian clinical practice guidelines: a systematic review. CMAJ Open. 2017;5(1):E66–73.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Heidari S, Babor TF, De Castro P, Tort S, Curno M. Sex and gender equity in research: rationale for the SAGER guidelines and recommended use. Res Integr Peer Rev. 2016;1:2.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Welch V, Petticrew M, Petkovic J, et al. Extending the PRISMA statement to equity-focused systematic reviews (PRISMA-E 2012): explanation and elaboration. Int J Equity Health. 2015;14:92.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Welch V, Petticrew M, Tugwell P, Moher D, O'Neill J, et al. PRISMA-Equity 2012 Extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9(10):e1001333.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews (MECIR): Standards for the conduct and reporting of new Cochrane Intervention Reviews, reporting of protocols and the planning, conduct and reporting of updates. 2018; http://community.cochrane.org/mecir-manual. Accessed 05 Feb 2018.

    Google Scholar 

  24. The Methods Group of the Campbell Collaboration. Methodological expectations of Campbell Collaboration intervention reviews: Reporting standards. Campbell Policies and Guidelines Series No. 4 2017; https://www.campbellcollaboration.org/library/campbell-methods-reporting-standards.html.

    Google Scholar 

  25. Welch V, Petticrew M, Ueffing E, et al. Does consideration and assessment of effects on health equity affect the conclusions of systematic reviews? A methodology study. PLoS One. 2012;7(3):e31360.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  26. Haider BA, Sharma R, Bhutta ZA. Neonatal vitamin a supplementation for the prevention of mortality and morbidity in term neonates in low and middle income countries. Cochrane DB Syst Rev. 2017;2:CD006980.

    Google Scholar 

  27. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Runnels V, Tudiver S, Doull M, Boscoe M. The challenges of including sex/gender analysis in systematic reviews: a qualitative survey. Syst Rev. 2014;3:33.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Abdel-Aleem H, El-Gibaly OM, El-Gazzar AF, Al-Attar GS. Mobile clinics for women’s and children’s health. Cochrane DB Syst Rev. 2016;8:CD009677.

    Google Scholar 

  30. Ashley PF, Parekh S, Moles DR, Anand P, MacDonald LC. Preoperative analgesics for additional pain relief in children and adolescents having dental treatment. Cochrane DB Syst Rev. 2016;8:CD008392.

    Google Scholar 

  31. Bernhardt MK, Southard KA, Batterson KD, Logan HL, Baker KA, Jakobsen JR. The effect of preemptive and/or postoperative ibuprofen therapy for orthodontic pain. Am J Orthod Dentofac. 2001;120(1):20–7.

    Article  CAS  Google Scholar 

  32. European Association of Science Editors (EASE) Gender Policy Committee. GPC Endorsement, Adoption & Collaboration. http://www.ease.org.uk/strategy-groups/gender-policy-committee/gpc-endorse-adopt-collab/. Accessed 06 Feb 2018.

  33. Gowing L, Ali R, White JM, Mbewe D. Buprenorphine for managing opioid withdrawal. Cochrane DB Syst Rev. 2017;2:CD002025.

    Google Scholar 

  34. Amaya-Guio J, Viveros-Carreno DA, Sierra-Barrios EM, Martinez-Velasquez MY, Grillo-Ardila CF. Antibiotic treatment for the sexual partners of women with bacterial vaginosis. Cochrane DB Syst Rev. 2016;10:CD011701.

    Google Scholar 

  35. De Buck E, Van Remoortel H, Hannes K, et al. Approaches to promote handwashing and sanitation on behaviour change in low- and middle-income countries. Campbell Syst Rev. 2017;7

  36. Abraha I, Luchetta ML, De Florio R, et al. Ultrasonography for endoleak detection after endoluminal abdominal aortic aneurysm repair. Cochrane DB Syst Rev. 2017;6:CD010296.

    Google Scholar 

  37. Wilson SJ, Lipsey M, Tanner-Smith E, Huang Huang C, Steinka-Fry KT. Dropout prevention and intervention programs: effects on school completion and dropout among school-aged children and youth. Campbell Syst Rev. 2011;8

  38. Fraquelli M, Casazza G, Conte D, Colli A. Non-steroid anti-inflammatory drugs for biliary colic. Cochrane DB Syst Rev. 2016;9:CD006390.

    Google Scholar 

  39. English C, Hillier SL, Lynch EA. Circuit class therapy for improving mobility after stroke. Cochrane DB Syst Rev. 2017;6:CD007513.

    Google Scholar 

  40. Papageorgiou A, Loke YK, Fromage M. Communication skills training for mental health professionals working with people with severe mental illness. Cochrane DB Syst Rev. 2017;6:CD010006.

    Google Scholar 

  41. Lawry S, Samii C, Hall R, Leopold A, Hornby D, Mtero F. The impact of land property rights interventions on investment and agricultural productivity in developing countries. Campbell Syst Rev. 2014;1

  42. Garjon J, Saiz LC, Azparren A, et al. First-line combination therapy versus first-line monotherapy for primary hypertension. Cochrane DB Syst Rev. 2017;1:CD010316.

    Google Scholar 

  43. Heslop R, Roberts H, Flower D, Jordan V. Interventions for men and women with their first episode of genital herpes. Cochrane DB Syst Rev. 2016;8:CD010684.

    Google Scholar 

  44. Oya C, Schaefer F, Skalidou D, McCosker C, Langer L. Effects of certification schemes for agricultural production on socio-economic outcomes in low- and middle-income countries: a systematic review. Campbell Syst Rev. 2017;3

  45. Petsky HL, Kew KM, Chang AB. Exhaled nitric oxide levels to guide treatment for children with asthma. Cochrane DB Syst Rev. 2016;11:CD011439.

    Google Scholar 

  46. Tripney J, Garcia Hombrados J, Newman M, et al. Post-basic technical and vocational education and training (TVET) interventions to improve employability and employment of TVET graduates in low- and middle-income countries. Campbell Syst Rev. 2013;9.

Download references

Acknowledgements

The authors gratefully acknowledge Jordi Pardo Pardo for his assistance in developing the search for the reviews in the Cochrane Library and Alison Riddle for reviewing the draft manuscript.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available from the corresponding author on request.

Author information

Authors and Affiliations

Authors

Contributions

VW conceived the study. JP and VW developed the screening and data extraction forms. JT, OD, and MY screened the reviews and extracted data. JP conducted data quality check and analyzed the data. JP wrote the first draft of the manuscript. JT, OD, MY, PT, and VW suggested revisions to the manuscript. All authors approved the final version.

Corresponding author

Correspondence to Jennifer Petkovic.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Petkovic, J., Trawin, J., Dewidar, O. et al. Sex/gender reporting and analysis in Campbell and Cochrane systematic reviews: a cross-sectional methods study. Syst Rev 7, 113 (2018). https://doi.org/10.1186/s13643-018-0778-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-018-0778-6

Keywords