Power and precision are greater in meta-analyses than individual study analyses. However, dichotomisation of continuous outcomes in certain studies poses a problem as estimates from primary studies can only be pooled if they have a common outcome. Meta-analyses may include pooled summaries of either or both the continuous and dichotomous forms, and potentially have a different combination of studies for each depending on whether the outcome was dichotomised in the primary studies or not. This dual-outcome issue can lead to loss of power and/or selection bias. In this study we aimed to illustrate how dichotomisation of a continuous outcome in primary studies may result in biased estimates of pooled risk and odds ratios in meta-analysis using secondary analyses of published meta-analyses with the outcome, birthweight, which is commonly analysed both as continuous, and dichotomous (low birthweight: birthweight < 2,500 g).
Methods
Meta-analyses published in January 2010 - December 2011 were obtained using searches in PubMed, Embase, Web of Science, and Cochrane Database of Systematic Reviews with the outcome birthweight. We used a distributional method to estimate the pooled odds/risk ratio of low birthweight and its standard error as a function of the data reported in the primary studies of the included meta-analyses where accessible.
Results
Seventy-six meta-analyses were identified. Thirty-seven percent (28/76) of the meta-analyses reported only the dichotomous form of the outcome while 26% (20/76) reported only the continuous form. In one meta-analysis (1/76), birthweight was analysed as continuous for one intervention and as binary for another and 36% (27/76) presented both dichotomous and continuous birthweight summaries. In meta-analyses with a continuous outcome, primary studies data were accessible in 39/48 and secondary analyses using the distributional approach provided consistent inferences for both the continuous and distributional estimates in 38/39.
Conclusion
The distributional method applied in primary studies allows both a continuous and dichotomous outcome to be estimated providing consistent inferences. The use of this method in primary studies may restrict selective outcome bias in meta-analyses.
Meta-analyses of medical studies are conducted in order to synthesise research evidence on the subject of interest and provide an epidemiological evaluation of results from primary studies [1]. The use of meta-analysis allows us to quantify the pooled effect of an exposure variable, such as a risk factor or intervention, on an outcome of interest using the results from all available primary studies [2]. The precision and the statistical power of the hypothesis tested in a meta-analysis are usually higher than that of the primary studies due to the increase in the amount of data contributing to the overall pooled estimate [3].
Only primary studies with a common outcome can be pooled in a meta-analysis and so dichotomisation of continuous outcomes presents a difficulty over and above the loss of power [4], underestimation of effect size [5], and the need for larger samples [6, 7] associated with the practice. When different cut-points for a particular continuous outcome have been used in primary studies, their results cannot be compared in a meta-analysis [4]. Pooling primary studies with the continuous and binary form of an outcome in separate meta-analyses [8], may lead to conflicting results and conclusions [8, 9] due to loss of power and selection bias. More precisely, primary studies included in the calculation of pooled estimates may differ for the continuous and dichotomous form according to data presented in the separate reports and, therefore, a meta-analysis may not include all the primary studies carried out on a research question, leading to an incomplete and potentially biased summary of the evidence. Further, information from the same primary study may be used in both meta-analyses, thus making the results repetitive and not necessarily confirmatory [9].
Peacock et al. [10] have previously described a distributional method for use in primary studies which permits researchers to present both the comparison of means and comparison of proportions. This method involves transforming the difference in means between two groups, into a comparison of proportions of subjects that fall below (or above) a threshold of interest, to give a ‘distributional estimate’ expressed as a difference in proportions, risk ratio (RR) or odds ratio (OR). The standard error for the distributional estimate is derived as a function of the means and standard deviations of the sample using the delta method and so inferences drawn from the comparison of proportions reflect inferences about the comparison of means. The purpose of this study was to use the distributional method described above to illustrate how dichotomisation of a continuous outcome in primary studies may result in biased estimates of pooled RRs and ORs in meta-analysis particularly when either outcome includes only a subset of the available primary studies. To do this, we considered an outcome that is commonly reported as dichotomous and/or continuous, birthweight (analysed as continuous (g), or as dichotomous (low birthweight (LBW): < 2,500 g). This threshold (2,500 g) is clinically relevant both in clinical trials and epidemiology spanning various areas of health research.
Methods
Search strategy
Searches of electronic databases were conducted in PubMed, Embase, Web of Science and the Cochrane Database of Systematic Reviews from January 2010 to December 2011 using search terms ‘birthweight’ OR ‘birth weight’. The search was limited to meta-analyses and human studies in PubMed and Embase. The references of the papers that met the inclusion criteria were searched for additional studies.
Meta-analyses in which birthweight was an outcome variable (either primary or secondary) were eligible. Meta-analyses were excluded if birthweight was a risk factor and not an outcome and if the systematic review did not include a meta-analysis on a birthweight outcome. If the birthweight outcome was ‘small and/or large for gestational age’ or presented in terms of correlation coefficient, the meta-analyses were excluded. Other exclusion criteria were genetic and ecological studies. Titles and abstracts were screened and the full texts of studies which met the eligibility criteria were retrieved. The flowchart showing the search strategy is presented in Figure 1 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [11].
Figure 1
Flow diagram of search process for meta-analyses included in this study.
Illustrative analyses
A secondary analysis was performed in order to illustrate the consequences of dichotomisation. The distributional method was used to obtain distributional RR/ORs for each of the primary studies included in each meta-analysis using the reported sample means and standard deviations. These distributional estimates for LBW were then pooled to obtain a summary distributional RR with confidence intervals (CIs) using either the fixed or random effects model as appropriate. We refer to this pooled estimate as the ‘pooled distributional estimate’. This process was undertaken using meta-analyses for which the means and standard deviations of the birthweight data in the pooled primary studies could be accessed. The various steps for the application of the distributional method in obtaining distributional risk and ORs have been set out in another paper [10] and a Stata ado-file is available (http://wwwhomes.uni-bielefeld.de/osauzet/distributional.htm).
For this illustrative study, we assume that birthweight follows a normal distribution as required by the distributional method [10].
All the analyses were performed using Stata version 12.0 [12].
Results
Meta-analyses included in study
A total of 772 papers were retrieved from the search and of these, 76 published meta-analyses which met the inclusion criteria were included in this study. Fifty-two (52/76) of the meta-analyses included in this study combined randomised controlled trials, and in 24/76, observational studies were pooled. Details of the meta-analyses included in this study can be found in an additional file [see Additional file 1].Thirty-seven percent (28/76) of the meta-analyses reported only the dichotomous form of the outcome while 26% (20/76) reported only the continuous form. In 1/76 meta-analysis, birthweight was analysed as continuous for one intervention and as binary for another. Thirty-six percent (27/76) of the meta-analyses presented both binary and continuous forms of the variable. Among these, 7/27 reported results that were statistically significant for one outcome and not for the other. The number of meta-analyses reporting either dichotomous or continuous outcomes is presented in a flow diagram (see Figure 2). There was no discussion related to the presentation of two separate meta-analyses for both forms of the outcome in any of the 76 meta-analyses papers included in this study.
Figure 2
Flow diagram showing details of meta-analyses included in this study.
Using the distributional method in secondary meta-analysis
Tables 1 and 2 show the results of the secondary meta-analyses performed to illustrate how the results might have looked had the distributional method been used to give means and proportions in all primary studies. Secondary analyses were performed using data reported in meta-analyses (N = 39/76) where primary study means and standard deviations were reported and could be accessed (that is meta-analyses reporting a continuous outcome: N = 21/39; meta-analyses reporting both continuous and dichotomous: N = 18/39).
Table 1
Secondary analyses in meta-analyses reporting only birthweight mean difference outcome (N = 21)
CI: confidence interval; LBW: low birthweight; NA: not applicable; OR: odds ratio; RR: risk ratio; SE: standard error; aFixed effects model used in published meta-analysis but there was significant heterogeneity between studies (P = 0.003) so secondary analysis was repeated here using the random effects model.
For the meta-analyses reporting only the mean difference outcome (N = 21/39) for an intervention/exposure, secondary analyses provide distributional estimates for low birthweight that reflect those of the mean differences (see Table 1).
Where both forms of outcome were reported, the number of pooled primary studies for each differed in 16/18 meta-analyses, so the results for different outcomes were based on different subsets of the available data (see Table 2). In 2/18 [39, 44], the same studies were combined for each outcome and although the distributional RR were similar to those of the published RR, the CIs were narrower with inferences consistent with those of the mean differences.
The distributional estimates provided similar inferences to the mean difference outcomes in 17/18 meta-analyses (see Table 2) confirming that the distributional estimates are valid. For 1/18 meta-analysis [40], where the results of the distributional estimates were not consistent with that of the mean difference outcome (see Table 2), many of its primary studies were very small with very low means and so the distributional method would not be recommended [10].
Secondary analyses could not be performed on 28/76 meta-analyses which reported only the dichotomous form of the outcome. In a further 9/76 meta-analyses reporting both dichotomous and continuous forms of the birthweight outcome, details of the pooled primary studies’ data could not be accessed and the reasons are outlined in an additional file [see Additional file 2].
Discussion
The aim of this study was to illustrate how dichotomisation of a continuous outcome in primary studies may result in biased estimates of pooled RRs and ORs in meta-analysis, using published meta-analyses reporting the birthweight outcome as an example. There is a difficulty in comparing results on the basis of statistical significance; a comparison of means will be more powerful than a comparison of proportions below a cut-point in the same datasets and, therefore, the former are more likely to be statistically significant. This is one reason why using a distributional method in primary studies to estimate proportions below a cut-point carries an advantage in that estimates of mean differences are comparable to estimates based on comparison of proportions [10].
Researchers commonly dichotomise continuous data such as birthweight as it may be difficult to interpret differences in means, but a difference in percentage of low birthweight may be more meaningful. When a continuous outcome is dichotomised in some primary studies but not others, this may cause difficulties for the meta-analyst. The process of selecting primary studies for inclusion in a meta-analysis may include deciding between studies reporting the continuous or the binary form of an outcome. In such cases, the set of studies reporting the continuous outcome may be different from those reporting the binary form. Where a different set of studies are combined for each of the two outcomes, there is the possibility of biased results. This is illustrated in Table 2 where the number of studies and/or sample sizes are very different for the two outcomes in 16/18 meta-analyses. For these meta-analyses, although the precision of the distributional estimates gave similar inferences as the mean differences, they were not comparable with those of the published LBW outcome because they are based on different sets of data.
We do not imply that the distributional approach gives the more accurate result in our illustration as the limited availability of data in published papers prevented its application for all studies. The best option is for primary studies to report both forms for birthweight as other researchers may wish to see them.
Several methods have been developed for combining individual studies reporting continuous and binary outcomes in one meta-analysis to obtain one summary measure in meta-analysis [8, 9, 52]. Whitehead et al. [8] obtained a summary log-odds ratio while other authors [9, 52] have recommended converting the estimates from individual studies to effect sizes and then combining these. These methods are helpful in allowing all studies to be pooled but do not overcome the problem of the loss of power when dichotomising.
We have used the distributional method in secondary meta-analysis to demonstrate how dichotomisation in primary studies may result in inconsistent estimates in the context of meta-analyses. We are not advocating the distributional method as a tool for meta-analysts who are using aggregate data, but rather wish to highlight its usefulness in primary studies.
Conclusions
Researchers who wish to dichotomise continuous outcomes in primary studies may consider using the distributional approach to obtain the difference in proportions or RR/OR to present alongside differences in means. Where this has not been done, and if the individual study outcomes follow a normal distribution and means, standard deviations, sample sizes are given, the meta-analyst can compute distributional estimates for use in pooled summaries. In this way, meta-analyses will be less subject to selective outcome bias.
Abbreviations
CI:
confidence interval
LBW:
low birthweight
OR:
odds ratio
RR:
risk ratio
SE:
standard error
PRISMA:
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
NA:
not applicable.
Declarations
Acknowledgements
We acknowledge support from the NIHR, Biomedical Research Centre at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London, United Kingdom.
Authors’ Affiliations
(1)
Division of Health and Social Care Research, King’s College London
(2)
AG Epidemiologie & International Public Health, Universitat Bielefeld
(3)
NIHR Biomedical Research Centre at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London
Whitehead A, Bailey AJ, Elbourne D: Combining summaries of binary outcomes with those of continuous outcomes in a meta-analysis. J Biopharm Stat. 1999, 9: 1-16. 10.1081/BIP-100100997.View ArticlePubMedGoogle Scholar
Chinn S: A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med. 2000, 19: 3127-3131. 10.1002/1097-0258(20001130)19:22<3127::AID-SIM784>3.0.CO;2-M.View ArticlePubMedGoogle Scholar
Peacock JL, Sauzet O, Ewings SM, Kerry SM: Dichotomising continuous data while retaining statistical power using a distributional approach. Stat Med. 2012, 31: 3089-3103. 10.1002/sim.5354.View ArticlePubMedGoogle Scholar
Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009, 151: 264-269. 10.7326/0003-4819-151-4-200908180-00135. W264View ArticlePubMedGoogle Scholar
StataCorp: Release 12 edition. Stata Statistical Software. 2011, College Station, TX: StataCorp LPGoogle Scholar
Abou El Senoun G, Dowswell T, Mousa HA: Planned home versus hospital care for preterm prelabour rupture of the membranes (PPROM) prior to 37 weeks’ gestation. Cochrane Database Syst Rev. 2010, CD008053-Google Scholar
Alfirevic Z, Stampalija T, Gyte Gillian ML: Fetal and umbilical Doppler ultrasound in high-risk pregnancies. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Alfirevic Z, Stampalija T, Gyte Gillian ML: Fetal and umbilical Doppler ultrasound in normal pregnancy. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Begley CM, Gyte GM, Murphy DJ, Devane D, McDonald SJ, McGuire W: Active versus expectant management for women in the third stage of labour. Cochrane Database Syst Rev. 2010, CD007412-Google Scholar
Bevilacqua E, Brunelli R, Anceschi MM: Review and meta-analysis: benefits and risks of multiple courses of antenatal corticosteroids. J Matern Fetal Neonatal Med. 2010, 23: 244-260. 10.3109/14767050903165222.View ArticlePubMedGoogle Scholar
Blanco CG, Ballesteros AC, Saladich IG, Pla RC: Glycemic control and pregnancy outcomes in women with type 1 diabetes mellitus using lispro versus regular insulin: a systematic review and meta-analysis. Diabetes Technol Therapeutics. 2011, 13 (9): 907-911. 10.1089/dia.2011.0032.View ArticleGoogle Scholar
Buchanan Sarah L, Crowther Caroline A, Levett Kate M, Middleton P, Morris J, Buchanan Sarah L, Crowther Caroline A, Levett Kate M, Middleton P, Morris J: Planned early birth versus expectant management for women with preterm prelabour rupture of membranes prior to 37 weeks’ gestation for improving pregnancy outcome. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Coleman T, Chamberlain C, Cooper S, Leonardi-Bee J: Efficacy and safety of nicotine replacement therapy for smoking cessation in pregnancy: systematic review and meta-analysis. Addiction (Abingdon, England). 2011, 106 (1): 52-61. 10.1111/j.1360-0443.2010.03179.x.View ArticleGoogle Scholar
Crowther CA, McKinlay CJ, Middleton P, Harding JE: Repeat doses of prenatal corticosteroids for women at risk of preterm birth for improving neonatal health outcomes. Cochrane Database Syst Rev. 2011, CD003935-Google Scholar
Dhulkotia JS, Ola B, Fraser R, Farrell T: Oral hypoglycemic agents vs insulin in management of gestational diabetes: a systematic review and metaanalysis. Am J Obstet Gynecol. 2010, 203: 457-e451-459View ArticlePubMedGoogle Scholar
Gebreselassie SG, Gashe FE: A systematic review of effect of prenatal zinc supplementation on birthweight: meta-analysis of 17 randomized controlled trials. J Health Popul Nutr. 2011, 29: 134-140.PubMedPubMed CentralGoogle Scholar
Imdad A, Bhutta ZA: Effect of balanced protein energy supplementation during pregnancy on birth outcomes. BMC Public Health. 2011, 11 (Suppl 3): S17-10.1186/1471-2458-11-S3-S17.View ArticlePubMedPubMed CentralGoogle Scholar
Lassi Zohra S, Haider Batool A, Bhutta Zulfiqar A: Community-based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Mackeen AD, Seibel-Seamon J, Grimes-Dennis J, Baxter Jason K, Berghella V: Tocolytics for preterm premature rupture of membranes. Cochrane Database of Systematic Reviews. 2011, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Mak A, Cheung MWL, Cheak AAC, Chun-Man Ho R: Combination of heparin and aspirin is superior to aspirin alone in enhancing live births in patients with recurrent pregnancy loss and positive anti-phospholipid antibodies: a meta-analysis of randomized controlled trials and meta-regression. Rheumatology. 2010, 49 (2): 281-288. 10.1093/rheumatology/kep373.View ArticlePubMedGoogle Scholar
Middleton P, Crowther Caroline A, Simmonds L, Muller P: Different intensities of glycaemic control for pregnant women with pre-existing diabetes. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Nabhan AF, Elsedawy MM: Tight control of mild-moderate pre-existing or non-proteinuric gestational hypertension. Cochrane Database Syst Rev. 2011, CD006907-Google Scholar
Quinlivan JA, Julania S, Lam L: Antenatal dietary interventions in obese pregnant women to restrict gestational weight gain to institute of medicine recommendations: a meta-analysis. Obstet Gynecol. 2011, 118 (6): 1395-1401. 10.1097/AOG.0b013e3182396bc6.View ArticlePubMedGoogle Scholar
Rumbold A, Middleton P, Pan N, Crowther Caroline A: Vitamin supplementation for preventing miscarriage. Cochrane Database of Systematic Reviews. 2011, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Stampalija T, Gyte Gillian ML, Alfirevic Z: Utero-placental Doppler ultrasound for improving pregnancy outcome. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Vazquez JC, Abalos E: Treatments for symptomatic urinary tract infections during pregnancy. Cochrane Database Syst Rev. 2011, CD002256-Google Scholar
Buppasiri P, Lumbiganon P, Thinkhamrop J, Ngamjarus C, Laopaiboon M: Calcium supplementation (other than for preventing or treating hypertension) for improving pregnancy and infant outcomes. Cochrane Database Syst Rev. 2011, CD007079-Google Scholar
Crowther CA, Han S: Hospitalisation and bed rest for multiple pregnancy. Cochrane Database Syst Rev. 2010, CD000110-Google Scholar
Dodd JM, Grivell RM, Crowther CA, Robinson JS: Antenatal interventions for overweight or obese pregnant women: a systematic review of randomised trials. BJOG. 2010, 117: 1316-1326. 10.1111/j.1471-0528.2010.02540.x.View ArticlePubMedGoogle Scholar
Gouin K, Murphy K, Shah PS, Knowledge Synth Grp D: Effects of cocaine use during pregnancy on low birthweight and preterm birth: systematic review and metaanalyses. Am J Obstet Gynecol. 2011, 204:Google Scholar
Gülmezoglu AM, Azhar M: Interventions for trichomoniasis in pregnancy. Cochrane Database of Systematic Reviews. 2011, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Kawai K, Spiegelman D, Shankar AH, Fawzi WW: Maternal multiple micronutrient supplementation and pregnancy outcomes in developing countries: meta-analysis and meta-regression. Bull World Health Organ. 2011, 89: 402-411B. 10.2471/BLT.10.083758.View ArticlePubMedPubMed CentralGoogle Scholar
Kenyon S, Boulvain M, Neilson James P: Antibiotics for preterm rupture of membranes. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Ladhani NNN, Shah PS, Murphy KE: Prenatal amphetamine exposure and birth outcomes: a systematic review and metaanalysis. Am J Obstet Gynecol. 2011, 205 (3): 219-e211-219.e217View ArticlePubMedGoogle Scholar
Lamont RF, Nhan-Chang CL, Sobel JD, Workowski K, Conde-Agudelo A, Romero R: Treatment of abnormal vaginal flora in early pregnancy with clindamycin for the prevention of spontaneous preterm birth: a systematic review and metaanalysis. Am J Obstet Gynecol. 2011, 205: 177-190. 10.1016/j.ajog.2011.03.047.View ArticlePubMedPubMed CentralGoogle Scholar
McDonald SD, Han Z, Mulla S, Ohlsson A, Beyene J, Murphy KE, Knowledge Synth G: Preterm birth and low birth weight among in vitro fertilization twins: A systematic review and meta-analyses. Euro J Obstet Gynecol Reprod Biol. 2010, 148: 105-113. 10.1016/j.ejogrb.2009.09.019.View ArticleGoogle Scholar
Murphy VE, Namazy JA, Powell H, Schatz M, Chambers C, Attia J, Gibson PG: A meta-analysis of adverse perinatal outcomes in women with asthma. BJOG. 2011, 118: 1314-1323. 10.1111/j.1471-0528.2011.03055.x.View ArticlePubMedGoogle Scholar
Salmasi G, Grady R, Jones J, McDonald SD: Environmental tobacco smoke exposure and perinatal outcomes: a systematic review and meta-analyses. Acta Obstet Gynecol Scand. 2010, 89: 423-441. 10.3109/00016340903505748.View ArticlePubMedGoogle Scholar
Whitworth M, Bricker L, Neilson James P, Dowswell T: Ultrasound for fetal assessment in early pregnancy. Cochrane Database of Systematic Reviews. 2010, Chichester, UK: John Wiley & Sons, LtdGoogle Scholar
Wiysonge CS, Shey M, Kongnyuy EJ, Sterne JA, Brocklehurst P: Vitamin A supplementation for reducing the risk of mother-to-child transmission of HIV infection. Cochrane Database Syst Rev. 2011, CD003648-Google Scholar
Pope DP, Mishra V, Thompson L, Siddiqui AR, Rehfuess EA, Weber M, Bruce NG: Risk of low birth weight and stillbirth associated with indoor air pollution from solid fuel use in developing countries. Epidemiol Rev. 2010, 32: 70-81. 10.1093/epirev/mxq005.View ArticlePubMedGoogle Scholar
Salvig JD, Lamont RF: Evidence regarding an effect of marine n-3 fatty acids on preterm birth: a systematic review and meta-analysis. Acta Obstet Gynecol Scand. 2011, 90: 825-838. 10.1111/j.1600-0412.2011.01171.x.View ArticlePubMedGoogle Scholar
Sanchez-Meca J, Marin-Martinez F, Chacon-Moscoso S: Effect-size indices for dichotomized outcomes in meta-analysis. Psychol Methods. 2003, 8: 448-467.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
