Investigation of continuous effect modifiers in a meta-analysis on higher versus lower PEEP in patients requiring mechanical ventilation - protocol of the ICEM study
© Kasenda et al.; licensee BioMed Central Ltd. 2014
Received: 28 January 2014
Accepted: 7 May 2014
Published: 20 May 2014
Categorizing an inherently continuous predictor in prognostic analyses raises several critical methodological issues: dependence of the statistical significance on the number and position of the chosen cut-point(s), loss of statistical power, and faulty interpretation of the results if a non-linear association is incorrectly assumed to be linear. This also applies to a therapeutic context where investigators of randomized clinical trials (RCTs) are interested in interactions between treatment assignment and one or more continuous predictors.
Our goal is to apply the multivariable fractional polynomial interaction (MFPI) approach to investigate interactions between continuous patient baseline variables and the allocated treatment in an individual patient data meta-analysis of three RCTs (N = 2,299) from the intensive care field. For each study, MFPI will provide a continuous treatment effect function. Functions from each of the three studies will be averaged by a novel meta-analysis approach for functions. We will plot treatment effect functions separately for each study and also the averaged function. The averaged function with a related confidence interval will provide a suitable basis to assess whether a continuous patient characteristic modifies the treatment comparison and may be relevant for clinical decision-making. The compared interventions will be a higher or lower positive end-expiratory pressure (PEEP) ventilation strategy in patients requiring mechanical ventilation. The continuous baseline variables body mass index, PaO2/FiO2, respiratory compliance, and oxygenation index will be the investigated potential effect modifiers. Clinical outcomes for this analysis will be in-hospital mortality, time to death, time to unassisted breathing, and pneumothorax.
This project will be the first meta-analysis to combine continuous treatment effect functions derived by the MFPI procedure separately in each of several RCTs. Such an approach requires individual patient data (IPD). They are available from an earlier IPD meta-analysis using different methods for analysis. This new analysis strategy allows assessing whether treatment effects interact with continuous baseline patient characteristics and avoids categorization-based subgroup analyses. These interaction analyses of the present study will be exploratory in nature. However, they may help to foster future research using the MFPI approach to improve interaction analyses of continuous predictors in RCTs and IPD meta-analyses. This study is registered in PROSPERO (CRD42012003129).
KeywordsFractional polynomials Interaction analysis MFPI Continuous predictors Individual patient data meta-analysis Acute respiratory distress syndrome
Dichotomizing or categorizing inherently continuous predictor variables raises several issues for statistical analysis and interpretation. These issues include dependence of the statistical significance of the interaction on the number and position of the chosen cut-points, loss of statistical power, and a faulty interpretation of the results if a non-linear association is incorrectly assumed to be linear . To overcome these issues, Royston and Sauerbrei proposed the so-called multivariable fractional polynomials interaction (MFPI) approach to investigate potential treatment modifying effects [2, 3]. For continuous variables they propose to estimate a treatment effect function, which avoids the well-known problems caused by categorizing continuous variables. To summarize functions across several studies they suggested a new strategy for meta-analysis .
A recent individual patient data meta-analysis of three randomized controlled trials (RCTs) showed that the pre-defined subgroup of patients who suffered from an acute respiratory distress syndrome (ARDS) had a clinical benefit across various endpoints if they were treated with a higher positive end-expiratory pressure (PEEP) ventilation strategy [5, 6]. We will use the MFPI approach [2, 3] and the new strategy to summarize functions across RCTs  to re-analyse this dataset of 2,299 critically ill patients from the previously reported individual patient data (IPD) meta-analysis .
The primary aim of the ICEM study is to demonstrate how methodological issues of interaction/subgroup analyses of continuous predictors can be handled by combining a new meta-analysis approach for functions with the MFPI approach. If IPD are available, MFPI allows investigating whether a continuous variable interacts with treatment in one RCT; combination of data from several RCTs strengthens the assessment concerning a treatment modifying effect. When comparing two (or more) treatments in an RCT, several continuous variables (for example, age) are suitable candidates to be investigated as potential modifiers of the treatment effect. The ICEM study will be the first example which combines estimation of treatment effect functions by using MFPI separately in each of several RCTs with a new approach for a meta-analysis of functions. As a secondary aim, we will re-analysis the available IPD data to investigate whether one or more continuous variables have an influence on the comparison of two treatment strategies (higher versus lower PEEP), which is a clinically relevant issue. This paper is an extended version of the registered protocol and shows in an exemplary way how to better use the information from continuous variables if individual patient data from several RCTs are available. In similar projects, it should be obvious how to adapt the relevant steps for a meta-analysis of treatment effect functions.
Our protocol is registered on PROSPERO (CRD42012003129 at http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42012003129).
Characteristics of ALVEOLI, LOVS, and ExPress studies
Diagnosis of acute lung injurya with PaO2/FIO2 ≤ 300
Diagnosis of acute lung injurya and PaO2/FIO2 ≤ 250
Diagnosis of acute lung injurya with PaO2/FIO2 ≤ 300
1999 to 2002
2000 to 2006
2002 to 2005
Number of recruiting hospitals, countries
23, United States
30, Canada, Australia, Saudi Arabia
Patients randomized to higher versus lower PEEP
276 versus 273
476 versus 509b
385 versus 383c
Follow-up for hospital mortality until day 60
Blinded outcome assessors and data analysts
Stopped for perceived futility
Stopped for perceived futility
Higher PEEP according to FiO2 chart, recruitment maneuvers for first 80 patients
Higher PEEP according to FiO2 chart, required plateau pressures ≤ 40 cmH2O, recruitment maneuvers
PEEP as high as possible without increasing the maximum inspiratory plateau pressure > 28 to 30 cmH2O
Conventional PEEP according to FiO2 chart, required plateau pressures ≤ 30 cmH2O, no recruitment maneuvers
Conventional PEEP according to FiO2 chart, required plateau pressures ≤ 30 cmH2O, no recruitment maneuvers
Conventional PEEP (between 5 and 9 cmH2O) to meet oxygenation goals
Target tidal volumes of 6 ml/kg of predicted body weight; plateau pressures ≤ 30 cmH2O (with exception as above); respiratory rate ≤ 35 breaths/minute, adjusted to achieve arterial pH 7.30 to 7.45; ventilator mode: volume-assist control (except higher PEEP group in LOVS required pressure control); oxygenation goals: PaO2 55 to 80 mmHg and SPO2 88 to 95%; standardized weaning)
Proposed statistical methodology
Investigation of interactions
We will use the MFPI approach  to investigate the potential treatment (higher versus lower PEEP) modifying effects of each of the continuous variables with respect to a defined outcome. A ‘pair’ of a potential modifier (for example, body mass index (BMI)) and an outcome (for example, in-hospital mortality) will be considered as one investigation. In total, with four potential modifiers and three outcomes we will have twelve investigations. There will be no P-value adjustment for multiple investigations. All patients will be analyzed in the group to which they were randomized (intention-to-treat principle). For all analyses, we will use the software STATA version 13.0 (Station College, TX, USA).
Individual patient data meta-analysis
Separately for each study we will conduct an MFPI analysis to estimate a treatment effect function. For each modifier-outcome pair, we will use weighted averaging to obtain a summary treatment effect function based on all three studies as previously outlined . We will use a fixed effects approach, because we consider three studies to be too few for a random effect model although a random distribution can be assumed. Usually this averaged treatment effect function is no FP function. It will be plotted to allow for a qualitative assessment of the possible interaction based on the full information of a potential modifier. The individual functions and the averaged function will be the main results to assess whether the variable is a treatment modifier for the specific outcome. We will not conduct any statistical test for the averaged treatment effect function. Combining P-values from the individual studies would be one possible way to obtain an overall P-value but this is probably not very helpful. More suitable ways to derive an overall P-value need to be investigated.
Potential clustering of data
We realized that the data of the three independent trials are clustered by recruiting hospitals. Although there is evidence of considerable ‘center effects’ with data from intensive care patients, Briel et al. found that the variance among the 90 recruiting hospitals explained very little (0.3%) of the total variance for hospital mortality . Differences in patient baseline characteristics such as age, probability of death in hospital from prognostic scores, and proportion of patients with severe sepsis largely (co-variables in the primary analyses of the present study) explained the between-hospital variance of 2.6% found with a basic hierarchical model including only PEEP group and a categorical trial variable as fixed effects and recruiting hospitals as a random effect. Given the negligible ‘between-hospital’ variance we decided to forgo any consideration of ‘center effects’ in the primary analyses of the present study.
Adjustment for confounders
Because of some imbalances regarding age [7, 8] and the proportion of patients with severe sepsis [8, 9], Briel et al. conducted an adjusted analysis for all outcomes . We will adopt this approach, thus each analysis will be conducted with adjustment for the following potential confounders: age (continuous), presence of severe sepsis (yes versus no), and predicted probability of dying in the hospital (based on Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology II scores, which have similar accuracy [12, 13]). Selection of these potential confounders resulted from a previous Delphi-like structured survey among experts from the intensive care field . We will apply the FP1 function selection strategy to the confounders, with FP1 as the most complex permitted functional form. Including all confounders mentioned above, the model will be determined separately for each of the three outcomes using MFP (1.0, 0.05), independent of treatment. In the notation MFP (alpha 1, alpha 2) the value of alpha 1 gives the significance level for the variable selection part of MFP and alpha 2 the significance level of the function selection procedure for continuous variables . Despite some imbalances in the covariate distributions between PEEP groups mentioned above, univariate approaches will be conducted as sensitivity analyses.
To circumvent the issue of influential points all continuous variables will be truncated at the 1% and 99%-tile; meaning that any value below the 1%-tile will be replaced by the value of the 1%-tile, and any value above the 99%-tile will be replaced by the value of the 99%-tile. These truncations will be performed for each study separately.
Handling missing data
Some of the potential modifiers and variables used for adjustment (see below) have missing values of up to about 10%. In order to use all information in all analyses, we will impute missing values before the main analysis starts. To try to ensure that the missing at random assumption is valid, we will include all outcomes and as many other variables as possible in the imputation models . Five imputations will be created using the multiple imputations by chained equations technique. Only the first imputation will be used in analyses. The remaining four will be reserved for sensitivity analysis of the main findings.
Description of outcomes and effect modifiers
We selected three clinical important outcomes of interest from a larger list of outcomes used in the analysis by Briel et al. :
In-hospital mortality at 60 days post randomization (outcome 1a) constitutes the primary efficacy outcome of interest. We will also consider in-hospital mortality as a time-to-event variable (outcome 1b) because we are additionally interested in the timing of mortality events in the randomized groups. Due to the differential follow-up across RCTs beyond day 60 and the fact that the intervention effects happen mainly within the first month, we will censor all surviving patients in the time to event analysis at day 60 as done in the original IPD meta-analysis.
Time to unassisted breathing (outcome 2), which is defined as time from randomization until breathing without mechanical support within the first 28 days is the secondary efficacy outcome of interest. Due to differential follow-up across RCTs for this outcome beyond day 28 and the fact that the intervention effect is supposed to happen before day 28, we will censor patients at day 28 as done in the original IPD meta-analysis. Patients who die before achieving unassisted breathing within the first 28 days will be censored at the day of death. With this procedure we circumvent the competing risk issue in the analysis of this outcome. We are aware of the fact that for prognostic questions, which will not be part of this analysis, cumulative incidence functions would be preferred.
Pneumothorax requiring chest tube drainage in first 28 days after randomization (outcome 3, binary variable) is the main safety outcome, because it captures the main potential adverse event directly associated with higher PEEP (experimental intervention). Again, the reason for choosing a 28-day period is that the follow-up for this outcome is different across included trials beyond day 28 and the intervention effect is supposed to happen within the first 28 days. In the present protocol we will not analyze outcome 3 (main safety outcome) because of competing risks with mortality . In the planned clinical report of this work we will refer to the results of the original IPD meta-analysis with respect to outcome 3, because the MFPI methodology has still to be adapted for a competing risk framework. We will deal with competing risks in an addition to this protocol. For the specified efficacy outcomes (outcomes 1a/b and 2) we anticipate no competing risk problems when using cause-specific Cox models.
The following four continuous potential effect modifiers were all pre-specified by Briel et al. :
Body Mass Index (BMI) at baseline
The BMI is calculated by the ratio of mass in Kg/(Height in m)2. There are no data that suggest a certain direction of the treatment effect modification, but Briel et al. hypothesized less benefit of higher PEEP in patients with higher BMI .
Respiratory compliance (RC) at baseline
A lower RC would reflect more severe lung injury. Briel et al. hypothesized that patients with lower RC have more recruitable lung units and would therefore benefit from higher levels of PEEP. In addition, one could argue that in patients suffering from most severe ARDS, which is commonly associated with very low RC, higher PEEP might no longer provide any benefit.
PaO2/FiO2 ratio at baseline
A low PaO2/FiO2 reflects impaired blood oxygenation and, therefore, more severe lung injury. Similar to RC, Briel et al. hypothesized that patients with a lower PaO2/FiO2 ratio benefit more from higher PEEP levels. It will be interesting to see how the widely accepted ARDS defining cut-off at 200 mmHg, is reflected in this analysis using the MFPI approach. Using this cut-off, a significant interaction was found by Briel et al. [5, 6].
Oxygenation Index at baseline
includes the mean airway pressure and can be regarded as the more reliable marker regarding blood oxygenation compared to the PaO2/FiO2 ratio alone. The higher the OI, the more severe the lung injury; therefore, Briel et al. hypothesized that patients with a higher OI benefit more from higher PEEP levels .
Further candidates (for example, age and sex) may be additionally investigated for interaction. Of note, irrespective of the results, all investigations will be included in a summary table similar to the REMARK profile for prognostic studies .
The ICEM study is the first example that combines estimation of treatment effect functions by using MFPI with a new approach for a meta-analysis of functions for a clinically relevant issue. The approach requires IPD data, which are available from an earlier meta-analysis project. The present article is an extended version of the registered protocol, and shows in an exemplary way how to better use the information from continuous variables if individual patient data from several RCTs are available. In similar projects, it should be obvious how to adapt the relevant steps for a meta-analysis of treatment effect functions. Besides the new application of the MFPI approach in meta-analysis, the available dataset from three RCTs also offers a unique opportunity to identify potential clinically important interaction effects. All these interaction analyses are exploratory in nature; however, they use the full information for a potential treatment modifier and may help in clinical decision-making. We hope that this project will also foster future research using the MFPI approach to improve interaction analyses of continuous predictors in RCTs and in meta-analyses, provided IPD are available.
Acute respiratory distress syndrome
Body mass index
Fraction of inhaled oxygen
Investigation of continuous effect modifiers
Individual patient data
Multivariable fractional polynomial
Multivariable fractional polynomial interaction
Partial arterial pressure of oxygen
Positive end-expiratory pressure
Randomized clinical trial.
We would like to thank all investigators from the original randomized controlled trials included in this individual patient data meta-analysis (namely Maureen Meade, Alain Mercat, Roy G Brower, Arthur S Slutsky, Deborah Cook, Gordon H Guyatt, Laurent Brochard, Jean-Christophe M Richard, and Thomas E Stewart) for their continuous support and collaboration.
The article processing charge was funded by the German Research Foundation (DFG) and the Albert Ludwigs University Freiburg in the funding program Open Access Publishing.
- Royston P, Altman DG, Sauerbrei W: Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006, 25: 127-141. 10.1002/sim.2331.View ArticlePubMedGoogle Scholar
- Royston P, Sauerbrei W: A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials. Stat Med. 2004, 23: 2509-2525. 10.1002/sim.1815.View ArticlePubMedGoogle Scholar
- Royston P, Sauerbrei W: Interactions between treatment and continuous covariates: a step toward individualizing therapy. J Clin Oncol. 2008, 26: 1397-1399. 10.1200/JCO.2007.14.8981.View ArticlePubMedGoogle Scholar
- Sauerbrei W, Royston P: A new strategy for meta-analysis of continuous covariates in observational studies. Stat Med. 2011, 30: 3341-3360. 10.1002/sim.4333.View ArticlePubMedGoogle Scholar
- Briel M, Meade M, Mercat A, Brower RG, Talmor D, Walter SD, Slutsky AS, Pullenayegum E, Zhou Q, Cook D, Brochard L, Richard JC, Lamontagne F, Bhatnagar N, Stewart TE, Guyatt G: Higher vs lower positive end-expiratory pressure in patients with acute lung injury and acute respiratory distress syndrome: systematic review and meta-analysis. J Am Med Assoc. 2010, 303: 865-873. 10.1001/jama.2010.218.View ArticleGoogle Scholar
- Briel M: Higher versus lower positive end-expiratory pressure in patients with acute lung injury: a systematic review and individual patient data meta-analysis (study protocol). 2008, McMaster University, Department of Clinical Epidemiology and BiostatisticsGoogle Scholar
- Brower RG, Lanken PN, MacIntyre N, Matthay MA, Morris A, Ancukiewicz M, Schoenfeld D, Thompson BT: National Heart, Lung and Blood Institute ACTN: higher versus lower positive end-expiratory pressures in patients with the acute respiratory distress syndrome. N Engl J Med. 2004, 351: 327-336.View ArticlePubMedGoogle Scholar
- Meade MO, Cook DJ, Guyatt GH, Slutsky AS, Arabi YM, Cooper DJ, Davies AR, Hand LE, Zhou Q, Thabane L, Austin P, Lapinsky S, Baxter A, Russell J, Skrobik Y, Ronco JJ, Stewart TE, Lung Open Ventilation Study Investigators: Ventilation strategy using low tidal volumes, recruitment maneuvers, and high positive end-expiratory pressure for acute lung injury and acute respiratory distress syndrome: a randomized controlled trial. J Am Med Assoc. 2008, 299: 637-645. 10.1001/jama.299.6.637.View ArticleGoogle Scholar
- Mercat A, Richard JC, Vielle B, Jaber S, Osman D, Diehl JL, Lefrant JY, Prat G, Richecoeur J, Nieszkowska A, Gervais C, Baudot J, Bouadma L, Brochard L, Expiratory Pressure (Express) Study Group: Positive end-expiratory pressure setting in adults with acute lung injury and acute respiratory distress syndrome: a randomized controlled trial. J Am Med Assoc. 2008, 299: 646-655. 10.1001/jama.299.6.646.View ArticleGoogle Scholar
- Royston P, Sauerbrei W: Multivariable Model-building. A Pragmatic Approach to Regression Analysis Based on Fractional Polynomials for Modelling Continuous Variables. 2008, Chichester: John Wiley & SonsGoogle Scholar
- Royston P, Sauerbrei W, Ritchie A: Is treatment with interferon-alpha effective in all patients with metastatic renal carcinoma? A new approach to the investigation of interactions. Br J Cancer. 2004, 90: 794-799. 10.1038/sj.bjc.6601622.View ArticlePubMedPubMed CentralGoogle Scholar
- Beck DH, Smith GB, Pappachan JV, Millar B: External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive Care Med. 2003, 29: 249-256.View ArticlePubMedGoogle Scholar
- Suistomaa M, Niskanen M, Kari A, Hynynen M, Takala J: Customized prediction models based on APACHE II and SAPS II scores in patients with prolonged length of stay in the ICU. Intensive Care Med. 2002, 28: 479-485. 10.1007/s00134-002-1214-9.View ArticlePubMedGoogle Scholar
- White IR, Royston P, Wood AM: Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011, 30: 377-399. 10.1002/sim.4067.View ArticlePubMedGoogle Scholar
- Wolkewitz M, Beyersmann J, Gastmeier P, Schumacher M: Modeling the effect of time-dependent exposure on intensive care unit mortality. Intensive Care Med. 2009, 35: 826-832. 10.1007/s00134-009-1423-6.View ArticlePubMedGoogle Scholar
- Altman DG, McShane LM, Sauerbrei W, Taube SE: Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): explanation and elaboration. PLoS Med. 2012, 9: e1001216-10.1371/journal.pmed.1001216.View ArticlePubMedPubMed CentralGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.