Protocol | Open | Open Peer Review | Published:
Investigation of continuous effect modifiers in a meta-analysis on higher versus lower PEEP in patients requiring mechanical ventilation - protocol of the ICEM study
Systematic Reviewsvolume 3, Article number: 46 (2014)
Categorizing an inherently continuous predictor in prognostic analyses raises several critical methodological issues: dependence of the statistical significance on the number and position of the chosen cut-point(s), loss of statistical power, and faulty interpretation of the results if a non-linear association is incorrectly assumed to be linear. This also applies to a therapeutic context where investigators of randomized clinical trials (RCTs) are interested in interactions between treatment assignment and one or more continuous predictors.
Our goal is to apply the multivariable fractional polynomial interaction (MFPI) approach to investigate interactions between continuous patient baseline variables and the allocated treatment in an individual patient data meta-analysis of three RCTs (N = 2,299) from the intensive care field. For each study, MFPI will provide a continuous treatment effect function. Functions from each of the three studies will be averaged by a novel meta-analysis approach for functions. We will plot treatment effect functions separately for each study and also the averaged function. The averaged function with a related confidence interval will provide a suitable basis to assess whether a continuous patient characteristic modifies the treatment comparison and may be relevant for clinical decision-making. The compared interventions will be a higher or lower positive end-expiratory pressure (PEEP) ventilation strategy in patients requiring mechanical ventilation. The continuous baseline variables body mass index, PaO2/FiO2, respiratory compliance, and oxygenation index will be the investigated potential effect modifiers. Clinical outcomes for this analysis will be in-hospital mortality, time to death, time to unassisted breathing, and pneumothorax.
This project will be the first meta-analysis to combine continuous treatment effect functions derived by the MFPI procedure separately in each of several RCTs. Such an approach requires individual patient data (IPD). They are available from an earlier IPD meta-analysis using different methods for analysis. This new analysis strategy allows assessing whether treatment effects interact with continuous baseline patient characteristics and avoids categorization-based subgroup analyses. These interaction analyses of the present study will be exploratory in nature. However, they may help to foster future research using the MFPI approach to improve interaction analyses of continuous predictors in RCTs and IPD meta-analyses. This study is registered in PROSPERO (CRD42012003129).
Dichotomizing or categorizing inherently continuous predictor variables raises several issues for statistical analysis and interpretation. These issues include dependence of the statistical significance of the interaction on the number and position of the chosen cut-points, loss of statistical power, and a faulty interpretation of the results if a non-linear association is incorrectly assumed to be linear . To overcome these issues, Royston and Sauerbrei proposed the so-called multivariable fractional polynomials interaction (MFPI) approach to investigate potential treatment modifying effects [2, 3]. For continuous variables they propose to estimate a treatment effect function, which avoids the well-known problems caused by categorizing continuous variables. To summarize functions across several studies they suggested a new strategy for meta-analysis .
A recent individual patient data meta-analysis of three randomized controlled trials (RCTs) showed that the pre-defined subgroup of patients who suffered from an acute respiratory distress syndrome (ARDS) had a clinical benefit across various endpoints if they were treated with a higher positive end-expiratory pressure (PEEP) ventilation strategy [5, 6]. We will use the MFPI approach [2, 3] and the new strategy to summarize functions across RCTs  to re-analyse this dataset of 2,299 critically ill patients from the previously reported individual patient data (IPD) meta-analysis .
The primary aim of the ICEM study is to demonstrate how methodological issues of interaction/subgroup analyses of continuous predictors can be handled by combining a new meta-analysis approach for functions with the MFPI approach. If IPD are available, MFPI allows investigating whether a continuous variable interacts with treatment in one RCT; combination of data from several RCTs strengthens the assessment concerning a treatment modifying effect. When comparing two (or more) treatments in an RCT, several continuous variables (for example, age) are suitable candidates to be investigated as potential modifiers of the treatment effect. The ICEM study will be the first example which combines estimation of treatment effect functions by using MFPI separately in each of several RCTs with a new approach for a meta-analysis of functions. As a secondary aim, we will re-analysis the available IPD data to investigate whether one or more continuous variables have an influence on the comparison of two treatment strategies (higher versus lower PEEP), which is a clinically relevant issue. This paper is an extended version of the registered protocol and shows in an exemplary way how to better use the information from continuous variables if individual patient data from several RCTs are available. In similar projects, it should be obvious how to adapt the relevant steps for a meta-analysis of treatment effect functions.
Our protocol is registered on PROSPERO (CRD42012003129 at http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42012003129).
The present interaction analyses will be based on IPD sets from three RCTs identified by a systematic review in 2010 [5, 7–9] (Table 1, total of 2,299 patients). These trials investigated the benefits and harms of higher-PEEP ventilation compared to lower-PEEP ventilation in patients with acute lung injury including ARDS. Trial eligibility criteria, literature search strategies, and main results have previously been reported . Standardization of variables and consistency checks have already been performed, thus no more data cleaning will be necessary. Before writing the protocol for the study we have updated the earlier (January 2010) literature search (MEDLINE, EMBASE, CENTRAL) and could not identify additional eligible RCTs. Therefore, the present analysis will focus on the three eligible RCTs from the previous IPD meta-analysis .
Proposed statistical methodology
Investigation of interactions
We will use the MFPI approach  to investigate the potential treatment (higher versus lower PEEP) modifying effects of each of the continuous variables with respect to a defined outcome. A ‘pair’ of a potential modifier (for example, body mass index (BMI)) and an outcome (for example, in-hospital mortality) will be considered as one investigation. In total, with four potential modifiers and three outcomes we will have twelve investigations. There will be no P-value adjustment for multiple investigations. All patients will be analyzed in the group to which they were randomized (intention-to-treat principle). For all analyses, we will use the software STATA version 13.0 (Station College, TX, USA).
We will use MFPI with FP2 functions as the most complex allowable function and we will test for an interaction at the 5% level in each trial. FP2 functions are extensions of conventional quadratic functions that provide considerably enhanced flexibility for more realistic modeling in real data. Instead of only powers 1 and 2, they utilize additional combinations of powers of the predictor (see Figure 1 for the powers that may be selected, adapted from Royston and Sauerbrei ). Having only two power terms, FP2 functions can exhibit at most one maximum or minimum. We assume that FP2 functions could be a suitable functional form, assuming that patients with extremely high or low values of the continuous predictor might not benefit from the experimental intervention. For each potential effect modifier the functional relationship between this predictor and the outcome will be illustrated using treatment effect functions, irrespective of the P-value from the test for interaction. The functional form derived with the MFPI procedure will be checked for potential mismodeling by considering the treatment effect in four subgroups of the predictor of about equal sample size . The analysis strategy needs re-consideration if the estimated treatment effect function disagrees severely with the corresponding results in subgroups, indicating mismodeling of the treatment effect function. For binary outcomes, we will estimate odds ratios with 95% confidence intervals (CIs) to quantify the magnitude of effect. Briel et al. had primarily calculated clinically more intuitive relative risks using log-binomial regression instead of odds ratios, but were confronted with computational problems of non-converging log-binomial models in some analyses. For all binary outcomes, they additionally calculated odds ratios and found similar results although event rates for hospital mortality were > 30% in treatment and control groups . Given the similarity of the results we decided to use the logistic regression model in the present study in order to prevent computational issues when applying the MFPI approach. For survival analysis, Kaplan-Meier estimates and hazard ratios with 95% CIs will be presented. Of note, all investigations of a survival outcome will start with a check of the proportional hazards assumption of the effect of treatment in a univariate Cox-model. We will use the Grambsch-Therneau test for this purpose. If the proportional hazards assumption is seriously violated, we will stop the corresponding investigation and will re-consider a suitable strategy for analysis.
Individual patient data meta-analysis
Separately for each study we will conduct an MFPI analysis to estimate a treatment effect function. For each modifier-outcome pair, we will use weighted averaging to obtain a summary treatment effect function based on all three studies as previously outlined . We will use a fixed effects approach, because we consider three studies to be too few for a random effect model although a random distribution can be assumed. Usually this averaged treatment effect function is no FP function. It will be plotted to allow for a qualitative assessment of the possible interaction based on the full information of a potential modifier. The individual functions and the averaged function will be the main results to assess whether the variable is a treatment modifier for the specific outcome. We will not conduct any statistical test for the averaged treatment effect function. Combining P-values from the individual studies would be one possible way to obtain an overall P-value but this is probably not very helpful. More suitable ways to derive an overall P-value need to be investigated.
Potential clustering of data
We realized that the data of the three independent trials are clustered by recruiting hospitals. Although there is evidence of considerable ‘center effects’ with data from intensive care patients, Briel et al. found that the variance among the 90 recruiting hospitals explained very little (0.3%) of the total variance for hospital mortality . Differences in patient baseline characteristics such as age, probability of death in hospital from prognostic scores, and proportion of patients with severe sepsis largely (co-variables in the primary analyses of the present study) explained the between-hospital variance of 2.6% found with a basic hierarchical model including only PEEP group and a categorical trial variable as fixed effects and recruiting hospitals as a random effect. Given the negligible ‘between-hospital’ variance we decided to forgo any consideration of ‘center effects’ in the primary analyses of the present study.
Adjustment for confounders
Because of some imbalances regarding age [7, 8] and the proportion of patients with severe sepsis [8, 9], Briel et al. conducted an adjusted analysis for all outcomes . We will adopt this approach, thus each analysis will be conducted with adjustment for the following potential confounders: age (continuous), presence of severe sepsis (yes versus no), and predicted probability of dying in the hospital (based on Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology II scores, which have similar accuracy [12, 13]). Selection of these potential confounders resulted from a previous Delphi-like structured survey among experts from the intensive care field . We will apply the FP1 function selection strategy to the confounders, with FP1 as the most complex permitted functional form. Including all confounders mentioned above, the model will be determined separately for each of the three outcomes using MFP (1.0, 0.05), independent of treatment. In the notation MFP (alpha 1, alpha 2) the value of alpha 1 gives the significance level for the variable selection part of MFP and alpha 2 the significance level of the function selection procedure for continuous variables . Despite some imbalances in the covariate distributions between PEEP groups mentioned above, univariate approaches will be conducted as sensitivity analyses.
To circumvent the issue of influential points all continuous variables will be truncated at the 1% and 99%-tile; meaning that any value below the 1%-tile will be replaced by the value of the 1%-tile, and any value above the 99%-tile will be replaced by the value of the 99%-tile. These truncations will be performed for each study separately.
Handling missing data
Some of the potential modifiers and variables used for adjustment (see below) have missing values of up to about 10%. In order to use all information in all analyses, we will impute missing values before the main analysis starts. To try to ensure that the missing at random assumption is valid, we will include all outcomes and as many other variables as possible in the imputation models . Five imputations will be created using the multiple imputations by chained equations technique. Only the first imputation will be used in analyses. The remaining four will be reserved for sensitivity analysis of the main findings.
Description of outcomes and effect modifiers
We selected three clinical important outcomes of interest from a larger list of outcomes used in the analysis by Briel et al. :
In-hospital mortality at 60 days post randomization (outcome 1a) constitutes the primary efficacy outcome of interest. We will also consider in-hospital mortality as a time-to-event variable (outcome 1b) because we are additionally interested in the timing of mortality events in the randomized groups. Due to the differential follow-up across RCTs beyond day 60 and the fact that the intervention effects happen mainly within the first month, we will censor all surviving patients in the time to event analysis at day 60 as done in the original IPD meta-analysis.
Time to unassisted breathing (outcome 2), which is defined as time from randomization until breathing without mechanical support within the first 28 days is the secondary efficacy outcome of interest. Due to differential follow-up across RCTs for this outcome beyond day 28 and the fact that the intervention effect is supposed to happen before day 28, we will censor patients at day 28 as done in the original IPD meta-analysis. Patients who die before achieving unassisted breathing within the first 28 days will be censored at the day of death. With this procedure we circumvent the competing risk issue in the analysis of this outcome. We are aware of the fact that for prognostic questions, which will not be part of this analysis, cumulative incidence functions would be preferred.
Pneumothorax requiring chest tube drainage in first 28 days after randomization (outcome 3, binary variable) is the main safety outcome, because it captures the main potential adverse event directly associated with higher PEEP (experimental intervention). Again, the reason for choosing a 28-day period is that the follow-up for this outcome is different across included trials beyond day 28 and the intervention effect is supposed to happen within the first 28 days. In the present protocol we will not analyze outcome 3 (main safety outcome) because of competing risks with mortality . In the planned clinical report of this work we will refer to the results of the original IPD meta-analysis with respect to outcome 3, because the MFPI methodology has still to be adapted for a competing risk framework. We will deal with competing risks in an addition to this protocol. For the specified efficacy outcomes (outcomes 1a/b and 2) we anticipate no competing risk problems when using cause-specific Cox models.
The following four continuous potential effect modifiers were all pre-specified by Briel et al. :
Body Mass Index (BMI) at baseline
The BMI is calculated by the ratio of mass in Kg/(Height in m)2. There are no data that suggest a certain direction of the treatment effect modification, but Briel et al. hypothesized less benefit of higher PEEP in patients with higher BMI .
Respiratory compliance (RC) at baseline
The RC is estimated by the ratio of:
A lower RC would reflect more severe lung injury. Briel et al. hypothesized that patients with lower RC have more recruitable lung units and would therefore benefit from higher levels of PEEP. In addition, one could argue that in patients suffering from most severe ARDS, which is commonly associated with very low RC, higher PEEP might no longer provide any benefit.
PaO2/FiO2 ratio at baseline
A low PaO2/FiO2 reflects impaired blood oxygenation and, therefore, more severe lung injury. Similar to RC, Briel et al. hypothesized that patients with a lower PaO2/FiO2 ratio benefit more from higher PEEP levels. It will be interesting to see how the widely accepted ARDS defining cut-off at 200 mmHg, is reflected in this analysis using the MFPI approach. Using this cut-off, a significant interaction was found by Briel et al. [5, 6].
Oxygenation Index at baseline
The oxygenation index (OI) defined as:
includes the mean airway pressure and can be regarded as the more reliable marker regarding blood oxygenation compared to the PaO2/FiO2 ratio alone. The higher the OI, the more severe the lung injury; therefore, Briel et al. hypothesized that patients with a higher OI benefit more from higher PEEP levels .
Further candidates (for example, age and sex) may be additionally investigated for interaction. Of note, irrespective of the results, all investigations will be included in a summary table similar to the REMARK profile for prognostic studies .
The ICEM study is the first example that combines estimation of treatment effect functions by using MFPI with a new approach for a meta-analysis of functions for a clinically relevant issue. The approach requires IPD data, which are available from an earlier meta-analysis project. The present article is an extended version of the registered protocol, and shows in an exemplary way how to better use the information from continuous variables if individual patient data from several RCTs are available. In similar projects, it should be obvious how to adapt the relevant steps for a meta-analysis of treatment effect functions. Besides the new application of the MFPI approach in meta-analysis, the available dataset from three RCTs also offers a unique opportunity to identify potential clinically important interaction effects. All these interaction analyses are exploratory in nature; however, they use the full information for a potential treatment modifier and may help in clinical decision-making. We hope that this project will also foster future research using the MFPI approach to improve interaction analyses of continuous predictors in RCTs and in meta-analyses, provided IPD are available.
Acute respiratory distress syndrome
Body mass index
Fraction of inhaled oxygen
Investigation of continuous effect modifiers
Individual patient data
Multivariable fractional polynomial
Multivariable fractional polynomial interaction
Partial arterial pressure of oxygen
Positive end-expiratory pressure
Randomized clinical trial.
Royston P, Altman DG, Sauerbrei W: Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006, 25: 127-141. 10.1002/sim.2331.
Royston P, Sauerbrei W: A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials. Stat Med. 2004, 23: 2509-2525. 10.1002/sim.1815.
Royston P, Sauerbrei W: Interactions between treatment and continuous covariates: a step toward individualizing therapy. J Clin Oncol. 2008, 26: 1397-1399. 10.1200/JCO.2007.14.8981.
Sauerbrei W, Royston P: A new strategy for meta-analysis of continuous covariates in observational studies. Stat Med. 2011, 30: 3341-3360. 10.1002/sim.4333.
Briel M, Meade M, Mercat A, Brower RG, Talmor D, Walter SD, Slutsky AS, Pullenayegum E, Zhou Q, Cook D, Brochard L, Richard JC, Lamontagne F, Bhatnagar N, Stewart TE, Guyatt G: Higher vs lower positive end-expiratory pressure in patients with acute lung injury and acute respiratory distress syndrome: systematic review and meta-analysis. J Am Med Assoc. 2010, 303: 865-873. 10.1001/jama.2010.218.
Briel M: Higher versus lower positive end-expiratory pressure in patients with acute lung injury: a systematic review and individual patient data meta-analysis (study protocol). 2008, McMaster University, Department of Clinical Epidemiology and Biostatistics
Brower RG, Lanken PN, MacIntyre N, Matthay MA, Morris A, Ancukiewicz M, Schoenfeld D, Thompson BT: National Heart, Lung and Blood Institute ACTN: higher versus lower positive end-expiratory pressures in patients with the acute respiratory distress syndrome. N Engl J Med. 2004, 351: 327-336.
Meade MO, Cook DJ, Guyatt GH, Slutsky AS, Arabi YM, Cooper DJ, Davies AR, Hand LE, Zhou Q, Thabane L, Austin P, Lapinsky S, Baxter A, Russell J, Skrobik Y, Ronco JJ, Stewart TE, Lung Open Ventilation Study Investigators: Ventilation strategy using low tidal volumes, recruitment maneuvers, and high positive end-expiratory pressure for acute lung injury and acute respiratory distress syndrome: a randomized controlled trial. J Am Med Assoc. 2008, 299: 637-645. 10.1001/jama.299.6.637.
Mercat A, Richard JC, Vielle B, Jaber S, Osman D, Diehl JL, Lefrant JY, Prat G, Richecoeur J, Nieszkowska A, Gervais C, Baudot J, Bouadma L, Brochard L, Expiratory Pressure (Express) Study Group: Positive end-expiratory pressure setting in adults with acute lung injury and acute respiratory distress syndrome: a randomized controlled trial. J Am Med Assoc. 2008, 299: 646-655. 10.1001/jama.299.6.646.
Royston P, Sauerbrei W: Multivariable Model-building. A Pragmatic Approach to Regression Analysis Based on Fractional Polynomials for Modelling Continuous Variables. 2008, Chichester: John Wiley & Sons
Royston P, Sauerbrei W, Ritchie A: Is treatment with interferon-alpha effective in all patients with metastatic renal carcinoma? A new approach to the investigation of interactions. Br J Cancer. 2004, 90: 794-799. 10.1038/sj.bjc.6601622.
Beck DH, Smith GB, Pappachan JV, Millar B: External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive Care Med. 2003, 29: 249-256.
Suistomaa M, Niskanen M, Kari A, Hynynen M, Takala J: Customized prediction models based on APACHE II and SAPS II scores in patients with prolonged length of stay in the ICU. Intensive Care Med. 2002, 28: 479-485. 10.1007/s00134-002-1214-9.
White IR, Royston P, Wood AM: Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011, 30: 377-399. 10.1002/sim.4067.
Wolkewitz M, Beyersmann J, Gastmeier P, Schumacher M: Modeling the effect of time-dependent exposure on intensive care unit mortality. Intensive Care Med. 2009, 35: 826-832. 10.1007/s00134-009-1423-6.
Altman DG, McShane LM, Sauerbrei W, Taube SE: Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): explanation and elaboration. PLoS Med. 2012, 9: e1001216-10.1371/journal.pmed.1001216.
We would like to thank all investigators from the original randomized controlled trials included in this individual patient data meta-analysis (namely Maureen Meade, Alain Mercat, Roy G Brower, Arthur S Slutsky, Deborah Cook, Gordon H Guyatt, Laurent Brochard, Jean-Christophe M Richard, and Thomas E Stewart) for their continuous support and collaboration.
The article processing charge was funded by the German Research Foundation (DFG) and the Albert Ludwigs University Freiburg in the funding program Open Access Publishing.
This project has no specific funding. MB is supported by Santésuisse and the Gottfried and Julia Bangerter-Rhyner Foundation. The funding sources have no role in the design and conduct of this study and the writing of this manuscript.
BK, WS, PR, and MB have designed the study and written the registered protocol and this manuscript. WS and PR developed the MFPI and the meta-analysis approach to combine several functions across studies. MB provided the database. BK, WS, and PR will conduct the analyses. All authors approved the final version before submission.
Authors’ original submitted files for images
About this article
- Fractional polynomials
- Interaction analysis
- Continuous predictors
- Individual patient data meta-analysis
- Acute respiratory distress syndrome