Background
Acute respiratory distress syndrome (ARDS) remains a life-threatening disease with mortality up to 55% [
1], and supportive care by prone positioning [
2] and protective lung ventilation [
3] improve survival. However, even with nitric oxide inhalation, or intravenous steroids, many patients still suffer from critical arterial hypoxaemia and hypercarbia [
4].
Extracorporeal membrane oxygenation (ECMO) therapy has received substantial and renewed interest due to improvements in ECMO technology and safety, with increased survival suggested by the CESAR trial, favourable outcome in A(H1N1) influenza-evoked ARDS, and indication of avoidance of substantial lung and organ injury with early ECMO [
5‐
8]. As a result, ECMO is now implemented more frequently and in a broader spectrum of patients.
However, ECMO demands specialized personnel and techniques, and can be associated with severe complications. Therefore, appropriate patient selection based on outcome prediction by scoring may help in decision-making. Several scoring systems have been proposed such as the ECMOnet-Score, developed for risk stratification in H1N1 pneumonia [
6], the RESP-Score including data from more than 2000 patients [
9], the Predicting Death for Severe ARDS on VV-ECMO (PRESERVE)-Score [
10], the Roch-Score [
11], and the Enger-Score [
12].
However, limitations of these previous scores include usage of different ECMO technologies and procedures, patient heterogeneity, and also statistical methods which did not always take into account optimum validation of results [
13,
14]. Moreover, for practical usefulness, any scoring system requires both categorization of risk variables and a limitation in the number of variables. In any case, external validation of published scores is mandatory before any scoring system can generally be accepted.
Accordingly, the aim of this study was to conduct an independent validation of the categorical ECMOnet-Score, the RESP-Score, the Roch-Score, and the PRESERVE-Score to investigate their usefulness in predicting survival in a single ECMO centre. Furthermore, we developed a novel and easy-to-use categorical score based on pre-ECMO clinical data, the PREdiction of Survival on ECMO Therapy-Score (PRESET-Score), to be used to facilitate decision-making and validated this score in two independent validation cohorts.
Methods
Study design
This non-interventional study was performed in agreement with the ethical principles and standards of the second Helsinki declaration and its later amendments. This retrospective study did not need ethical approval, which was waived by the ethics committee of the University Hospital Essen (AZ 15-6729-BO), as our data analysis precluded possible interference between this prospective observational study and the decisions regarding the patients’ clinical management.
Derivation cohort
Data were collected from 108 consecutive ARDS patients receiving vv-ECMO therapy in our intensive care unit (ICU) at the University Hospital Essen, Germany, between 30 December 2009 and 10 February 2015.
Internal validation cohort
Data were prospectively collected from a further 82 consecutive ARDS patients receiving vv-ECMO therapy in the ICU of the University Hospital Essen, Germany, between 11 February 2015 and 9 January 2017.
External validation cohort
Data were collected from 59 ARDS patients receiving vv-ECMO therapy in the ICU of the Marienhospital Osnabrück, Germany, between 23 February 2013 and 26 December 2015.
ECMO therapy regime
After referral of patients for ECMO therapy, a conservative treatment protocol consisting of prone positioning, optimized ventilator settings, and exclusion of reversible conditions (pneumothorax) was performed. Each patient in the derivation and the validation groups was ventilated with biphasic positive airway pressure (BIPAP) prior to ECMO initiation. ECMO was initiated at the referring hospital by our retrieval team if transport without extracorporeal support was not feasible. No single cut-off value was used to make this decision. ECMO was initiated when hypoxaemia and/or profound hypercarbia with haemodynamic instability persisted despite optimized treatment, as reported previously [
8]. Contraindications for vv-ECMO were cardiogenic shock and terminal pulmonary disease with no prospect for lung transplantation in the near future. Eighty-six per cent of patients were cannulated at the referring hospital and transported with ECMO.
vv-ECMO was initiated by performing percutaneous cannulations, preferably bi-femoral venous cannulation, with a minimum distance of 15 cm between cannulae tips to prevent recirculation of oxygenated blood. The circuit configuration was as follows: HLS cannulae (19–25 F depending on patient size) (Maquet, Getinge Group, Rastatt, Germany), heparin-coated (Bioline) tubing (Cardiohelp HLS Set 7.0; Maquet), centrifugal pump, oxygenator, and heater (HU35; Maquet).
At the beginning of ECMO treatment the ECMO pump speed was set to achieve maximum flow. A Swan–Ganz catheter was placed in all patients on admission to the ICU. After determination of cardiac output, the target ECMO flow was set in order to keep the SaO
2 over 90% and the ratio of ECMO flow to cardiac output above 60%, which is supposed to be sufficient for adequate blood oxygenation [
15]. Initially, 100% oxygen was used as the sweep gas. The sweep gas flow was adjusted to maintain normocapnia with protective ventilation.
Data collection
Demographic and clinical characteristics as well as haemodynamic, respiratory, and physiologic variables were obtained immediately before initiation of ECMO therapy and data for hospital survival were recorded. The ECMOnet-Score [
6], RESP-Score [
9], PRESERVE-Score [
10], and Roch-Score [
11] were calculated according to the original publications.
Statistical analysis
Data were tested for normal distribution using the Kolmogorov–Smirnov and Shapiro–Wilk tests. Continuous variables were presented as mean ± standard deviation (SD) in the case of normally distributed data or as median (MD), first quartile (Q1), and third quartile (Q3) in the case of not normally distributed data. Analysis was carried out using Student’s
t test or the Mann–Whitney
U test, as appropriate. Categorical variables were recorded, frequency percentages were calculated, and the χ
2 test was used for these analyses. Each score’s discriminatory power was assessed by calculating the area under the receiver operating characteristic (ROC) curve (AUC), and calibration was evaluated using the Hosmer–Lemeshow goodness-of-fit
C test [
16]. In this test,
p > 0.05 suggests good calibration. Comparison of ROC curves was carried out using the method described by Hanley and McNeil [
17].
Variables relating to patients, diagnoses, or associated organ dysfunction prior to ECMO initiation were considered. Univariate comparison of all variables in survivors and non-survivors was undertaken. Variables with
p ≤ 0.1 in univariate analysis were entered into a backward stepwise binary logistic regression model to identify candidate variables for inclusion in the PRESET-Score after exploration of linearity. Non-significant variables were removed at each step. Linear variables were converted into categorical variables based on Wald statistics for weighting variables, with groups of similar quantity using the relative contribution of each beta parameter [
1].
In the “internal validation set”, binary logistic regression was used to reassess the PRESET-Score performance in an independent data internal dataset. Model discrimination and calibration were assessed using the area under the ROC curve and the Hosmer–Lemeshow C statistic with associated p value, respectively.
The “external validation set” consisted of a completely independent cohort after building the PRESET-Score from the derivation cohort. The external validation of the PRESET-Score was performed on a dataset of 59 patients from the Marienhospital Osnabrück, Osnabrück, Germany (external validation cohort).
Statistical analyses were performed using SPSS (V 22.0; SPSS Inc., Chicago, IL, USA).
Discussion
Acute respiratory distress syndrome (ARDS) is frequently fatal and ECMO is currently considered over a wide range of indications from a last therapeutic resort to a protective and perhaps even “prophylactic” therapy. Accordingly, there is much discussion and controversy about the indications/contraindications of ECMO and the time of initiation. In any case, prognostic systems should enable outcome prediction of such a therapy and the ECMOnet-Score, RESP-Score, PRESERVE-Score, and ROCH-Score have all served to improve such a prediction.
However, for any prognostic system to be generally applicable, it is essential to validate this system in at least one independent cohort.
We therefore not only evaluated these latter previously proposed predictive scores, but also generated and validated a new score, based on pre-ECMO clinical variables, the PRESET-Score, and prospectively validated this score in two independent cohorts.
Interestingly, in the context of this new score derived from clinical variables immediately before ECMO initiation, only extrapulmonary variables were identified as predictors, namely mean arterial blood pressure, platelet concentration, pHa, lactate concentration, and hospital stay before ECMO therapy. Of note, respiratory variables themselves were not predictive of survival.
Platelet concentration
As does the Sepsis-related Organ Failure Assessment (SOFA) score [
18], our data support the independent prognostic value of the platelet count. In our patients, a decrease in platelets by 100,000 μl
–1 increases mortality by 30%.
This result is supported by similar observations in critically ill patients and ECMO patients [
19‐
21].
In this context, it should be noted that there may be an effect on the platelet count by the high number of potentially immunocompromised patients. While interaction of these two factors cannot be ruled out, multivariate analysis including presumed immunocompromised patients as a covariate demonstrated a higher weighting of “platelet count” and, therefore, remained in the final model.
Lactate concentration
Lactate concentration is an established prognostic marker in ICU patients, and a concentration ≥ 4 mmol l
–1 at admission increased mortality by sixfold within the first 3 days [
22]. Comparable results were seen in 830 patients with severe sepsis admitted to an emergency department of a tertiary-care academic centre [
23].
Concerning ECMO therapy, lactate has been shown to independently predict mortality when measured before ECMO initiation [
24].
These results are in line with our findings. In our cohort, an initial lactate concentration ≥ 4 mmol l–1 was associated with a 5.7 times greater mortality, with an increase by 1 mmol l–1 increasing mortality by 30%.
pHa
A multi-centre database comprising 1473 adult ARDS patients with ECMO therapy demonstrated a significant influence of pH
a on outcome [
25], with a median pH
a of 7.29 in survivors, but 7.26 in non-survivors. Moreover, pH < 7.18 was associated with a 2.5-fold increased mortality compared to a subgroup with pH
a > 7.36. In our study, a decrease in pH by 0.1 was found to be associated with an increase in mortality by 40%.
Hospital stay pre ECMO
The timing of ECMO initiation was and is a matter of debate. In a joint study by a French hospital and two Australian hospitals [
26] the time from ICU admission to ECMO initiation was an independent predictor of death, and this was confirmed by a Swiss study [
27]. In our study, each additional hospital day before ECMO initiation was associated with a 10% increase in mortality. One possible explanation might be that any ventilator day before ECMO may increase lung trauma and promotes multiple organ failure.
Mean arterial pressure
While intuitively an association between decreased arterial pressure and mortality may be assumed, various recent studies have not confirmed such a specific correlation [
28,
29]. Nevertheless, targeting a decreased arterial pressure is associated with an increased incidence of acute kidney injury [
30,
31].
In our cohort, average mortality of 62% was greater when compared to other prediction models—that is, 32–56% in the studies comprising the ECMOnet-Score [
6], ENGER-Score [
12], PRESERVE-Score [
10], RESP-Score [
9], and ROCH-Score [
11].
However, an unreflecting direct comparison of the mortality rates in ECMO patients in the different studies may neglect a bias since neither the practical implementation of ECMO nor the indication for its initiation is subject to a specific internationally or nationally standardized protocol.
One possible explanation for the lower survival rate of 38% may be due to ECMO initiation in patients who may have been excluded from ECMO therapy or are associated with a poor outcome when ELSO criteria [
32] or other criteria [
33] would be taken into account. For example, 41 (38%) patients in whom ECMO therapy was initiated may be considered immunocompromised, as suggested by their past medical history (see Table
1).
Our institution traditionally employed liberal inclusion criteria even in patients with a high probability of non-survival. On the other hand, a large proportion of patients explicitly referred to our centre for ECMO therapy were salvaged without ECMO. Thus, we feel that patients treated on ECMO at our centre tend to be rather sick. While this aggressive approach provides a potential rescue therapy for very sick patients with a low chance of survival, it consequently results in ECMO therapy at our centre being associated with a less positive outcome.
These considerations are reflected by higher SOFA score and SAPS II in the patients of the present study, which correlated well with mortality in different studies and settings [
34,
35]. In fact, cohorts from other prognostic models as well as the external validation cohort had a much lower SOFA score and SAPS II in comparison to our patients, potentially explaining different survival rates (Table
5). Differences in the patients’ conditions, therefore, might also be an explanation for a poor performance of the other prediction models in our cohorts. Another reason could be that compared to the other studies—except the ECMOnet-Score, which only refers to the very specific H1N1 population and included significantly healthier patients—we identified as predictors only extrapulmonary variables which in addition to ARDS detrimentally influence outcome.
All 108 ARDS patients were treated with an identical ECMO system (Cardiohelp®; Maquet, Getinge Group) which was only approved in 2009, so that patients are well matched and good data and results could be achieved despite the limited sample size of our derivation group. Compared to the existing prediction scores derived from other studies, we included a similar number of patients (ECMOnet-Score, 60 patients; Roch-Score, 85 patients; PRESERVE-Score, 140 patients) and each patient received similar treatment regimens. This is a strength of the present study since avoiding technical heterogeneity makes results very comparable.
Furthermore, we had a heterogeneous patient population with a wide range of disease patterns. Thus, the novel PRESET-Score is likely applicable to a broad range of ARDS-evoked ECMO candidates and not limited to a specific underlying disease.
In contrast, the ECMOnet-Score reflects a young and more homogeneous population (82% H1N1 patients) and there were various contraindications, potentially creating a bias through exclusion of patients with a poor prognosis. Interestingly, subanalysis of our patients with H1N1 only demonstrated a better discrimination using the ECMOnet-Score, indicating that the ECMOnet-Score may be more appropriate in cohorts with H1N1 patients.
The potential practical usefulness of the new score should be emphasized. We created and validated in two independent cohorts an easy-to-use score covering a broad range of ARDS patients. In the derivation cohort, an ICU mortality of 93% was found in patients with a PRESET-Score ≥ 10 (risk class III; Fig.
1a). Furthermore, the internal validation cohort showed an ICU mortality of 91% in the same risk class, which raises the question of whether initiation of ECMO therapy in these patients is still reasonable.
On the other hand, patients with a low (≤5) PRESET-Score showed a 74% survival rate in the derivation cohort (risk class I; Fig.
1a) and 86% in the internal validation cohort, respectively, assuming that these patients are good candidates for ECMO therapy when conventional therapy is unsuccessful. Here, the PRESET-Score might indeed serve as a good tool in decision-making, especially when resources are limited.
Limitations
This is a retrospective study performed at a single medical centre. Therefore, the results and the prognostic relevance of the PRESET-Score may not be directly applicable to other institutions harbouring patients with different ARDS aetiologies.
The aim of our study was the prediction of mortality before ECMO initiation and no extended follow-up addressing 6-month/long-term survival, quality of life, or permanent disability was conducted. This may be considered a limitation.
Furthermore, although the PRESET-Score is a useful prediction model, it should only supplement individual judgement based on history, condition, prognosis, and the assumed living will of any specific patient. Beyond scores, weighing treatment options requires experienced physicians but our score serves as an additional block to build and facilitate decision-making.