Support vector machine versus logistic regression modeling for prediction of hospital mortality in critically ill patients with haematological malignancies
 T Verplancke^{1}Email author,
 S Van Looy^{2},
 D Benoit^{1},
 S Vansteelandt^{3},
 P Depuydt^{1},
 F De Turck^{2} and
 J Decruyenaere^{1}
DOI: 10.1186/14726947856
© Verplancke et al; licensee BioMed Central Ltd. 2008
Received: 11 July 2008
Accepted: 05 December 2008
Published: 05 December 2008
Abstract
Background
Several models for mortality prediction have been constructed for critically ill patients with haematological malignancies in recent years. These models have proven to be equally or more accurate in predicting hospital mortality in patients with haematological malignancies than ICU severity of illness scores such as the APACHE II or SAPS II [1]. The objective of this study is to compare the accuracy of predicting hospital mortality in patients with haematological malignancies admitted to the ICU between models based on multiple logistic regression (MLR) and support vector machine (SVM) based models.
Methods
352 patients with haematological malignancies admitted to the ICU between 1997 and 2006 for a lifethreatening complication were included. 252 patient records were used for training of the models and 100 were used for validation. In a first model 12 input variables were included for comparison between MLR and SVM. In a second more complex model 17 input variables were used. MLR and SVM analysis were performed independently from each other. Discrimination was evaluated using the area under the receiver operating characteristic (ROC) curves (± SE).
Results
The area under ROC curve for the MLR and SVM in the validation data set were 0.768 (± 0.04) vs. 0.802 (± 0.04) in the first model (p = 0.19) and 0.781 (± 0.05) vs. 0.808 (± 0.04) in the second more complex model (p = 0.44). SVM needed only 4 variables to make its prediction in both models, whereas MLR needed 7 and 8 variables in the first and second model respectively.
Conclusion
The discriminative power of both the MLR and SVM models was good. No statistically significant differences were found in discriminative power between MLR and SVM for prediction of hospital mortality in critically ill patients with haematological malignancies.
Background
Rationale
This study investigates the use of a support vector machine based classification model for determining the prognosis of ICU patients with haematological malignancies by comparing it with a logistic regression based classification model. The main goal of this article is to be a proof of concept for the use of SVM technology in the ICU, rather than to develop a predictive or prognostic model. Discrimination will be studied for both the MLR and the SVM classification models.
Methods
Data collection
Initial 12 input variables for model 1 before start of MLR and SVM modeling process and their descriptive statistics for the training and validation data sets.
Input variable  Training  Validation 

gender, % male  58  65 
age, yrs  55 (± 18)  58 (± 15) 
% highgrade malignancy  61  54 
% active disease of relapse  34  39 
% allogeneic bone marrow transplant./stem cell transplant.  13  10 
weeks since BMT, median (IQR)*  15 (54)  8 (102) 
% chemotherapy<3 we since ICU admission  41  52 
days of hospitalisation before ICU admission, median (IQR)  4(16)  6(16) 
% bacterial infection  44  43 
APACHE II score  24.5 (± 7.4)  25 (± 7.4) 
% ventilated on day 1  49  46 
% vasopressor need on day 1  41  49 
Initial 17 input variables for model 2 before start of MLR and SVM modeling process and their descriptive statistics for the training and validation data sets.
Input variable  Training  Validation 

age, yrs  55 (± 18)  58 (± 15) 
% highgrade malignancy  61  54 
% active disease of relapse  34  39 
% allogeneic bone marrow transplant./stem cell transplant.  13  10 
days of hospitalisation before ICU admission, median (IQR)  4 (16)  6 (16) 
% bacterial infection  44  43 
pulse (/min)  123 (± 28)  118 (± 33) 
mean blood pressure (MAP), mmHg  73 (± 27)  69 (± 22) 
respiration frequency (/min)  32 (± 10)  33 (± 13) 
Pa02/Fi02 (p/f)  198 (± 130)  194 (± 126) 
platelets (1000/mm^{3})  125 (± 700)  90 (± 114) 
urea<24 h (g/l)  0.86 (± 59)  0.82 (± 55) 
creatinine<24 h (mg/dl)  1.6 (± 1.08)  1.7 (± 1.7) 
albumin<24 h (g/dl)  2.6 (± 1.97)  2.4 (± 0.70) 
prothrombin time (%)<24 h  56 (± 20.7)  57 (± 19.4) 
% ventilated on day 1  49  46 
% vasopressor need on day 1  41  49 
Development of the logistic regression models for prediction of hospital mortality
MLR model 1: variables retained for final MLR analysis after variable selection process, coefficients, standard errors of the coefficients, odds ratios, 95% confidence intervals (CI) for the odds ratios for the model variables (x), and pvalue.
Variable  Coefficient  SE  Odds Ratio  95% CI  pvalue 

gender*  .636  .305  .530  .292–.962  0.037 
highgrade malignancy  .689  .304  1.992  1.099–3.613  0.023 
active disease  .797  .321  2.218  1.181–4.165  0.013 
bone marrow transplant.  .914  .443  2.495  1.048–5.941  0.039 
bacterial infection  .739  .316  .478  .257–.887  0.019 
APACHE II (per point)  .084  .024  1.088  1.037–1.140  0.001 
ventilation < 24 h  1.221  .323  3.391  1.800–6.388  <0.001 
Constant  2.006  .707  .135  0.005 
MLR model 2: variables retained for final MLR analysis after variable selection, coefficients, standard errors of the coefficients, odds ratios, 95% confidence intervals for the odds ratios for the model variables (x), and pvalue
Variable  Coefficient  SE  Odds Ratio  95% CI  pvalue 

Highgrade malignancy  .670  .324  1.954  1.034–3.690  0.039 
active disease  .850  .328  2.340  1.229–4.456  0.010 
bacterial infection  781  .324  .458  .243–.863  0.016 
thrombocytopenia (<50.000/mm^{3})  .867  .314  2.379  1.287–4.399  0.006 
ventilation < 24 h  1.414  .327  4.111  2.167–7.798  <0.001 
prothrombin time (%)  0.016  .008  .984  .970–1.000  0.045 
PaO2/FiO2 (p/f)  .003  .001  .997  .995–1.000  0.025 
urea < 0.5 g/l (reference)  0.011  
urea 0.5–1 g/l  0.583  .415  1.791  .876–.3.663  0.033 
urea > 1 g/l  1.249  .387  3.486  1.545–7.866  0.085 
Constant  0.457  .716  0.633  0.021 
Development of the support vector machine based models for prediction of hospital mortality
Two SVM models were developed consecutively. The SVM based model building processes were carried out with a modified Java version of the libSVM 2.82 software package available at http://www.csie.ntu.edu.tw/~cjlin/libsvm. For both models the model construction process consisted consecutively of: (i) selection of the input variables (out of the 12 and 17 variables at the beginning of the modeling), (ii) selection of the training parameters (C and γ), (iii) construction of the model, (iv) performance evaluation and finally (v) validation of the model. The method of input variable selection was based on the approach in [15] which was itself based on the recursive feature elimination method as proposed in [16]. The common part in these approaches is that the input variables are ranked by iteratively eliminating the least important input variable in each step. In the approach used in this study, a second ranking is constructed by iteratively adding the most important input variable to the model. The libSVM training algorithm is a stochastic process, meaning that two consecutive runs do not necessary result in identical results. Therefore the rankings were repeated 160 times, after which the median ranking of each input variable was calculated. The last step in the input variable selection process, is to determine the exact number of input variables. In order to fix this number, the performance of the prediction model is estimated for an increasing number of input variables. This results in an initially increasing performance estimation, which after reaching a peak will decrease again. The number of input variables at peak estimated performance determines how much input variables are used in the final model. After the input variables for model construction are known, the model training parameters can be tuned. The SVM with a Gaussian kernel function has two such training parameters: C which controls overfitting of the model, and gamma (γ) which controls the degree of nonlinearity of the model. Gamma is inversely related to sigma which is a degree for spread around a mean in statistics: the higher the value of gamma, the lower the value of sigma, thus the less spread or the more nonlinear the behavior of the kernel. The values of these training parameters C and gamma are determined by grid search and cross validation: the model with the highest estimated performance determines the selected training parameters. Then, the performance of the constructed model is estimated by using 5fold cross validation on the training data. Finally, the constructed model is validated by predicting the validation data and comparing these predictions with the real observations by means of ROC curves.
Comparison between models
Accuracy (ACC), sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV) for model 1 and 2 for prediction of hospital mortality (95%CI)
MLR model1  SVM model1  MLR model2  SVM model2  

ACC  0.730 (0.632–0.814)  0.680 (0.579–0.770)  0.740 (0.643–0.823)  0.680 (0.579–0.770) 
SN  0.740 (0.603–0.850)  0.630 (0.487–0.760)  0.722 (0.583–0.835)  0.630 (0.487–0.757) 
SP  0.717 (0.565–0.840)  0.740 (0.589–0.857)  0.761 (0.612–0.874)  0.739 (0.589–0.857) 
PPV  0.755 (0.617–0.862)  0.740 (0.589–0.857)  0.780 (0.640–0.885)  0.739 (0.589–0.857) 
NPV  0.702 (0.551–0.827)  0.630 (0.487–0.757)  0.700 (0.554–0.821)  0.630 (0.487–0.757) 
Results
Hospital mortality in the training data set was 54.4% and 54.0% in the validation set.
Discussion
The equal distribution of survivors and nonsurvivors in the training and validation data sets makes logistic regression modeling ideal for comparison with other algorithms such as SVM [19]. Both the MLR an SVM models perform well in this study and the small differences between the MLR and SVM results in these data sets were statistically not significant. This is the first study to explore the future clinical use of SVM algorithms for mortality prediction in the critically ill, although an SVM based application has already been described by the authors in an ICU setting for prediction of the tacrolimus blood concentration in post liver transplantation patients [5]. SVM acknowledged fewer variables as significant to make its prediction of hospital mortality: it used only 4 variables for both models. In most ICU databases, there is a high percentage of missing values, making modeling of these data more difficult. If a prediction model uses less variables, the chance of having a high percentage of missing values will be lower with, possibly, a more accurate prediction as a result. For example, in this study, the SVM model 2 only needed 4 input variables namely ventilation < 24 h, bone marrow transplantation, bacterial infection and pulse. From these four variables, three of them are readily available. From this, it can be argued to consider variables for model development which are usually available in most patients, thereby reducing the number of missing values. The use of fewer variables in a SVM prediction model in comparison with a MLR model, was also demonstrated in prior SVM research by the authors [5]. Table 5 demonstrates that the PPV of the MLR and SVM algorithm have similar values, although the NPV for the SVM algorithm is lower than in the MLR model. Worthwhile mentioning is the fact that both the CSSIscore developed by Groeger and coworkers [14, 20] and the more general APACHE II score and SAPS II score, predicted hospital mortality less accurate than the studied MLR and SVM models and this in both the training and validation data sets. This could be expected due to the fact that the tested models in this study were validated on patient data of the same ICU. Indeed, a truly independent validation of the findings of this study should be performed on separate data sets in different ICU's. A frequent problem with risk prediction models, especially prognostic models that have not been recently developed, is the weakening calibration of the model [21]. This problem can be dealt with by implementing a locally developed risk prediction model that can be updated over time, ideally in an automated way, e.g. by retraining the SVM algorithm or other artificial intelligence learning algorithms such as artificial neural networks by excluding one month of data at the start of the time series analysis and adding the last month's data [22]. The conclusion of equivalence in discrimination performance between the MLR and SVM results did not change when validating these methods with a 10fold cross validation, in addition to the train and validation cohort methodology that was reported in the manuscript. The authors only reported the results obtained by a train and validation cohort methodology due to the overall acceptance of this methodology in medical community. In practice, SVM technology could be incorporated – after thorough validation – as an intelligent agent into the intensive care information systems and hence give decision support to the ICU clinician. This implementation of SVM technology in the ICU will be the subject of future research by this study group. While no discriminative model is capable of predicting the outcome of any individual patient and although some studies show the equivalence of the prognostic capacities of ICU clinicians [21] in comparison with the accuracies of risk prediction models, these locally developed models can, when well validated, give the ICU clinicians a perspective from which the care for the individual critically ill patient can only benefit.
Conclusion
The discriminative power of both the MLR and SVM models was good. No statistically significant differences were found in discriminative power between MLR and SVM for prediction of hospital mortality in critically ill patients with haematological malignancies.
Key messages

Logistic regression is still the current standard in ICU prognostic modeling.

New artificial intelligence methods are emerging for classification or prediction purposes in the ICU.

The Support Vector Machine (SVM) algorithm has been proven to be a good classifier and prediction method in diverse scientific research areas.

The accuracy for predicting hospital mortality by SVM is comparable to that of logistic regression prediction.

SVM has the possibility – after further validation – to improve patient care in the near future by facilitating data modeling in the ICU.
Abbreviations
 ACC:

accuracy
 APACHE score:

Acute Physiology and Chronic Health Evaluation score
 AUC:

Area Under the Receiver Operating Characteristic curve
 CSSI:

score Cancer Specific Severity of Illness score
 ICU:

Intensive Care Unit
 LR:

logistic regression
 MLR:

Multiple Logistic Regression
 NPV:

negative predictive value
 PPV:

positive predictive value
 ROC:

Receiver Operating Characteristic curve
 SAPS:

Simplified Acute Physiology Score
 SN:

sensitivity
 SP:

specificity
 SVM:

Support Vector Machine.
Declarations
Authors’ Affiliations
References
 Benoit DD, Vandewoude KH, Decruyenaere JM, Hoste EA, Colardyn FA: Outcome and early prognostic indicators in patients with a hematologic malignancy admitted to the intensive care unit for a lifethreatening complication. Crit Care Med. 2003, 31 (1): 104112.View ArticlePubMedGoogle Scholar
 Boser B, Vapnik V: A training algorithm for optimal margin classifiers. Proceedings of the fifth annual workshop on Computational learning theory. 1992, (Pittsburgh, Pennsylvania, USA), 144152.View ArticleGoogle Scholar
 Cortes C, Vapnik V: Supportvector networks. Machine Learning. 1995, 20 (3): 273297.Google Scholar
 Schölkopf B, Burges C, Vapnik V: Incorporating invariances in support vector learning machines. Proceedings of the 1996 International Conference on Artificial Networks, Bochum, Germany Lecture notes in computer science. 1996, 1112: 4752.Google Scholar
 Van Looy S, Verplancke T, Benoit D, Hoste E, Van Maele G, De Turck F, Decruyenaere J: A novel approach for prediction of tacrolimus blood concentration in liver transplantation patients in the intensive care unit through support vector regression. Crit Care. 2007, 11 (4): R83View ArticlePubMedPubMed CentralGoogle Scholar
 Giraldo B, Garde A, Arizmendi C, Jane R, Benito S, Diaz I, Ballesteros D: Support vector machine classification applied on weaning trials patients. Conf Proc IEEE Eng Med Biol Soc. 2006, 1: 55875590.PubMedGoogle Scholar
 Byvatov E, Schneider G: Support vector machine applications in bioinformatics. Appl Bioinformatics. 2003, 2 (2): 6777.PubMedGoogle Scholar
 Vinayagam A, Konig R, Moormann J, Schubert F, Eils R, Glatting KH, Suhai S: Applying Support Vector Machines for Gene Ontology based gene function prediction. BMC Bioinformatics. 2004, 5: 116View ArticlePubMedPubMed CentralGoogle Scholar
 Hand D, Mannila P, Smyth P: Principles of Data Mining. 2001, MIT Press, Cambridge, MAGoogle Scholar
 Chu A, Ahn H, Halwan B, Kalmin B, Artifon EL, Barkun A, Lagoudakis MG, Kumar A: A decision support system to facilitate management of patients with acute gastrointestinal bleeding. Artif Intell Med. 2007Google Scholar
 Noble WS: What is a support vector machine?. Nat Biotechnol. 2006, 24 (12): 15651567.View ArticlePubMedGoogle Scholar
 Benoit DD, Depuydt PO, Peleman RA, Offner FC, Vandewoude KH, Vogelaers DP, Blot SI, Noens LA, Colardyn FA, Decruyenaere JM: Documented and clinically suspected bacterial infection precipitating intensive care unit admission in patients with hematological malignancies: impact on outcome. Intensive Care Med. 2005, 31 (7): 934942.View ArticlePubMedGoogle Scholar
 Zimmerman JE, Kramer AA, McNair DS, Malila FM: Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006, 34 (5): 12971310.View ArticlePubMedGoogle Scholar
 Groeger JS, Lemeshow S, Price K, Nierman DM, White P, Klar J, Granovsky S, Horak D, Kish SK: Multicenter outcome study of cancer patients admitted to the intensive care unit: a probability of mortality model. J Clin Oncol. 1998, 16 (2): 761770.PubMedGoogle Scholar
 Van Looy S, Meeus J, Wyns B, Cruyssen Vander B, De Keyser F, Boullart L: 'Feature selection in the prediction of Infliximab dose increase'. Proceedings of the 9th IASTED International Conference Artificial Intelligence and Soft Computing. 2005Google Scholar
 Guyon I, Weston J, Barnhill S, Vapnik V: Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3);389–422,2002. 2002, 46 (1–3): 389422.Google Scholar
 Lemeshow S, Le Gall JR: Modeling the severity of illness of ICU patients. A systems update. Jama. 1994, 272 (13): 10491055.View ArticlePubMedGoogle Scholar
 DeLong ER, DeLong DM, ClarkePearson D: 'Comparing the Areas Under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach'. Biometrics. 1988, 44: 837845.View ArticlePubMedGoogle Scholar
 Hosmer D, Lemeshow S: 'Applied Logistic Regression'. Wiley Series in Probability and Statistics. 2000, John Wiley & Sons, Inc, secondGoogle Scholar
 Soares M, Salluh JI, Ferreira CG, Luiz RR, Spector N, Rocco JR: Impact of two different comorbidity measures on the 6month mortality of critically ill cancer patients. Intensive Care Med. 2005, 31 (3): 408415.View ArticlePubMedGoogle Scholar
 OhnoMachado L, Resnic FS, Matheny ME: Prognosis in critical care. Annu Rev Biomed Eng. 2006, 8: 567599.View ArticlePubMedGoogle Scholar
 Zeger SL, Irizarry R, Peng RD: On time series analysis of public health and biomedical data. Annu Rev Public Health. 2006, 27: 5779.View ArticlePubMedGoogle Scholar
 The prepublication history for this paper can be accessed here:http://www.biomedcentral.com/14726947/8/56/prepub
Prepublication history
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.