A novel time series analysis approach for prediction of dialysis in critically ill patients using echo-state networks
© Verplancke et al; licensee BioMed Central Ltd. 2010
Received: 2 July 2009
Accepted: 21 January 2010
Published: 21 January 2010
Echo-state networks (ESN) are part of a group of reservoir computing methods and are basically a form of recurrent artificial neural networks (ANN). These methods can perform classification tasks on time series data. The recurrent ANN of an echo-state network has an 'echo-state' characteristic. This 'echo-state' functions as a fading memory: samples that have been introduced into the network in a further past, are faded away. The echo-state approach for the training of recurrent neural networks was first described by Jaeger H. et al. In clinical medicine, until this moment, no original research articles have been published to examine the use of echo-state networks.
This study examines the possibility of using an echo-state network for prediction of dialysis in the ICU. Therefore, diuresis values and creatinine levels of the first three days after ICU admission were collected from 830 patients admitted to the intensive care unit (ICU) between May 31th 2003 and November 17th 2007. The outcome parameter was the performance by the echo-state network in predicting the need for dialysis between day 5 and day 10 of ICU admission. Patients with an ICU length of stay <10 days or patients that received dialysis in the first five days of ICU admission were excluded. Performance by the echo-state network was then compared by means of the area under the receiver operating characteristic curve (AUC) with results obtained by two other time series analysis methods by means of a support vector machine (SVM) and a naive Bayes algorithm (NB).
The AUC's in the three developed echo-state networks were 0.822, 0.818, and 0.817. These results were comparable to the results obtained by the SVM and the NB algorithm.
This proof of concept study is the first to evaluate the performance of echo-state networks in an ICU environment. This echo-state network predicted the need for dialysis in ICU patients. The AUC's of the echo-state networks were good and comparable to the performance of other classification algorithms. Moreover, the echo-state network was more easily configured than other time series modeling technologies.
Construction of the echo-state network
The AUC results for the three compared methods (ESN, SVM and NB) were calculated using a 10-fold cross-validation. In each of the different methods, the same folds were used. The AUC results obtained by the echo-state network were then compared with the AUC results of the SVM classifier and the Naive Bayes (NB) algorithm by a non-parametric statistical test  within SAS version 9.1.3 (macro %roc). A Dunn-Sidak correction  for multiple testing was performed on the obtained p-values.
AUC's for the three test runs with their respective 95% CI and Dunn-Sidak corrected p-values as statistical difference in comparison with the ESN performance: ESN as reference (ref.) algorithm.
95% CI AUC
p-value (ESN = ref.)
This is the first study to investigate the clinical application of echo-state networks for classification in large ICU databases. In general, it is non-trivial to model time series data with classical statistical techniques such as longitudinal data analysis, due to the high degree of correlation within the data. In recent years there has been an evolution towards the development of risk-prediction models that use daily assessment of organ function to evaluate the patient status, and thus incorporate already a certain degree of time dependency . Echo-state networks are specifically designed for the analysis of time series. Other algorithms such as Hidden Markov modeling or dynamic time warping are outside of the scope of this study, but can be suitable alternatives for time series analysis as are methods like functional data analysis and survival analysis methods with consideration of competing risks. The presence of time series in the ICU is ubiquitous and hence the number of possible future ICU applications for this technology are hudge. Echo-state networks have successfully been employed for numerous prediction problems in telecommunication research  and robotics , as well as in linguistics to detect grammatical structure . Most of these applications come down to prediction of future states of a time series. In this study however, the basic echo-state network architecture is being adapted so that not only prediction by the network of future states is possible, but finding solutions to classification problems becomes possible too. It is noticed that the results from the SVM and NB are slightly better than the results obtained by the echo-state network. All AUC's were above 0.8 and clinically acceptable. The time series modeling process in itself was much harder to realize for the SVM and NB, which are not easily configured for time series analysis applications, in contrast to the developing of the echo-state network which is perfectly suitable for time series analysis and therefore relatively easily configured. To be able to input time series in NB and SVM, preprocessing of the data is needed by extracting non-correlated data out of the time series. This preprocessing step needs not to be performed in the echo-state network configuration. The NB and SVM algorithms needed a much longer computation time than the ESN method. These are all clear advantages in favour of the echo-state network approach. It can therefore be concluded that ESN perform well at the task at hand. As a limitation of the study, we can state that no competing risk analysis for competing events (e.g. discharge, death, dialysis before day 5) was performed relating to the more general problem of missing data as seen in other survival analysis methods. The results obtained in this study can be considered as a proof of concept for the use of reservoir computing methods in the ICU. It is clear for every clinician working in an ICU environment that possible future applications for this new data modeling method are amply found: there are a vast number of continuously monitored physiological variables retrieved at the bedside that have time series characteristics. Just to name a few, haemodynamic parameters, ventilatory settings and consecutively retrieved blood samples, are all potential candidates for time series analysis through an echo-state network approach in the ICU. Till now, most of the dynamical and thus time-dependent features of these patient variables were lossed during the modeling process of ICU databases, in spite of the fact that analysis of the trend of physiological data are of vital importance in an ICU environment. The fact that now and in the near future advanced dynamical modeling capabilities through novel technologies such as these described in this study will become possible in clinical practice, is a thrilling evolution for every clinician caring for the welfare of his patients.
This proof of concept study evaluated the performance of echo-state networks for the first time in predicting the need for dialysis in an ICU population. The classification performance of the echo-state network was good. Moreover, the echo-state network was easily configured compared to SVM and NB modeling techniques, and the echo-state network needed much less computation time. Since time series data in the ICU are amply available and since the modeling of ICU time series data with regression techniques are more difficult due to the problem of high correlation within the data, the authors state that ESN might contribute to the development of future modeling methods of ICU databases.
The heuristic behind the SVM algorithm is quite different from that of the commonly used logistic regression modeling for prediction. This latter approach is the golden standard for prognostic modeling in the ICU and is best known by clinicians. The LR algorithm uses a weighted least squares algorithm, i.e. the prediction is based on construction of a regression line as the best fit through the data points by minimizing a weighted sum of the squared distances to the fitted regression line. SVM, in contrast, tries to model the input variables by finding the separating boundary - called hyperplane - to reach classification of the input variables: if no separation is possible within a high number of input variables, the SVM algorithm still finds a separation boundary for classification by mathematically transforming the input variables and thereby increasing the dimensionality of the input variable space. The general term for a separating straight line in a high-dimensional space is a hyperplane. Moreover, statistical learning theory predicts that the SVM algorithm will find the hyperplane with the maximum-margin to the nearest data point on either side of the hyperplane.
b. Naive Bayes algorithm
Bayesian theory and Bayesian probability are named after Thomas Bayes, a British eighteenth century mathematician. Bayesian logic combines the result of a test for a particular patient with a pre-test probability (of the population), to forecast or determine the chance of finding a disease: clinicians intuitively combine these two probabilities routinely. Bayesian theory suggests that Bayes' theorem can be used as a rule to infer or update the degree of 'belief' in light of new information (hence the name 'belief networks'). Bayesian networks can be seen as an alternative to logistic regression models where statistical dependence or independence between different variables are explicitly formulated and not hidden in the regression coefficients as in logistic regression. In a naive Bayes network, as used in this study, there are no dependencies between the different feature variables, they are thus considered to be conditionally independent, hence the term 'naive'. A nice example of the applicability in classification problems of these naive Bayesian networks is the article by Price et al. for the classification of cercival cancer patients .
Artificial Neural Network
area under the receiver operating characteristic curve
Intensive Care Unit
medical intensive care unit
Length of Stay
naive Bayes algorithm
surgical intensive care unit
support vector machine
- Jaeger H, Haas H: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 2004, 304 (5667): 78-80. 10.1126/science.1091277.View ArticlePubMedGoogle Scholar
- Salmen M, Ploger P: Echo-state networks used for Motor Control. Robotics and Automation. 2005, 18: 1953-1958.Google Scholar
- Tong M, Bickett A, Christiansen E, Cottrell G: Learning grammatical structure with echo state networks. Neural Networks. 2007, 20: 424-432. 10.1016/j.neunet.2007.04.013.View ArticlePubMedGoogle Scholar
- Toma T, Abu-Hanna A, Bosman RJ: Discovery and inclusion of SOFA score episodes in mortality prediction. J Biomed Inform. 2007, 40 (6): 649-660. 10.1016/j.jbi.2007.03.007.View ArticlePubMedGoogle Scholar
- Toma T, Abu-Hanna A, Bosman RJ: Discovery and integration of univariate patterns from daily individual organ-failure scores for intensive care mortality prediction. Artif Intell Med. 2008, 43 (1): 47-60. 10.1016/j.artmed.2008.01.002.View ArticlePubMedGoogle Scholar
- Jaeger H: The echo state approach to analyzing and training recurrent neural networks. (Tech. Rep. No. 148). 2001, Bremen: German National Research Center for Information TechnologyGoogle Scholar
- Rosenblatt F: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev. 1958, 65 (6): 386-408. 10.1037/h0042519.View ArticlePubMedGoogle Scholar
- DeLong ER, DeLong DM, D C-P: Comparing the Areas Under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics. 1988, 44: 837-845. 10.2307/2531595.View ArticlePubMedGoogle Scholar
- Ludbrook J: On making multiple comparisons in clinical and experimental pharmacology and physiology. Clin Exp Pharmacol Physiol. 1991, 18 (6): 379-392. 10.1111/j.1440-1681.1991.tb01468.x.View ArticlePubMedGoogle Scholar
- Le Gall J: The use of severity scores in the intensive care unit. Intens Care Med. 2005, 31: 1618-1623. 10.1007/s00134-005-2825-8.View ArticleGoogle Scholar
- Price GJ, McCluggage WG, Morrison MM, McClean G, Venkatraman L, Diamond J, Bharucha H, Montironi R, Bartels PH, Thompson D: Computerized diagnostic decision support system for the classification of preinvasive cervical squamous lesions. Hum Pathol. 2003, 34 (11): 1193-1203. 10.1016/S0046-8177(03)00421-0.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/10/4/prepub