Mortality prediction of ICU patients using EDA

182

Int. J. Services Operations and Informatics, Vol. 7, Nos. 2/3, 2012

Mortality prediction of ICU patients using EDA-enhanced logistic model Lili Chen, Xi Zhang* and Xiaoyun Xu Department of Industrial Engineering and Management, Peking University, Beijing 100871, China Email: [email protected] Email: [email protected] Email: [email protected] *Corresponding author

Liang Zhao Peking University Third Hospital, Peking University, Beijing 100871, China Email: [email protected] Abstract: Due to the different health conditions of an increasing number of serious patients, the Intensive Care Unit (ICU) of a hospital has to correctly classify patients according to their conditions so that medical resources could be properly utilised. The seriousness of the illness can be classified based on the significant risk factors and its corresponding impacts on the patients’ survival. How to quickly identify the significant variables is a major task for classification. This paper proposes a Multistage-EDA-Enhanced Logistic Regression (MEDAeLR) approach to precisely classify the patients and quickly diagnose with three-stage analysis. A cohort of 200 consecutive ICU patients was borrowed for validation. Regular MLR, classification trees and Linear Discriminant Analysis (LDA) are carried to compare the performance with proposed method. The results show that MEDAeLR provides more satisfactory identification performance in terms of Receiver Operating Characteristic (ROC) curve and Area under the ROC Curve (AUC). Keywords: ICU data; patient classification; MEDAeLR. Reference to this paper should be made as follows: Chen, L., Zhang, X., Xu, X. and Zhao, L. (2012) ‘Mortality prediction of ICU patients using EDAenhanced logistic model’, Int. J. Services Operations and Informatics, Vol. 7, Nos. 2/3, pp.182–196. Biographical notes: Lili Chen is a PhD candidate in the Department of Industrial Engineering and Management, Peking University, China. Her research interests are on process monitoring and optimisation in healthcare service delivery. Xi Zhang is an Assistant Professor in the Department of Industrial Engineering and Management, Peking University, China. He received his PhD degree in Industrial Engineering from the University of South Florida University, Florida. His research interests focus on data-driven process monitoring and diagnoses in complex systems. Copyright © 2012 Inderscience Enterprises Ltd.

Mortality prediction of ICU patients

183

Xiaoyun Xu is an Assistant Professor in the Department of Industrial Engineering and Management, Peking University, China. He received his PhD in Industrial Engineering from Arizona State University, Tempe, Arizona. His research interests include optimisation, algorithm design and discrete event simulation applied for service industries. Liang Zhao is an Associate Professor and Associate Chief Physician in Beijing Third Hospital. His research interests are operation management and informatics for hospital resource. This paper is a revised and expanded version of a paper entitled ‘Statistical modeling and evaluation of the survival data from the discharge of hospital intensive care unit’ presented at ‘2011 IIE Asian Conference’, Shanghai, 10–12 June 2011.

1

Introduction

1.1 Motivation Intensive Care Unit (ICU) has been recognised as one of the most important components in contemporary hospital treatment for many years. It usually integrates special units where the efforts are concentrated in one locality with special care and supervision by skilled personnel. It has been reported that the utilisation of the ICU has functioned efficiently in reducing the expected mortality up to 60% (Takrouri, 2004). Due to the different health conditions of serious patients, the ICU of a hospital has to classify patients according to their healthcare needs so that more patients can receive appropriate care. Serious patients could be correctly classified, provided that significant risk factors and the corresponding impacts on the patients’ survival can be identified. Large numbers of approaches have been proposed to identify patients with severe conditions and likely incurables for optimal uses of medical resource and utilities, and mortality prediction for ICU patients summarised from these approaches has served as one of the most important measures for choosing medical services. These prediction models for patient’s mortality have been widely studied to improve the prediction accuracy at patient’s admission. However, these proposed models are usually complex and require a large amount of historical records as training dataset, and most of them are lack of validations especially when patients’ historical ICU data are limited. Meanwhile, because of patients’ diverse conditions in different regions (Yaguchi et al., 2005), there is no widely accepted approach that is well suited for precise mortality prediction. Therefore, it is desirable to develop an efficient analysis and modelling strategy for ICU patients under this concern.

1.2 Literature review Two types of strategies, namely: multiple utility strategy and actuarial strategy have been developed in literature to solve the mortality prediction problem. The actuarial strategy uses statistical modelling including classification and prediction approaches, to relate empirical data for many patient variables to outcomes of interest (e.g. mortality). In this study, actuarial strategy is adopted for identifications of patient’s conditions in terms of statistical prediction and classification models.

184

L. Chen et al.

Several classification and prediction models have been purposed in the literature to identify ICU patients. The most popular mortality prediction models include multiple versions of the Acute Physiology and Chronic Health Evaluation (APACHE, APACHE II, III and IV) (Knaus and Zimmerman, 1981; Knaus et al., 1985; Knaus et al., 1991; Zimmerman et al., 2006), the Simplified Acute Physiology Systems (SAPS and SAPS II) (Le Gall et al., 1984; Le Gall et al., 1993) and Mortality Prediction Model (MPM0) (Lemeshow et al., 1988). SAPS II and APACHE II are commonly regarded as the two most popular mortality scoring systems that are capable of obtaining the probability of mortality. APACHE III is more accurate than APACHE II; however the third version is more complex and not as widely employed as its predecessors. Some researchers also study the differences between APACHE II and SAPS II since both approaches are more renowned and accurate than others. It has been found that these two models are of equal accuracy and predicting power, but SAPS II can be manipulated more easily than APACHE II (Lin and Hsieh, 1999; Arabi et al., 2002; Gosling et al., 2003). Recently semi-quantitative PCT-Q test (Lobo et al., 2003) and Support Vector Machines (SVM) (Luaces et al., 2009) have been proposed as predictors of ICU. Analysis has shown that PCT-Q test performs better than APACHE II and SVM has superior predicting ability than APACHE III (Luaces et al., 2009). Besides these commonly used methods, in another perspective, some other approaches such optimisation algorithms have been developed to simulate the procedure with limited healthcare resources. For example, ant colony algorithm is embedded to optimise the medical resources in emergency department (Fruggiero et al., 2008; Samanta and Nataraj, 2008). Regardless of the successes of these well-known models, there are no widely accepted criteria and strategies to predict the mortality of ICU patients. On the other hand, there are limited studies to analyse and evaluate the robustness of available models with consideration of sample size of patients’ data. Therefore, investigating model robustness under limited sample data for model regulation is of great theoretical and practical significance. This paper proposes a Multistage-EDA-Enhanced Logistic Regression (MEDAeLR) method aiming at ICU mortality prediction. Before modelling the relationship between the outcome and risk factors, Exploratory Data Analysis (EDA) is employed in an attempt to reduce the number of insignificant variables, and this variable shrinkage procedure will potentially improve the future mortality prediction under limited sample data since some degrees of freedom will be released via this procedure. Multiple Logistic Regression (MLR) is applied to fit the significant variables in the second stage. Unlike most previous work on model selection (Gheissari and Bab-Hadiashar, 2008; Cui et al., 2010; Choi et al., 2011), classical hypothesis testing frameworks are proposed for model comparison and stepwise selection is adapted to variable elimination in the final stage. There are two major advantages using stepwise analysis: One is that the number of variables is less than the full model, and the interpretation to this model becomes easier. Another one is that the variability of the estimated parameters can be reduced. ROC is then used to compare the performance of the method. In addition, this paper explores the performance of the proposed method with two other popular statistical modelling techniques known as: (a) Linear Discriminant Analysis (LDA) and (b) Classification and Regression Trees (CART). Comparisons among these three models are conducted by measuring accuracy sensitivity under different testing sample sizes. Model robustness of MLR with Simple Logistic Regression (SLR) and regular MLR are analysed via the Receiver Operating Characteristic (ROC) curve and Area under the ROC Curve (AUC).


185

The rest of the paper is organised as follows: Section 2 gives an introduction to MEDAeLR and the evaluation tool known as ROC. In Section 3, a cohort of 200 ICU data is used to verify the proposed approaches. Pearson’s chi-square test, Deviance test and Hosmer–Lemeshow statistics are employed to test model adequacy and stepwise regression then is applied for insignificant variable elimination. Comparison to LDA and CART is also presented in this section. The paper is concluded with future research direction in Section 4.

2

Statistical elements in MEDAeLR approaches

MLR is one of the basic statistical models commonly applied in performing classification and regression tasks, especially when dependent variables are categorical rather than continuous. In this section, the proposed models are briefly introduced with literal description.

2.1 Multiple logistic regression MLR is one of the popular regression models which is used to predict the probability of an occurrence or an event by fitting data to a logistic function (Mark and Goldberg, 2001). It can be applied in situations where the observations are not normally distributed or where some observations are discrete or categorical. The functional form of MLR can be denoted as follows: p( y  1 | x) 

ez 1  ez

z   0  1 x1   2 x2  ...   k xk  

(1) (2)

where 0 is called the ‘intercept’ and 1, 2,…, k are called the ‘regression coefficients’ of independent variables x,  is the model error, which follows N (0,  2 ). Each of the regression coefficients, loosely speaking, describes the size of the contribution of that risk factor. MLR is a commonly used model in classification, data mining and machine learning. In our proposed framework, MLR is mainly used as the classification tools associated with EDA.

2.2 Statistical test tools Deviance test, Pearson’s chi-square statistics and Hosmer–Lemeshow test (O’Brien and Fleming, 1979; Montgomery et al., 2006) are three widely used methods for model selection in logistic regression. These three introduced testing tools are designed to test the goodness of fit. In our proposed method, these tools are employed to check the model adequacy after model fitting in the second stage. The deviance is defined as twice the difference in log-likelihoods between the saturated model and the full model (see equation (3)).  L(Saturated Model )  D  2 ln    L( Full Model ) 

(3)

186

L. Chen et al.

Small values of the deviance imply that the model provides a satisfactory fit to the data. Large values illustrate that current model is not adequate. Let the vector of regression coefficients be partitioned as:  1    2 

 

1 is the coefficients matrix of the reduced model (saturated model). The deviance between saturated model and full model can be used to test subsets of the model parameters. The hypotheses is H 0 :  2  0, H1 :  2  0 D(  2 | 1 )  D( 1 )  D(  )

(4)

If D(  2 | 1 )   2 , r reject the null hypothesis. If D (  2 | 1 )   2 , r fails to reject the null hypothesis. Pearson’s chi-square statistics is another approach to do goodness of fit test, which is defined as: ^ ^ ^  2 2  2  ( yi  ni  i ) [(ni  yi )  ni (1   i )]  n ( yi  ni  i )          ^ ^ i 1   i 1 ni  i (1   i ) ni  i ni (1  ni  i )  2

n

(5)



ni is the total observation in the i-th group.  i is the common estimated probability in the i-th group. Small values of the statistics imply the model provides a satisfactory fit to the data. When there are no replicates of the regression variables, Hosmer–Lemeshow test is applied to test goodness of fit of the model. The observations are classified into n groups based on the estimated probabilities. The statistics is defined as: 

n

HL   j 1



where  i 

(o j  N j  i ) 2 



N j  i (1   i )

(6)





igroup j

i Nj

is the average estimated success probability in the j-th group. Nj is

the total observation in the j-th group. Small values imply an adequate fit to the data.

2.3 Stepwise analysis In regression analysis, there is a large pool of possible candidate regressors, of which only a few may be important. This subset of significant variables can be identified through stepwise selection by using statistical criteria (Hansen et al., 2000; Hosmer, 2000; Montgomery et al., 2006; Lorene et al., 2008). Among various kinds of stepwise selection methods, Backward Elimination (BE) is often a good variable selection procedure. BE attempts to find a good model by eliminating an insignificant variable according to some statistical criterion, such as Akaike Information Criterion (AIC).


187

2.4 ROC Curve This technique has been adopted for a long time to evaluate system especially when radio signal is contaminated by noise (Aoki et al., 2000). Typically, a ROC curve is a graphical representation of the trade-off between the true negative and true positive rates for every possible cut off. In the ROC graph, the x-axis is plotted with False Positive (FP) rate against True Positive (TP) rate in y-axis. Although ROC curves are isomorphic to prediction curve, they have an added advantage, that is, they are insensitive to changes in marginal class distribution. The comparison of two or more ROC curves is used to perform by calculating the AUC. It should be pointed out that the MEDAeLR is addressed for two aspects: (a) the proposed method can quickly identify the significant variables. Efficient allocation of the ICU resource can be achieved with the guidance provided our proposed method and (b) patient mortality prediction can be improved through the proposed method, which is critical to the ICU patients and hospitals.

3

Modelling and evaluation of ICU survival data

In this section, three models discussed above are tested and their performances are evaluated under varying training and testing sample sizes using random sub-sampling strategy. Datasets are randomly partitioned multiple times into disjoint training and testing sets through random sub-sampling strategy. This procedure could be repeated unlimited number of times, and the robustness of the underlying model could be well evaluated by taking the average of accuracies. To validate our approach, a small size of sample data is necessary. The chosen dataset was obtained from Baystate Medical Center for clinical studies.

3.1 Data description This collected dataset contains 200 ICU patients’ information with 19 variables from discharge of ICU, the size of which is greatly smaller than most of other common methods choose. Dummy variables are coded for unordered categorical variables with more than two values. The descriptive demographic characteristics of the 200 ICU patients are shown in Table 1. It is important to compute the descriptive statistics for each variable as it provides with an overview perspective of the data. In this study, histograms are constructed for the three continuous variables (AGE, SYS and HRA) (see Figure 1). For discrete variables, the frequency or relative frequency for each category level is calculated for each variable.

3.2 Exploratory data analysis Before quantifying the impacts of significant factors on the patients’ mortality, the key factors that are related to the mortality have to be identified. EDA is conducted to screen out the insignificant variables based on SLR. The significance of each variable is presented in Table 2.

188 Figure 1

Table 1

L. Chen et al. Histograms of continuous variables: AGE, systolic blood pressure (SYS), heart rate (HRA) (see online version for colours)

Demographic characteristic of ICU patients Characteristics

Values

No. of patients

200

Age,(mean, [range])

57[16–92] Sex

Male

124

Female

76 Type of admission, No. (%)

Medical

107(54%)

Surgical

93(46%)

No.(%) of deaths

40(20%)

It can be observed from Table 2 that nine variables that have shown statistically significance (p  0.05) are selected via SLR. They are AGE, SER, CRN, INF, CPR, SYS, TYP, CRE and LOC.

Mortality prediction of ICU patients Table 2

189

Statistics of ICU variables

Factors

Estimate coefficient

Std. error

Z value

Pr(>|z|)

Signif. code **

AGE

0.03341

0.01193

2.801

0.0051

SEX

5.972E-17

0.3946

0

1

RACE

–0.2679

0.4119

–0.65

0.5155

R1

0.4551

0.6555

0.694

0.4875

R2

–1.249

1.056

–1.183

0.237

R3

0.4932

0.8596

0.574

0.566

SER

–1.1417

0.4142

–2.756

0.0005848

CAN

–0.08814

0.6713

–0.131

0.869

CRN

1.12518

0.5122

2.444

0.0145

*

INF

1.1485

0.4005

2.867

0.00663

**

CPR

1.7258

0.5952

2.900

0.00374

**

SYS

–0.017868

0.006436

–2.776

0.0055

**

HRA

–0.002253

0.007121

0.316

0.7517

PRE

0.4678

0.4910

0.953

0.341

TYP

2.792

1.031

2.708

0.006774

FRA

0.1983

0.6881

0.288

0.773

PO2

1.0147

0.6060

1.674

0.094

PH

0.7577

0.6451

1.175

0.24

**

**

PCO

0.1501

0.6025

0.249

0.803

BIC

1.1560

0.6206

1.863

0.0625

CRE

1.4816

0.7355

2.015

0.044

*

LOC

1.9781

0.4952

3.994

0.0000649

***

Notes:

Signif. Codes: 0: ‘***’; 0.001: ‘**’; 0.01: ‘*’; 0.05: ‘.’; 0.1: ‘ ’ 1.

3.3 Comparison between SLR-based MLR and MLR The significant variables from SLR are selected to construct MLR model (denoted as SLR-MLR thereafter.) The training sample sizes vary from 180 to 140, and the samples are randomly drawn from the pool of 200 ICU survival data. Five runs of model fittings are carried out in each individually specified training sample size to avoid negative impact from outliers in dataset. ROC curves are plotted based on the fitted models to demonstrate the model robustness. Figure 2 shows an example of the discrepancy in performance between SLR-MLR and MLR under testing sample size 55. In Figure 2, it can be observed that SLR-based MLR outperforms the MLR. To further confirm this result, AUC is calculated based on the ROC curves under varying testing sample sizes (which are chosen with an interval of 5 units from 20 to 55.) It can be seen in Figure 3 that AUC values produced by SLR-based MLR are larger than those produced by MLR under each testing sample, and the curve is relative stable with little variation (the mean of AUC is 0.77).

190

L. Chen et al.

Figure 2

ROC curve under testing sample size of 55 (see online version for colours)

Figure 3

Comparison between AUCs of MLR with EDA and regular MLR (see online version for colours)

3.4 Result comparison to traditional classification methods The prediction accuracy is chosen as an evaluation tool to gauge the performance of these basic classification approaches. The SLR-based MLR with EDA is also included in this round of comparison. Figure 4 shows the result of comparison. The SLR-based MLR with EDA (83.57% ± 2.63%) and MLR (80.74% ± 2.61%) have relative high accuracy


191

rate compared to LDA and CART, while the lowest prediction level (61.67% ± 3.15%) is produced by classification tree. The SLR-based MLR with EDA beats MLR by almost 3% in average without any overtaking under a series of varying testing samples. The testing results suggest that conducting EDA prior to model fitting is necessary to achieve a more satisfactory classification performance. Figure 4

Accuracy comparisons under varying testing sample size (see online version for colours)

Another interesting observation is that MLR (80.74% ± 2.61%) outperforms LDA (76.44% ± 1.7%) by approximately 4%. The main reason probably is LDA makes assumptions of Gaussian distribution for the class densities. The data suggests that not all continuous variables follow Gaussian distribution (see Figure 1), hence in some cases logistic regression seems a safer, more robust bet than LDA model when dealing with ICU data, as it relies on fewer assumptions.

3.5 Comparison between SLR-based MLR and MEDAeLR Pearson’s chi-square statistics, Deviance and Hosmer–Lemeshow test are employed to check the model adequacy in regression. The results are shown in Table 3, since all the p-value of the tests are greater than the threshold 0.05, the conclusion that MLR with EDA is a more appropriate fit to ICU data than regular MLR model can be drawn. Table 3

Model adequacy tests Tests

Value

p-value

Pearson

193.303

0.400

Deviance

146.689

0.990

Hosmer–Lemeshow

8.337

0.401

192

L. Chen et al.

Although the MLR with EDA outperforms other classification methods, there still exist some insignificant variables in the model. Table 4 shows some details result about the coefficients with Wald test. Table 4

Coefficients of MLR with EDA Estimate

Std. Error

z-value

Pr(>|z|)

Code

Intercept

–3.921369

1.545749

–2.537

0.01118

*

AGE

0.029164

0.012292

2.373

0.01766

*

SER

–0.127594

0.482626

–0.264

0.79149

CRN

0.456918

0.664667

0.687

0.49181

INF

0.211498

0.460439

0.459

0.64599

CPR

0.515000

0.835857

0.616

0.53781

SYS

–0.009061

0.006976

–1.299

0.19403

TYP

1.784387

0.826068

2.160

0.03077

CRE

0.518736

0.837440

0.619

0.53563

LOC

1.746042

0.578524

3.018

0.00254

Notes:

* **

Signif. codes: 0: ‘***’; 0.001: ‘**’; 0.01: ‘*’; 0.05:‘.’; 0.1: ‘ ’ 1.

AGE, TYP and LOC show a significant influence on response variable death. However, some redundant variables may contribute slightly to model. In order to identify these variables, stepwise regression analysis with backward selection is carried for variable selection. The principle for selection is AIC. The larger AIC value is given, the more significant variable is obtained. Table 5 illustrates the selection results for stepwise regression. Table 5

AIC of stepwise regression Start: AIC = 166.69

End: AIC = 158.81

Death ~ AGE + SER + CRN + INF + CPR + SYS + TYP + CRE + LOC

Death ~ AGE + SYS + TYP + LOC

Df

Deviance

AIC

Df

SER

1

146.76

164.76

INF

1

146.9

164.9

SYS

1

Deviance

AIC

AIC

148.81

158.81

158.81

151.02

159.02

159.02

CPR

1

147.05

165.05

AGE

1

157.62

165.62

165.62

CRE

1

147.06

165.06

TYP

1

160.27

168.27

168.27

CRN

1

147.15

165.15

LOC

1

167.82

175.82

175.82

SYS

1

148.48

166.48

–

–

–

–

–

146.69

166.69

–

–

–

–

–

TYP

1

152.69

170.69

–

–

–

–

–

AGE

1

153.01

171.01

–

–

–

–

–

LOC

–

161.47

179.47

–

–

–

–

–


193

Finally, only four variables, namely AGE, SYS, TYP and LOC are significantly contributing to the model fit. Again, Pearson chi-square statistics, Deviance test and Hosmer–Lemeshow test are conducted to test model adequacy (see Table 6). Table 6

Statistic test for MEDAeLR Tests

Value

p-value

Pearson

194.007

0.466

Deviance

148.814

0.992

Hosmer–Lemeshow

9.01

0.341

All tests are passed compared to the threshold. Hence, the MEDAeLR model is a more appropriate fit to the data. Additionally, in order to check the simplified model for accuracy, ROC and AUC are used as the criteria for efficiency evaluation. 40 simulations are performed to calculate AUC with different testing sample sizes (50, 40 and 30). ROC curves of six replicates under testing sample size of 50 are presented in Figure 5. Figure 5

ROC curves for sample size of 50 (see online version for colours)

Figure 5 shows a higher prediction rate, and this implies MEDAeLR is a better fit for the data. AUCs of 40 simulations under different testing sample sizes are plotted in Figure 6.

194

L. Chen et al.

Figure 6

AUCs boxplot for various sample sizes

The results show that the mean AUC exceeds 0.8 (0.819 ± 0.0562, 0.8204 ± 0.0868, 0.8496 ± 0.0898, respectively) for all sample sizes, which is higher than the MLR model with EDA (0.77). Hence, it can be seen that the larger training sample size may lead to a better fit to our model.

4

Conclusions

Mortality prediction of ICU patients plays a critical role in ICU treatment decision. Although numerous prediction models have been suggested, few have been found effective universally under the conditions of limited historical data. This paper proposed a novel method MEDAeLR to increase the classification accuracy and robustness for ICU prediction that doesn’t require a large size of training dataset. Prior to model fitting, EDA is conducted to reduce the insignificant variables. The comparison between SLRbased MLR and MLR using the criterion AUC indicates that EDA is a necessary procedure to enhance model robustness. When comparing with LDA, classification trees show that MLR model performs better than LDA. This might due to the fact that MLR imposes fewer assumptions on data than LDA. The results presented in this study may provide data analyst with a prospective of data pre-processing. The MEDAeLR further eliminates insignificant variables in the SLR-based MLR, which potentially save the computational resources and shows a more satisfactory result than SLR-based MLR. The significant variables can be further utilised to construct warning systems after practical validation. It should be pointed out that the interaction effects in MEDAeLR model are not considered in this paper. Although prevalent research works in ICU survival data demonstrate that high-ordered interaction will hinder statistician from proper interpretation, a lack of interaction analysis may cause misleading results when drawing conclusions. This issue will be addressed in the future research topics.


195

Acknowledgements This research is partially supported by National 985 Program Research Grant III (P.R. China) and Joint Seed Grant from Peking University. The authors also thank Dr. Hosmer and Dr. Lemeshow for providing the ICU data set from Baystate Medical Center.

References Aoki, N., Wall, M.J., Demsar, J., Zupan, B., Granchi, T. Schreiber, M.A., Holcomb, J.B., Byrne, M., Liscum, K.R., Goodwin, G., Beck, J.R. and Mattox, K.L. (2000) ‘Predictive model for survival at the conclusion of a damage control laparotomy’, The American Journal of Surgery, Vol. 180, No. 6, pp.540–545. Arabi, Y., Haddad, S., Goraj, R. and AI-Shimemeri, A (2002) ‘Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit’, Critical Care, Vol. 6, No. 2, pp.166–174. Choi, I., Wells, B.J., Yu, C. and Kattan, M.W. (2011) ‘An empirical approach to model selection through validation for censored survival data’, Journal of Biomedical Informatics, Vol. 44, No. 4, pp.595–606. Cui, G., Wong, M.L and Zhang, G. (2010) ‘Bayesian variable selection for binary response models and direct marketing forecasting’, Expert Systems with Applications, Vol. 37, No. 12, pp.7656–7662. Fruggiero, F., Lambiase, A. and Fallon, D. (2008) ‘Computer simulation and swarm intelligence organization into an emergency department: a balancing approach across ant colony optimization’, International Journal of Services Operations and Informatics, Vol. 3, No. 2, pp.142–161. Gheissari, N. and Bab-Hadiashar, A. (2008) ‘A comparative study of model selection criteria for computer vision applications’, Image and Vision Computing, Vol. 26, No. 12, pp.1636–1649. Gosling, P., Brudney, S. and McGrath, L. (2003) ‘Mortality prediction at admission to intensive care: a comparison of micro albuminuria with acute physiology scores after 24 hours’, Critical Care Medicine, Vol. 31, No. 1, pp.98–103. Hansen, M.C., DeFries, R.S., Townshend, J.R.G. and Sholberg, R. (2000) ‘Global land cover classification at 1 km spatial resolution using a classification tree approach’, International Journal of Remote Sensing, Vol. 21, Nos. 6/7, pp.1331–1364. Hosmer, D.W. (2000) Applied Logistical Regression, Wiley. Knaus, W.A., Draper, E.A. and Wagner, D.P. (1985) ‘APACHE II: a severity of disease classification system’, Critical Care Medicine, Vol. 13, No. 10, pp.818–829. Knaus, W.A., Draper, E.A. and Wagner, D.P. (1991) ‘The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults’, Chest, Vol. 100, No. 6, pp.1619–1636. Knaus, W.A. and Zimmerman, J.E. (1981) ‘APACHE – acute physiology and chronic health evaluation: a physiologically based classification system’, Critical Care Medicine, Vol. 9, No. 8, pp.591–597. Le Gall, J.R., Lemeshow, S. and Saulnier, F. (1993) ‘A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study’, JAMA, Vol. 270, No. 4, pp.2957–2963. Le Gall, J.R., Loirat, P. and Alperovitch, A. (1984) ‘A simplified acute physiology score for ICU patients’, Critical Care Medicine, Vol. 12, No. 11, pp.975–977. Lemeshow, S., Teres, D. and Avrunin, J.S. (1988) ‘Predicting the outcome of intensive care unit patients’, Journal of the American Statistical Association, Vol. 83, No. 402, pp.348–356.

196

L. Chen et al.

Lin, C.C. and Hsieh, W.B. (1999) ‘The introduction of disease severity scoring systems in critical care medicine’, Thoracic Medicine, Vol. 14, pp.44–57. Lobo, S., Lobo, F.R.M. and Bota, D.P. (2003) ‘C-reactive protein levels correlate with mortality and organ failure in critically ill patients’, Chest, Vol. 123, No. 9, pp.2043–2049. Lorene, P., David, M. and Jeuffroy, M-H. (2008) ‘Comparison of stepwise selection and Bayesian model averaging for yield gap analysis’, Ecological Modeling, Vol. 219, Nos. 1/2, pp.66–76. Luaces, O., Toboada, F. and Albaiceta, G.M. (2009) ‘Predicting the probability of survival in intensive care unit patients from a small number of variables and training examples’, Artificial intelligence in Medicine, Vol. 45, No. 1, pp.63–76. Mark, J. and Goldberg, M.A. (2001) ‘Multiple regression analysis and mass assessment: a review of the issues’, The Appraisal Journal, pp.89–109. Montgomery, D.C., Peck, E.A. and Vining, G.G. (2006) Introduction to Linear Regression Analysis (Wiley Series in Probability and Statistics), John Wiley & Sons, Inc., Hoboken, New Jersey. O’Brien, P.C. and Fleming, T.R. (1979) ‘A multiple testing procedure for clinical trials’, Biometrics, Vol. 35, No. 3, pp.549–556. Samanta, B. and Nataraj, C. (2008) ‘Automated diagnosis of cardiac state in healthcare systems using computational intelligence’, International Journal of Services Operations and Informatics, Vol. 3, No. 2, pp.162–177. Takrouri, M.S. (2004) ‘Intensive care unit’, The Internet Journal of Health, Vol. 3, No. 2, 2004. Yaguchi, A., Truog, R.D., Cutis, J.R., Luce, J.M., Levy, M.M., Melot, C. and Vincent, J.L. (2005) ‘International differences in end-to-life attitudes in the intensive care unit’, Archives of Internal Medicine, Vol. 165, No. 17, pp.1970–1975. Zimmerman, J.E., Crammer, A.A. and McNair, D.S. (2006) ‘Intensive care unit length of stay: benchmarking based on acute physiology and chronic health evaluation (APACHE) IV’, Critical Care Medicine, Vol. 34, No. 10, pp.2517–2529.