BMC Medical Informatics and Decision Making

Background: Several models for prediction of acute coronary syndrome (ACS) among chest pain patients in the emergency department (ED) have been presented, but many models predict only the likelihood of acute myocardial infarction, or include a large number of variables, which make them less than optimal for implementation at a busy ED. We report here a simple statistical model for ACS prediction that could be used in routine care at a busy ED.


Background
Unstable angina pectoris and acute myocardial infarction (AMI), together denoted acute coronary syndrome (ACS), are consequences of acute coronary artery disease with myocardial ischemia. Despite considerable progress in the treatment of ACS with antithrombotic drugs and catheter-based interventions (balloon angioplasty), the ability to diagnose ACS in the emergency department (ED) remains relatively poor. Since missed cases of ACS carry a high morbidity and mortality from heart failure and arrhythmia, the number of "rule-out" admissions are high, and some 7 or more out of 10 patients admitted with the sus-picion of ACS do no not have it [1,2]. This large overadmission implies a unsatisfactory quality of care for the patients and a high cost for the health care system [3,4]. To improve the situation, new diagnostic methods such as immediate stress tests [5], myocardial perfusion imaging [6], echocardiography [7], and new blood tests have been suggested. In addition, decision support tools in the form of prediction models have been developed to help the physician handle the clinical information, and thereby to better triage and treat the patient. A large number of such models have been presented [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24], and most of them have been focused on the detection of AMI. With the current ACS paradigm, however, models that predict the probability of AMI are less useful in routine ED care, where the likelihood of ACS (rather than AMI) is normally decisive for admission or discharge, and for immediate treatment. Also, many of these models need substantial input from the ED personnel [8][9][10][11][12]18], and hence are not ideally suited for implementation in standard care at a busy ED. Only one previous model, the ACI-TIPI [14], has been both easy to use and predictive of ACS.
The aim of this study was to develop a simple statistical model, based on ECG and clinical data available immediately at presentation, which predicts the likelihood for ACS among chest pain patients in the ED. The intention was to include ECG data in the form of simple amplitude measurements to allow for machine reading. A prediction model of this type could form the basis for development of a user-friendly decision support system for ED personnel.

Setting and patient material
Lund University Hospital, Sweden, is a 1200 bed institution which serves as the primary hospital for some 250,000 inhabitants, has a cardiac intensive care unit with 19 beds and an intermediate care ward with ECG monitoring at 19 beds. Percutaneous coronary intervention (PCI) and coronary bypass surgery (CABG) are available 24 hours/day. There is a traditional ED with approximately 50000 patients per year. During the patient inclusion period, there was no systematic diagnostic protocol for patients with suspected ACS, and no dedicated chest pain unit.
Six hundred and sixty-five consecutive visits for chest pain for which electronic ECG data could be retrieved were retrospectively included at the ED of Lund University Hospital from July 1 to November 20, 1997. Characteristics of the 634 unique patients are presented in table 1. From the clinical data collected for each patient, 18 variables available immediately at ED presentation (table 1) were chosen for further study based on their likely importance for ACS prediction.

Reference standard
The ED visit ended in hospitalization in 422 (67%) cases, and among these a discharge diagnosis of ACS was assigned in only 130 (31%), of which 28 (22%) had Qwave AMI, 53 (41%) had non-Q-wave AMI and 49 (38%) unstable angina. Discharge diagnoses were made by the senior ward physician or the ED physician (in cases discharged from the ED), reviewed by a senior research nurse, and when ambiguous, further reviewed by a senior cardiologist (HÖ). In the review of diagnoses for cases discharged from the ED, available data from the patient records indicated that the rate of missed diagnosis of ACS was low (not more than 2%). AMI was defined by the WHO criteria [25] where the biochemical criterion was at least one measurement of CK-MB>10 μg/l or Troponin T>0.1 μg/l. The criteria for unstable angina were ischemic symptoms (chest pain >15 min., syncope, acute heart failure or pulmonary edema) together with at least one of the following: a) Electrocardiogram (ECG) changes: transient or persisting ST segment depression (≥ 1 mm) and/or Twave inversion (≥ 1 mm) without developing Q waves or loss of R wave height, or b) Biochemical markers: CK-MB 5-10 μg/l or Troponin T 0.05-0.1 μg/l. This study was approved by the Lund University Research Ethics Committee.

Electrocardiogram
Two methods of ECG analysis were used, machine reading and expert reading.

Machine reading
The 12-lead ECGs were recorded by the use of computerized ECG recorders (Siemens-Elema AB, Solna, Sweden). The ECG measurements were obtained from the measurement program of the ECG recorders. From the leads I, aVF and V2, the two measurements ST-J amplitude and ST amplitude 3/8 were selected for further analysis. These measurements were selected to detect ST-elevation patterns caused by ischemia in the anterior, septal, inferior or lateral wall. The ST amplitude 3/8 was obtained by dividing the interval between ST-J point and the end of the T wave into eight parts of equal duration. The amplitudes at the end of the third interval were denoted ST amplitude 3/ 8. ECG data were entered in the logistic regression models as categorical variables, with the grouping defined in Table 2. The categorisation was based on widely used ECG criteria for ST-elevation and depression [26].

Expert reading
For comparison, ECG for 628 of the 634 unique patients were retrieved at a later point in time, and 608 of these ECGs were possible to assess. Two physician experts first classified the 608 ECGs, which originated from 120 patients with ACS and 488 patients without ACS, inde-pendently of each other and then agreed on a consensus classification using four categories ("ACS and Transmural ischemia (TMI)", "ACS but not TMI", "Probably ACS", and "No signs of ACS"). The experts did not strictly apply conventional criteria such as ST-segment elevation of >0.1 mV in two adjacent limb leads or >0.2 mV in two adjacent precordial leads. Instead, they also considered the configuration of the ST-segment and the shape of the QRS complex, i.e. a pattern recognition analysis was applied as in the clinical routine interpretation of ECGs.

Statistical analyses
The statistical analyses were conducted using SPSS release 12.0.1 (SPSS Inc, Chicago, U.S.). Odds ratios (OR) were calculated for the association between each potential risk factor and ACS. We regarded 95% confidence intervals (CIs) that excluded unity, or, equivalently, p < 0.05, as statistically significant. In the multivariable analysis, the probability of ACS was predicted using multiple logistic regression [27]. All independent variables except age were entered in the regression models as categorical variables, categorized as shown in tables 1, 2. In order to obtain meaningful baseline odds for ACS in the models, we chose 40 years old as reference value for age. Thus, we used the number of years above 40 as age variable in the models and allowed for both positive (patients above 40) and negative (patients below 40) numbers. Starting with the full multivariable model with all independent variables included, we excluded one insignificant independent variable at a time, starting with the variable with highest p-value, until only significant and important predictors remained. Categorical variables with more than two categories were kept in the model if the OR associated with any of the categories was significant. Variables with estimated OR of at least 2.5 (or, equivalently, at most 0.4) were considered as important predictors and thus kept in the model even if they were not statistically significant. The ECG variables were not considered for exclusion. For the final set of independent variables, categories with ORs close to one were collapsed with the reference category for that variable. Unknown response to one of the variables (hypertension) was also added to the reference category. We then assessed the significance of the interaction with age, implemented as cross-product terms together with main effects, for the following variables: sex, chest discomfort at presentation, symptom duration, and previous myocardial infarction. For comparison, we also established a prediction model where we replaced the ST amplitude characteristics of the ECG with the consensus ECG assessments by the two experts (LE and HÖ). Differences in specificity between the two models were tested using McNemar's test for correlated proportions.
The area under the receiver-operating-characteristic curve (ROC) was used as an overall measure of the discrimination abilities of the prediction models [28]. The area under ROC, measured in percent, can be interpreted as the probability that a randomly chosen patient with ACS has a higher outcome probability than a randomly chosen patient without ACS. For calculation of specificity and predictive values, the sensitivity was set to 95%. This somewhat arbitrary level was chosen because with current standard evaluation, some 2-5% of the ACS patients are erroneously discharged from the ED [29,30], which implies a sensitivity of at least 95 % for the routine ED work-up. In order to get more correct estimates of the generalization performance, i.e. the performance of the prediction models on an unseen study population, we used a k-fold cross-validation procedure [31]. Here we repeated times estimated the regression parameters based on randomly chosen training sets, comprising approximately 80% of all patients, and then evaluated the obtained model by calculating the area under ROC as well as the specificity at 95% sensitivity among the remaining 20% patients in a validation set kept completely isolated from the corresponding training set. This k-fold cross-validation procedure was implemented as follows: 1) the data set was randomly split in 5 groups of approximately equal size 2) the parameters of each model were estimated based on the patients of the training set only, which was established by excluding group k [k = 1, 2, ...., 5] 3) the group left out from parameter estimation was used as a validation set. We repeated the steps 1-3 above 20 times, which implied 20 * 5 = 100 sets of validations sets, 100 areas under ROC, and 100 estimates of the specificity at 95% sensitivity. Median values, together with 2.5 -97.5 percentiles, from these 100 validation sets were used as measures of the generalization performance of the prediction models.

Results
When each potential risk factor was considered separately, significant associations with ACS were observed for age, hypertension, previous myocardial infarction, chest discomfort at presentation, and symptom duration (table 1).
Associations with ACS were also indicated for gender, angina pectoris, treatment with cardiac drugs, tachypnea, lung rales, and blood pressure, whereas no clear associations with smoking status, diabetes, congestive heart failure, previous PCI, previous CABG and heart rate were discerned. All ECG characteristics, ST amplitudes in leads I, aVF and V2, were markedly associated with ACS (table  2).
Many of the potential risk factors were strongly interrelated. In the multivariable analysis, older age, hypertension, angina pectoris, previous myocardial infarction, chest discomfort at presentation, short symptom duration, and abnormal ECG characteristics were associated with increased ACS risk (Table 3). Moreover, congestive heart failure and previous CABG were inversely associated with ACS. The prediction model based on these variables yielded a high discrimination percentage (area under ROC 80.6%; figure 1). For this model, hospitalizing all patients with a probability of ACS of at least 8.6%, corresponding to 65% of all patients, would yield a sensitivity of 95% and a specificity of 43%. This sensitivity-specificity pair would, with an ACS prevalence of 21% like in the present material, produce a negative predictive value (NPV) of 97% and a positive predictive value (PPV) of 30%. Adding the interaction terms related to age did not lead to any significant improvement of the model (results not shown).
As an illustration of how the statistical model of

ECG assessments by experts
The ECG assessments made in consensus by the two experts were strong predictors of ACS (table 5)

-Specificity
Sensitivity included in the prediction model. Adding other clinical data yielded markedly higher discriminating percentage (area under ROC 88.0%; table 6 and figure 2). For this model, a sensitivity of 95% would yield a specificity of 50%, which at an ACS-prevalence of 21% would produce a NPV of 98% and a PPV of 33%. The specificity in this model (50%) was significantly higher (p = 0.001) than in the model based on ECG amplitude data (43%). The discriminating percentage increased only marginally when the interaction terms related to age were incorporated (area under ROC 88.3%; regression model and ROC curve not shown).

Discussion
The present study was undertaken to develop a simple and user-friendly statistical model for ACS prediction in the ED. The model was developed using logistic regression on clinical and ECG data from chest pain patients at the ED of a university hospital. Simple categorized ECG amplitude data were included, which allows automated ECG reading, makes ACS prediction stable, and makes the  model useful for personnel inexperienced in ECG reading. To keep the model simple and immediately usable in the ED, results from blood samples were not included. Besides ECG, the following 8 variables were found to be important for ACS prediction, and included in the model: age, chest discomfort at presentation, symptom duration and previous hypertension, angina pectoris, AMI, congestive heart failure or PCI/CABG. The interaction terms, which involved only clinical characteristics and not the ECG amplitude data, did not improve the performance substantially and were therefore not included in the final model.

Estimate
Many of the variables included in the present model have previously appeared in models for AMI or ACS prediction. All previous models have included the ECG, and there is a general consensus that the ECG is the most important clinical variable to predict ACS in the ED [32,33]. Most models have also had a factor for previous coronary artery disease, e.g [13,15,[18][19][20][21], many have included age [12,14,18,19,23] and pain duration [12,14,18,23], but few have included congestive heart failure [19] or hypertension [18]. Other variables that have appeared in previ-ous models are diabetes [12,18,19], sex [14,18], pain localization [12,14,15,18,20,22] pain similar to previous angina or AMI [15,18], the quality of the pain [12,15,18], and previous use of nitrates [18,20]. A systematic review of studies on AMI probability [34] has shown that additional variables with predictive power may be a third heart sound, hypotension, chest pain reproduced by palpation and positional chest pain.
With current standard evaluation in the ED, some 2-5% of the ACS patients are erroneously discharged from the ED [29,30], which indicates a sensitivity of at least 95 % for the routine ED work-up. Set to this level of sensitivity, our simple model reached a specificity of about 35%, and a PPV and NPV of 96% and 27%, respectively, in the crossvalidation. In the present patient material, a sensitivity of 95% could be reached by admitting all patients with an ACS probability above 8.6%, which would not imply a significant change of the actual admission rate. A PPV of 27% may seem very low, but it is likely at a level similar to that of the ED physician's decision after the current standard ED assessment, where some 70% of those admitted for suspected ACS prove not to have it [1,2,35]. We have been unable to find any published data on the PPV of standard ED assessment for possible ACS. Thus, our model did not seem to perform any better than the average physician, and will therefore likely not be useful for the expert physician in the ED. However, if a simple prediction model with automated ECG reading, like ours, can maintain a NPV of 96% in a larger prospective study, we believe it can be useful in real-life routine care. In addition, in healthcare settings with an ACS prevalence below 21% among chest pain patients [2,36], NPV may increase. Seemingly better performances have been reported for previous ACS prediction models [8,12,14,24], but some of these models were both developed and tested in chest pain populations with clearly higher prevalence of ACS than ours. Further, with the exception of the ACI-TIPI [14], these models required the input of a large number of variables, and also ECG interpretation by the ED physician.
The model in the present study with ECG interpretation by experts had much higher overall discrimination ability (area under ROC) and specificity at 95% sensitivity, whereas its predictive ability was only somewhat better than the model based on ST amplitude data, with PPV and NPV of 33 vs 27% and 96 vs 98%. The significance of these differences, translated into a real-world clinical practice, is probably small. It is worth noting that basing a prediction model only on the categorized ST amplitude data would not yield sufficient predictive power since about half (52%; not in results) of the ACS patients were in fact normal according to the used cut-offs for the ST amplitudes in all three leads (I, aVF and V2). However, the combination of simple categorized ST amplitude data and clinical characteristics resulted in almost as much predictive power as expert interpretation of the entire ECG. Further, since the average ED physician is less experienced in ECG interpretation than our experts, the model with computerized ECG reading is probably a better choice for clinical implementation. ACS prediction will be more stable, and personnel with little or no experience in ECG reading, e.g. triage nurses, can use the model with no loss in predictive power. Incorporating ST amplitude data as numeric measurements rather than as categorizations and then allowing for interaction between the ECG measures would most likely increase the predictive power of the statistical model further. In addition, since we did not test all theoretically possible ECG measurements, it may be that additional ECG variables or combinations of variables could have increased the predictive power of the ECG and the entire model further. An important disadvantage of a more complex prediction model is however that the rationale behind its output will be much more difficult to understand by potential users.
A model for ACS prediction based on automated ECG reading together with clinical characteristics can probably be applied in many different settings. For the present model, it seems wise to exploit the NPV, since PPV was low at a sensitivity of 95%. If our model can be validated in prospective studies, it is thus probably best used as support for discharging patients in settings where the ACS prevalence is low, e.g. in primary care, in the initial ED triage or in telemedicine situations, where information is limited. It would also seem possible to use the model as support for the early discharge of low risk patients after a brief observation in the ED or in a chest pain unit, or to assist inexperienced junior staff in the ED, as a means to  improve quality of care. Adding results of blood samples, repeated ECGs and physician judgment would probably increase NPV to close to 100%. Whatever the use of our model, a good feature is the limited number of variables, which implies a small need for manual input, and an increased likelihood that the model will actually be used in a busy healthcare environment. With the exception of the ACI-TIPI model [14], the need for a time-consuming large input has been a weak point of several earlier prediction models [8][9][10][11][12]18], where up to 40 questions need to be answered before the model gives decision support. Our model needs answers to 8 questions.

Limitations of the study
The patients included in the present model were retrospectively collected and from one center only. Because of the retrospective nature of the study, there may of course be errors despite a careful collection of all data. For example, the diagnoses of the patients discharged from the ED were not tested with routine post-discharge ECG or blood samples for cardiac markers. Therefore, before clinical implementation, the model clearly needs to be validated prospectively, preferably at multiple centers. An old definition of AMI was used in the present study. Newer definitions of AMI have lower cut-off values for biochemical markers, and some of the unstable angina diagnoses in this study would currently be classified as AMI [37]. However, the total number of ACS cases would probably be little changed. Furthermore, the baseline risk (odds) for ACS of the model has to be tuned in each population before implementation. Since it is impossible to analyze all information with a potential impact on the likelihood of ACS, it is of course possible that variables in addition to those included in our model could be important for ACS prediction. The size of the sample might also have limited our abilities to detect clinical or ECG characteristics of low prevalence that nevertheless are important for ACS classification.

Conclusion
Previous studies have shown that decision support systems are an effective means of improving patient care [38]. The aim of this study was to create a simple and practical statistical model for ACS prediction that could be used in routine care at a busy ED. The model includes simple ECG data (suitable for computerized reading) and 8 clinical variables and can be used immediately on patient arrival by personnel inexperienced in ECG interpretation.
At the set sensitivity of 95% and an ACS prevalence of 21%, the NPV was 96%. We believe that this prediction model, combined with the judgment of trained healthcare personnel, could be useful for the early discharge of chest pain patients in populations with a low prevalence of ACS. However, the model must be prospectively validated before it can be used in clinical practice.