A genetic fuzzy system for unstable angina risk assessment

Background Unstable Angina (UA) is widely accepted as a critical phase of coronary heart disease with patients exhibiting widely varying risks. Early risk assessment of UA is at the center of the management program, which allows physicians to categorize patients according to the clinical characteristics and stratification of risk and different prognosis. Although many prognostic models have been widely used for UA risk assessment in clinical practice, a number of studies have highlighted possible shortcomings. One serious drawback is that existing models lack the ability to deal with the intrinsic uncertainty about the variables utilized. Methods In order to help physicians refine knowledge for the stratification of UA risk with respect to vagueness in information, this paper develops an intelligent system combining genetic algorithm and fuzzy association rule mining. In detail, it models the input information’s vagueness through fuzzy sets, and then applies a genetic fuzzy system on the acquired fuzzy sets to extract the fuzzy rule set for the problem of UA risk assessment. Results The proposed system is evaluated using a real data-set collected from the cardiology department of a Chinese hospital, which consists of 54 patient cases. 9 numerical patient features and 17 categorical patient features that appear in the data-set are selected in the experiments. The proposed system made the same decisions as the physician in 46 (out of a total of 54) tested cases (85.2%). Conclusions By comparing the results that are obtained through the proposed system with those resulting from the physician’s decision, it has been found that the developed model is highly reflective of reality. The proposed system could be used for educational purposes, and with further improvements, could assist and guide young physicians in their daily work.


Background
Unstable Angina (UA) is a kind of chest discomfort or pain that occurs in a continuous and unpredictable way [1,2]. The unstable pain can result from the disruption of an atherosclerotic plaque in narrowed coronary vessels with lessened flexibility, embolization and vasospasm. As a major type of Cardiovascular Disease (CVD), UA lays its symptoms between stable angina and acute myocardium infarction and a further sudden death [2]. While the risk of UA is high, the population of UA is huge, especially for aged people and those with associated disease such as *Correspondence: zhengxing.h@gmail.com 2 College of Biomedical Engineering and Instrument Science, Zhejiang University, 310008, Zhou Yiqing Building 510, Zheda road 38#, Hangzhou, Zhejiang, China Full list of author information is available at the end of the article hypertension and diabetes [3]. To this end, reliable assessment of risk levels for individual UA patients will be of significant value and interest.
A number of models for UA risk assessment have been proposed in literature. Most of these models are derived from databases of clinical trials, e.g., the Thrombolysis in Myocardial Infarction (TIMI) [4], platelet glycoprotein IIb/IIIa in unstable angina: Receptor Suppression Using Integrilin (PURSUIT) [5], and the Global Registry of Acute Coronary Events (GRACE) [6], etc. They use standard patient features that are part of the routine medical evaluation of UA patients, and lead to a score to define prognostic groups [2]. Although there are many benefits related to the design and use of these prognostic models, a number of studies have highlighted possible shortcomings [2,4]. One serious drawback is that existing models http://www.biomedcentral.com/1472-6947/14 /12 lack the ability to deal with the intrinsic uncertainty about patient features utilized in UA risk assessment. Note that vagueness is fundamental and indispensable aspects of knowledge, so as in many practical problems, the experts face vagueness in feature vectors. According to Bellman and Zadeh "much of the decision making in the real world takes place in an environment in which the goals, the constraints, and consequences of possible actions are not known precisely" [7]. Regarding UA risk assessment, many patient features are vague, and not easy to be handled by existing models. It is, therefore, necessary to develop a new UA risk assessment model to deal with vague information.
In this paper, a novel UA risk assessment model has been developed using fuzzy set theories. The proposed model represents patient features with fuzzy sets and then extracts useful information with a descriptive rule induction approach based on fuzzy systems. To derive fuzzy rules from data, the proposed model employs genetic algorithms (GAs) to learn rule base from the collected data-set. GAs are search algorithms based on natural genetics that provide robust search capabilities in complex spaces [8]. The hybridization between fuzzy systems and GAs, called genetic fuzzy system (GFS), has attracted considerable attention in the computational intelligence community [9][10][11][12]. Our main goal is to develop a novel GFS such that we derive from clinical data-set a set of assessment rules, which has good interpretability before determining an efficient assessment model in order to get high accuracy of UA risk stratification. Since the accuracy of an assessment model can be largely affected by processing vague patient features, this paper also discusses a clustering-based method for patient feature partitioning.
This paper is organized as follows. Section 'Preliminary' presents preliminary knowledge used in this paper. Section 'Method' describes the development of the genetic-fuzzy system for UA risk assessment. Experimental studies of the performance of the proposed approach are presented in Section 'Results and discussion'. Section 'Conclusion' concludes the paper.

Preliminary
Let D = {σ 1 , · · · , σ n } be a patient data-set consisting of a finite set of UA patient cases. Let A = {a 1 , · · · , a n } represent all patient features that appear in D and Class = {low-risk, medium-risk, high-risk} be a set of UA risk levels. Each feature a may have a categorical or numerical underlying domain, denoted dom(a). Each patient case σ (σ ∈ D) contains values of some patient features from A. Let σ (a) (σ (a) ∈ dom(a)) be the target feature value for the patient case σ for feature a.
For example, Table 1 shows an example patient data-set, which consists of five patient cases. Each case contains 1 numerical patient features (i.e., age) and 3 categorical patient features (i.e., sex, smoking, and has event recently).
For numerical patient feature a (a ∈ A), let {l 1 a , l 2 a , · · · , l m a } be a set of linguistic terms defined over a. Let μ a,j (σ (a)) be the membership degree on the value of a feature a of the patient case σ to the fuzzy set corresponding to the linguistic label l j a for this feature a. Note that the degree of membership of each value of a in any of the fuzzy sets specified for a is directly based on the evaluation of the membership function of the particular fuzzy set with the value of a as input. The fuzzy partition of dom(a) is composed of {l 1 a , · · · , l m a } that satisfies m j=1 μ a,j (x) = 1, ∀x ∈ dom(a).
In this study, we employ fuzzy rules of the following form [9,12]: where Cond is the antecedent part of the rule, and Class is the consequent part of the rule. For example, a fuzzy rule can be expressed as: R : IF a 1 is (l 1 a or l 3 a ) and a 5 is l 4 a THEN low-risk.
(2) It must be noted that any subset of the complete set of patient features, with any combination of linguistic labels related to the operators and and or, can take part in the rule antecedent. For this kind of fuzzy rule, we say that a patient case σ supports the antecedent part of a rule r if where μ a i ,j (σ (a i )) is the membership degree of patient feature a i for σ to the fuzzy set corresponding to the linguistic label l j a i for a i ; and APC is the antecedent part compatibility between a patient case and the antecedent part of a fuzzy rule. For the categorical features, the degrees of membership are zero or one.
For a patient case σ , the support degree of σ by a specific rule r is calculated as follows: http://www.biomedcentral.com/1472-6947/14 /12 Supp(σ , r) = APC(σ , r) the actual risk level of σ is equal to the class of r 0 o t h e r w i s e (4) In general, a fuzzy rule can be considered to be a classification rule if the antecedent contains fuzzy item sets, and the consequent part contains only one class label such as low-risk, medium-risk, or high-risk in this study. A fuzzy rule r : Cond → Class could be measured directly in terms of support and confidence as follows:

Method
In this section, we describe the process of utilizing GFS to develop an intelligent system for the problem of UA risk assessment. As shown in Figure 1, the proposed method consists of three steps. At first, all the numerical patient features of the data-set are given as input for the fuzzy clustering module for calculating the membership functions. Then, calculated function values are given to the rule generation module for obtaining UA risk assessment rules. Based on the derived rules, a classification model for UA risk assessment is generated. The case study was performed in the Cardiology Department at the Chinese PLA General hospital. Prior approval was obtained from the data protection committee of the hospital to conduct the study. We state that the patient data was anonymized in this study and in the Method section of this paper.

Fuzzy clustering for numerical feature discretization
One of the most important steps in UA risk assessment is to deal with the intrinsic uncertainty about the variables utilized. As described in [13], fuzzy set is a common tool for facilitating the interpretation of rules in linguistic terms, and avoiding unnatural boundaries in the partitioning of the variable domains. It is especially useful in clinical settings where the boundaries of a piece of information used may not be clearly defined. Regarding our task of UA risk assessment, the quality of the results produced relies quite crucially on the appropriateness of fuzzy sets to the given patient features. So, fuzzy sets must be consistent with the values of the corresponding feature.
Fuzzy sets can be provided by physicians. However, the provided fuzzy sets by physicians may not be suitable for mining fuzzy association rules from data-set. Also, it is extremely difficult for physicians to estimate the most appropriate fuzzy sets. In order to cope with these problems, we first concentrate on how fuzzy sets of the given features are determined automatically from the collected data-set. Clustering techniques are usually employed as a preprocessing step to partition numerical features [14]. In this study, we employed a hierarchical agglomerative clustering [15][16][17] algorithm to partition numerical features.
As shown in algorithm 1, hierarchical agglomerative clustering begins with each value as a separate cluster and merges them into successively larger clusters. The process is repeated until the similarity between any pair of clusters is less than a threshold value ε. Consequently, the algorithm builds a structure called dendogram, i.e., a tree illustrating the merging process and intermediate clusters.
Similarity between two clusters c 1 and c 2 can be measured as follows: where |c 1 | and |c 2 | are the number of clusters c 1 and c 2 , respectively. δ is a cluster similarity measure 4: ε is a merging threshold value 5: Output: 6: C is the set of clusters 7: Steps: 8: Let C = {initial clusters} where each value in D a forms an initial cluster This way, the values of each patient feature in the dataset are distributed over a set of derived clusters using Algorithm 1. For each patient feature, the centroids of the clusters are the set of midpoints of the fuzzy sets. To illustrate the process, suppose we want to find fuzzy sets for a specific patient feature a, which is quantitative with a range from min(dom(a)) to max(dom(a)). Let {v 1 a , · · · , v m a } be the set of mid-points of the fuzzy sets for a. As a result, the derived fuzzy sets will have the following ranges: where v 0 a = min(dom(a)), and v m+1 a = max(dom(a)). After the fuzzy sets of each numerical feature are obtained, the corresponding membership function can be generated for each fuzzy set. In this study, we used membership functions of both semi-trapezoidal shape and triangular shape because they are in general the most appropriate shapes and the most widely used in fuzzy systems. For example, for the fuzzy set with a range from v 0 a to v 1 a , the membership function is given by For each fuzzy set with midpoint v j a , where 1 ≤ j ≤ m, the membership function is given by And for the fuzzy set with a range from v m a to v m+1 a , the membership function is given by  Figure 2.

Fuzzy association rule mining
This subsection presents a GFS for mining fuzzy association rules from a data-set. The proposed system uses fuzzy rule format defined in Equation (2), which offers a flexible structure to the rules, allowing each patient feature to take more than one value and facilitating the extraction of general UA risk assessment rules.

Chromosome representation
As mentioned above, a US risk assessment rule r consists of an antecedent part Cond and a consequent part Class.
In this study, we code the antecedent part Cond of r as one chromosome consisting of a set of segments. Each segment corresponds to a specific patient feature. The set of possible values for the categorical features is that indicated by the problem, and for numerical features, it is the set of linguistic terms determined by the clustering method presented above. The consequent part Class of r is prefixed to one of the possible values of risk levels, i.e., high-risk, medium-risk, and low-risk. Table 2 describes a representation for a rule with numerical and categorical features for the values of a specific risk level high-risk. Note that a bit for each one of the possible values of each feature is stored. In this way, if the value of the corresponding element is 0, it indicates that the value is not used in the rule. Otherwise, if the value is 1, it indicates that the corresponding value is included. If a rule contains all the elements corresponding to a feature of the value 1, or all of them contain the value 0, this indicates  that this feature has no relevance for the information contributed in the rule, and so this feature is ignored. In these cases, the feature does not take part in the rule. For example, as shown in Table 2, the rule is represented by a binary string (0011)(00) (10) (10) : high-risk , where parentheses are to separate segments, and ":" is to separate the IF part and the THEN part of the rule. In this example, a 1 has four possible values and a 2 , a 3 , and a 4 have two possible values. Note that a 2 does not take part in the rule as a 2 takes none of its values, and thus a 2 is irrelevant for the rule. This binary string can be interpreted as the following rule: "IF age is (old or very old) and smoking is true and has heart events recently is true THEN UA risk is high".

Fitness function and selection process
The objective of this step is to find the accurate and general rules for UA risk assessment. Thus, given a specific rule r, the GA method uses the composite fitness function consisting of support and confidence in the following way: The objective of the fitness function is defined as the composite measure of support and confidence. This composite measurement provides an effective selection environment which balances the accuracy and generality of the rules. Three operators, i.e., selection, crossover, and mutation, are applied in the proposed GA method to generate the offspring population, which are illustrated as follows: • The selection procedure is used for evolution where two individuals are selected randomly from the current population and used for crossover and mutation operators. During each generation, individuals with higher fitness values survive while those with lower fitness values are destroyed. • Two parents are selected and recombined according to the predefined crossover probability during crossover. In this work, one point crossover is applied due to its simplicity, which can randomly select different cutoff points for each parent to generate offspring rule sets. • Each element in the chromosome is applied to mutation with a predefined mutation probability. The value of a randomly selected element is converted to 0 if its value is 1, and vice versa. Elimination of existing rules and addition of new rules can also be used as mutation operations. As a result, the number of rules in the rule sets string can be changed accordingly.
During GA operations, redundant rules might be produced. For example, we say a rule "If Age is (Young) and ST is (Low) then Risk is (Low)" is redundant w.r.t the other rule "If Age is (Young OR Meddle-ages) and ST is (Low) then Risk is (Low)", if both rules have the same support degree on a data-set. Thus, the proposed algorithm must check the rule sets and maintains single among all the rules, to guarantee the consistency of fuzzy systems. The stopping criterion of the proposed algorithm is the number of generations. The scheme of the proposed algorithm is shown in Algorithm 2.

Algorithm 2
The GA-based rule mining algorithm for UA risk assessment. 1: Input: 2: UA data-set, algorithm parameters 3: Output: 4: is the set of derived UA risk assessment rules 5: Steps: 6: For each target class c of UA risk levels, apply the following steps 7: Step 1: Initialize P c as a set of UA risk assessment rules 8: Step 2: Fitness evaluation for each individual r ∈ P c 9: Step 3: While not (termination condition) do 10: Step 3.1: Select r p 1 and r p 2 from P c with high fitness value 11: Step 3.2: Apply crossover to r p 1 and r p 2 for obtaining r c 1 and r c 2 12: Step 3.3: Apply mutation to r c 1 and r c 2 13: Step 3.4: Check if r c 1 and r c 2 have been stored in P c , if not 14: Step 3.4.1: Update P c by deleting the worst solution and adding r c 1 and r c 2 15: Step 4: Add P c to rule pool 16: Output selected rule set from 17: End Procedure Taking the data set shown in Table 1 as an example, and assuming the population size as 50, the number of generation as 1000, the crossover rate as 0.5, and the mutation http://www.biomedcentral.com/1472-6947/14/12 rate as 0.2 in the proposed genetic algorithm, we can obtain an example rule-set, as shown in Table 3.

UA risk assessment model
Based on the derived rule set, we can generate a classification model for UA risk assessment. Formally, let be the set of derived UA risk assessment rules. For each r : Cond → Class (r ∈ ), the score value of the target class (class ∈ {low-risk, medium-risk, high-risk}) of r with respect to a given patient case σ can be assessed, by using the following equation: where Confidence(r) is the confidence value of rule r, and β(r, σ ) is the firing strength of the input patient case σ on the antecedent part of rule r. The firing strength β(r, σ ) is defined as β(r, σ ) = (a,l 1 a ),··· ,(a,l m a ) ∈ Cond max{μ a,1 (σ (a)), · · · , μ a,m (σ (a i ))} (13) where μ a,j represents the fuzzy membership function for the pair of a and the linguistic term l j a in the (a, l 1 a ), · · · , (a, l m a ) (1 ≤ j ≤ m) in the antecedent part Cond of rule r.
With respect to the target values of risk levels, i.e., low-risk, medium-risk, high-risk, the corresponding scores v l , v m , v h can be generated based on Equation (12). And the risk level with the top score in the scoring vector v will be the predicted risk level for the patient case σ .
Taking σ 1 shown in Table 1 as an example, and using the derived rule-set shown in Table 3, the score values for σ 1 are calculated as v l = 0.4, v m = 2.088, and v h = 1.745, by Equation (12). Thus, the predicted risk level for σ 1 is medium-risk.

Results and discussion
To evaluate the feasibility of the presented methods, a clinical case study is conducted through the cooperation with the Cardiology Department of the Chinese PLA General Hospital. The data-set collected from the hospital consists of 54 patient cases. The target classes of UA risk levels include: low-risk, medium-risk, and high-risk. Physicians that evaluated these cases are experienced clinicians working for the hospital with 10 years of working experience on average. As a result, 16 cases are classified into the low-risk group, 33 cases are classified into the mediumrisk group, and 5 cases are classified into the high-risk group, respectively. Patient features (9 numerical features and 17 categorical features) that appear in the data-set are shown in Table 4. These features are regularly recorded in UA treatment practice. All experiments were performed on a Lenevo Compatible PC with an Intel Pentium IV CPU 2.8 GHz, 4G byte main memory running on Microsoft Windows 7. The algorithms were implemented using Microsoft C#. A 10-fold cross-validation was performed to evaluate the proposed method, by using a 90% of the data-set as the training set, and the remaining 10% as the validation set.
To reduce variability, 10 rounds of this validation process were performed by using different partitions.
For the first step of patient feature discretization, we applied Hierarchical Agglomerative Clustering method to each numerical feature to generate a set of fuzzy sets. The derived fuzzy sets of input numerical patient features are shown in Figure 3.
To mine fuzzy association rules, we have taken the population size as 100, the number of generation as 1000, the crossover rate as 1.0, and the mutation rate as 0.2 in the proposed genetic algorithm. By using these parameters, we run our genetic algorithm for each target class of UA risk level, one by one, to obtain a set of fuzzy rules for a given class. Table 5 shows the rules obtained, which have the best fitness values for the target classes of UA risk level (i.e., low-risk, medium-risk, and high-risk). In this table, the number of patient features involved in each rule (# of Feature.), and the Support and Confidence of each rule are shown. The values of Support and Confidence are between zero and one. High values in support means that the rule covers most of patient cases which are categorized into the class, and high values in confidence means that  the rule has few negative patient cases [12]. Note that the knowledge discovered for each target value of risk level is understandable by physicians due to the use of fuzzy logic and the low number of rules and conditions in the rule antecedents (below 40% of 26 patient features). Tables 6, 7 and 8 show the rules obtained which have the best fitness values corresponding to the target classes of risk level. Now, acquiring fuzzy rule base, it is possible to complete UA risk assessment through the proposed classification model. As we mentioned above, the ensemble of fuzzy rules and the proposed classification model perform the role of a mathematical function to obtain the system output. This output is the stratification of UA risk. This way, for each patient whose data are informed as system inputs, the most likely risk level for that patient is generated.
The comparison between the proposed model and physicians' decisions is done for each partition of dataset. The comparison is given in Figure 4. The proposed system made the same decisions as the physician in 46 (out of a total of 54) tested cases (85.2%). From Figure 4, the proposed model did not predict the risk well in the case of the fifth (the physician assessment is mediumrisk while the proposed model assessment is low-risk), the twenty third (the physician assessment is low-risk while the proposed model assessment is medium-risk), the twenty ninth (the physician assessment is mediumrisk while the proposed model assessment is low-risk), the thirty third (the physician assessment is low-risk while the proposed model assessment is medium-risk), the thirty fourth (the physician assessment is medium-risk while the  proposed model assessment is low-risk), the forty (the physician assessment is medium-risk while the proposed model assessment is low-risk), the forty second (the physician assessment is medium-risk while the proposed model assessment is low-risk), and the forty seventh (the physician assessment is medium-risk while the proposed model assessment is low-risk). Furthermore, we measure the accuracy of the proposed approach using the sum of two performance measures: sensitivity (probability that the test correctly classifies a case with a specific risk level) and specificity (probability of correctly classifying a case without a specific risk level).
where v class ∈ {low-risk, medium-risk, high-risk}; TP is the set True Positive, patient cases with the specific risk level v class classified correctly; FN is the set False Negative, patient cases with the specific risk level v class classified as other risk levels; TN is the set True Negative, patient cases without the specific risk level v class classified; FP is the  set False Positive, patient cases without the specific risk level v class classified as v class . In Table 9 the sensitivity and specificity obtained for each risk level are presented. The experimental results indicate that the proposed method is feasible for predicting risk levels of unstable angina patients.

Conclusion
In this paper, we have presented an intelligent system for UA risk assessment by combining genetic algorithm and fuzzy association rule mining. The developed approach has been tested on a data-set consisting of 54 UA patient cases from the Cardiology department of Chinese PLA General hospital. The experimental results show that considerable agreement is achieved between the proposed approach and physicians' problem solving knowledge. The main novelty of the developed model is that it represents a valuable objective tool for UA risk assessment. In medical literature, physicians are in discrepancies about the risk factors highlighted. This research has focused on the application of computational intelligence. In particular, a genetic-fuzzy system, to identify the key factors behind UA, is proposed, which could be used for educational purposes, and with further improvements, could assist and guide young physicians in their daily work.
For future studies, there may be a comparison of effectiveness in terms of the proposed system with traditional UA risk assessment models, such as TIMI, GRACE, etc. The application of the proposed system to other kinds of CVD, such as heart failure, will also be investigated.