Skip to main content

Comparing the performance of machine learning and conventional models for predicting atherosclerotic cardiovascular disease in a general Chinese population



Accurately predicting the risk of atherosclerotic cardiovascular disease (ASCVD) is crucial for implementing individualized prevention strategies and improving patient outcomes. Our objective is to develop machine learning (ML)-based models for predicting ASCVD risk in a prospective Chinese population and compare their performance with conventional regression models.


A hybrid dataset consisting of 551 features was used, including 98 demographic, behavioral, and psychological features, 444 Electrocardiograph (ECG) features, and 9 Echocardiography (Echo) features. Seven machine learning (ML)-based models were trained, validated, and tested after selecting the 30 most informative features. We compared the discrimination, calibration, net benefit, and net reclassification improvement (NRI) of the ML models with those of conventional ASCVD risk calculators, such as the Pooled Cohort Equations (PCE) and Prediction for ASCVD Risk in China (China-PAR).


The study included 9,609 participants (mean age 53.4 ± 10.4 years, 53.7% female), and during a median follow-up of 4.7 years, 431 (4.5%) participants developed ASCVD. In the testing set, the final ML-based ANN model outperformed PCE, China-PAR, recalibrated PCE, and recalibrated China-PAR in predicting ASCVD. This was demonstrated by the model's higher area under the curve (AUC) of 0.800, compared to 0.777, 0.780, 0.779, and 0.779 for the other models, respectively. Additionally, the model had a lower Hosmer–Lemeshow χ2 of 9.1, compared to 37.3, 67.6, 126.6, and 18.6 for the other models. The net benefit at a threshold of 5% was also higher for the ML-based ANN model at 0.017, compared to 0.016, 0.013, 0.017, and 0.016 for the other models, respectively. Furthermore, the NRI was 0.089 for the ML-based ANN model, while it was 0.355, 0.098, and 0.088 for PCE, China-PAR, and recalibrated PCE, respectively.


Compared to conventional regression ASCVD risk calculators, such as PCE and China-PAR, the ANN prediction model may help optimize identification of individuals at heightened cardiovascular risk by flexibly incorporating a wider range of potential predictors. The findings may help guide clinical decision-making and ultimately contribute to ASCVD prevention and management.

Peer Review reports


Atherosclerotic cardiovascular disease (ASCVD), defined as nonfatal acute myocardial infarction, coronary heart disease (CHD) death, and stroke, has become the leading cause of morbidity and mortality worldwide [1]. In China, cardiovascular disease (CVD) accounts for 38.9% of deaths among females and 35.5% of deaths among males [2], and approximately 61% of these deaths were attributed to ASCVD [3]. Identification of high-risk individuals is critical for the primary prevention of ASCVD. However, personalized assessment of cardiovascular risk remains a challenge in clinical practice. Current medical guidelines recommend the use of risk calculator to perform risk assessment and guide decision-making in ASCVD management [4,5,6,7]. The 2017 American College of Cardiology/American Heart Association (ACC/AHA) guideline highlighted the use of the pooled cohort equations (PCE) to determine 10-year ASCVD risk [5], which present a widely used guideline-endorsed risk calculator in clinical setting. The PCE were derived primarily from non-Hispanic Whites and African-American populations, over- and underestimation of risk have been reported for specific population cohorts [8,9,10,11]. Recently, the Prediction for ASCVD Risk in China (China-PAR) equations were published as a risk prediction calculator designed for Chinese adults and has been approved by the Chinese guidelines in 2019 [7], while more evidence is warranted to confirm its generality for different population.

Notably, recent evidence suggested that besides traditional risk factors of ASCVD (age, sex, smoking, etc.), parameters from electrocardiography (ECG) and echocardiography (Echo) are also helpful in predicting the risk of CVD by detection of subclinical impairment before cardiac symptoms appear [12]. Abnormality in P wave, PR intervals, and left ventricular ejection fraction (LVEF) has been demonstrated to be associated with adverse cardiovascular outcomes [12,13,14,15,16,17]. However, these massive amounts of parameters reflecting the electrophysiology, structure, and function of the heart were unlikely to be included in traditional prediction models. In this context, developing a trainable model for different populations may provide novel insights for the prevention of ASCVD.

Machine learning (ML) offers a novel approach for individualized risk assessment, which could incorporate variables that may not be considered in traditional regression algorithms to develop risk prediction model by exploring the multidimensional and nonlinear relationship between potential variables [18]. Many studies have applied machine learning to the evaluation of various disease and with good results [19,20,21]. Although there has been a rapid expansion of ML being applied to cardiology [22,23,24], few direct comparisons have been made between ML and traditional ASCVD models [25, 26], none of these studies have included the Chinese population.

The present study aimed to establish ML-based risk prediction models from the dataset that integrates demographic, behavioral, psychological, Electrocardiograph and Echocardiography variables to predict ASCVD in a community-based general population in Northeast China. Meanwhile, we compared the performance of ML algorithms to traditional Cox regression models (PCE and China-PAR) to evaluate which method provided superior predictive performance.


Study population

The Northeast China Rural Cardiovascular Health Study (NCRCHS) is a multistage, stratified, random cluster sampling prospective population cohort of 11,956 participants aged ≥ 35 years, recruited between Jan 9, 2013, to Aug 23, 2013, from rural residents living in the Liaoning Province, China. Demographics, physical status and vitals, medical histories, echocardiography data, ECG exams, and laboratory data were collected. Consistent with the target population of contemporary risk prediction scores, participants were included in case of age between 35 and 85 years, no history of CVD. Of 11,956 participants assessed for eligibility, 10,349 (86.6%) participants completed at least one follow-up visit.

Clinical demographics

At baseline face-to-face interviews, detailed information included Clinical demographics (sex, age, marriage, education, nation, etc.) as well as lifestyle factors and (family) medical histories (heart disease, stroke, diabetes, hypertension, etc.) were collected using standardized questionnaires by trained staff (Supplementary file 1). Weight (the nearest 0.1 kg), height and waist circumference (the nearest 0.1 cm) of participants were measured. The body mass index (BMI) was calculated as weight in kilograms divided by height in meters squared. Blood pressure (BP) was measured using a standardized automatic electronic sphygmomanometer (HEM-907; Omron, Tokyo, Japan) after 5 min of rest for three times, and the mean values of systolic/diastolic blood pressure were calculated. The questionnaire was checked by trained staff at the end of each participant’s follow-up to ensure that the data collected were complete and accurate. The paper questionnaire was manually double-entered and subsequently saved. Blood samples from all participants were collected in the morning after > 12 h of overnight fasting.

Hypertension was defined as a mean systolic blood pressure > 140 mmHg and/or diastolic blood pressure > 90 mmHg or taking antihypertensive medications. Diabetes mellitus was defined by medical history and/or use of insulin or oral hypoglycemic agents. Participants were considered to be current smokers/drinkers if they had smoked/drank at any point in the 3 months prior to the date of the ECG examination.

Electrocardiograph and Echocardiography measurement

Standard 12-lead ECGs (MAC 5500, GE Healthcare, Little Chalfont, UK) were recorded in the resting supine position at baseline and were analyzed automatically with the MUSE Cardiology Information System, version 7.0.0 (GE Marquette™ 12SL™ ECG analysis program) [27]. Total of 645 parameters from the unprocessed digital ECG data were disposed by the GE system. 201 parameters (9 not lead-specific and 192[16*12] lead-specific) were temporally stored that including the relative coordinate points (the start point of the p-wave, etc.), and calculated values (QTc Framingham and QTc Fridercia, etc.) were excluded. The remaining 444 variables were used for analysis.

Transthoracic doppler echocardiography (Vivid; GE Healthcare, Connecticut, USA) constituted M-mode, two-dimensional, spectral, and color Doppler formats were operated by the sonographers with a 3.0-MHz transducer. Three professional doctors performed readings and analysis of the echocardiogram and had the option to consult two additional specialists if questions or uncertainties arose. Total of 9 echocardiographic parameters were used for analysis.

Outcome assessment

Primary endpoints include stroke and CHD. Health status, hospital admissions, outpatient diagnosis, and deaths of each participant were followed up from 2015 to 2018. Two physicians reviewed medical records independently, categorized the events and specified the event dates. Stroke was defined as a sudden onset of focal neurological dysfunction lasting 24 h or until death, or less than 24 h but with a clinically relevant brain lesion. CHD was defined to include any myocardial infarction (MI), resuscitated cardiac arrest, definite angina, probable angina followed by revascularization, and CHD death.

Machine learning

Feature selection

In this study, a total of 635 candidate variables were collected and 84 variables with a missing ratio greater than 10% were excluded. Missing values were imputed using the Multiple imputation method when missingness was < 10%. The filtered dataset included 551 variables: 98 demographics, behavioral and psychological variables (age, sex, BP, BMI, lifestyle, biochemical test, etc.), 444 ECG parameters and 9 Echo parameters. The detailed descriptions of features are available in Table S1. Feature selection was implemented using an approach known as ‘Recursive Feature Elimination’ (RFE) to reduce the feature dimension and find out the most discriminative information by selecting the most relevant variables and removing redundant variables. During the recursion process, an optimal subset of candidates is generated by eliminating the least important features from the complete feature set (Figure S1).

Feature importance

To determine the major predictors of ASCVD in our study population, the importance of each permutation feature was judged from the final model. Permutation feature importance weighs the importance of each feature by calculating the increase in the prediction error of the model after permuting its values. A feature is considered important if removing its values decreases the discriminative capability of the model, as the model depends significantly on that feature for prediction. A feature is immaterial if removing its values but the mean area under the receiver operating characteristic curve (AUC) remains the same, as the model ignores the feature for prediction in this case.

Model building and testing

The datasets were randomized into training (80%) and testing sets (20%). Model development included trials of several ML classifiers such as Artificial Neural Network (ANN), Random Forest (RF), Gradient Boosting Machine (GBM), K Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Support Vector Machine (SVM), Categorical Boosting (CatBoost). Models were trained using optimal subset and evaluated with stratified 10-fold cross-validation on the training set and we used a grid search approach to determine the appropriate hyperparameters of each ML model [9, 28] (Table S2). To solve the class imbalance in the datasets, we assigned more weights to the minority class sample to increase the misclassification cost of minority class samples. We then evaluated the performance of PCE (White), China-PAR, Recalibrated PCE (White), Recalibrated China-PAR and ML-based risk prediction model in terms of discrimination, calibration, net benefit, and net reclassification improvement (NRI) (Table S3).

Statistical analysis

Categorical variables are presented as count (%), and continuous variables are reported as mean (± SD). Brier score and Matthews correlation coefficient (MCC) were used to assessing the overall performance [29, 30]. The calibration of the models was tested with Hosmer–Lemeshow χ2 statistic [31]. Pairwise comparisons were performed between all predictive models using the DeLong test [32]. Decision-curve analysis (DCA) was used to quantify the net benefit of each risk prediction model [33]. Statistical significance was defined as two-tailed P < 0.05. All analyses were performed with R version 4.1.2.


Study population

A total of 9,609 participants (mean [SD] age: 53.4 [10.4] years; male [46.3%]) with digital ECG and Echo free of CVDs at baseline were included in the final cohort (Fig. 1). During a median of 4.7 (IQR, 4.4–4.9) years, 431 (4.5%) participants developed ASCVD (Table 1). 7,688 participants were included in the training cohort and 1,921 in the test cohort.

Fig. 1
figure 1

Flow chart of inclusion of participants for final analyses

Table 1 Baseline clinical characteristics

Performance of traditional ASCVD prediction models

Figure 2 depicted the discrimination and calibration of the PCE (White) and China-PAR models before and after recalibration. All models showed moderate discrimination, and the highest discrimination was showed in the China-PAR model with an AUC of 0.780. However, all models exhibited poor calibration with a Hosmer–Lemeshow χ2 value greater than 18 (p < 0.05). The Brier score was between 0.043 and 0.057 and MCC was between 0.186 and 0.194 (Table 2).

Fig. 2
figure 2

Discrimination and Calibration of Contemporary Prediction Models in Our Cohort. Discrimination and calibration of contemporary prediction models in each cohort. A Receiver operating characteristic curve (AUC) analysis for contemporary prediction models. B Hosmer–Lemeshow calibration plots of contemporary prediction models. Abbreviations: ReChina-PAR, Recalibrated China-PAR; RePCE, Recalibrated PCE

Table 2 Performance of risk prediction models in the test cohort

Compared of ML-based ASCVD and traditional models

Through stepwise model building and RFE algorithm, the final ML-based ASCVD models were reduced to 30 key predictor variables (Table 3). Figure 3 depicted the comparison of discrimination and calibration between the established ML classifiers. As shown, the ANN algorithm outperformed other classifiers and had the greatest AUC value and consistency. The AUC value of the ANN model was 0.800, which was higher than that in the China-PAR and PCE model (p = 0.12, 0.08). Calibration of the ANN model showed a significant improvement, the Hosmer–Lemeshow χ2 value was 9.1 (p = 0.33), and the Brier score and MCC of the ANN model were respectively 0.041 and 0.216, indicating a superior overall performance of the ANN model than traditional regression models. Decision Curve Analysis (DCA) demonstrated that ANN model provided a greater net benefit within a range of thresholds (Fig. 4). When the threshold was 5%, the ANN model had the greatest net benefit value of 0.017 among all models (Table 2). We also assess the NRI when using ML models compared to the traditional models, ANN model correctly classified more events and more non-events than China-PAR, PCE, Recalibrated China-PAR, and Recalibrated PCE (NRI: 0.355, 0.089, 0.088, 0.098, all p < 0.05) (Table 4).

Table 3 Predictor variables in ASCVD models
Fig. 3
figure 3

Discrimination and Calibration of Machine Learning-based ASCVD Models in Test Cohort. Discrimination and calibration of machine learning-based ASCVD prediction models in test cohort. A Receiver operating characteristic curve (AUC) analysis for machine learning-based ASCVD prediction models. B Hosmer–Lemeshow calibration plots of machine learning-based ASCVD prediction models. Abbreviations: ASCVD, Atherosclerotic cardiovascular disease; ANN, Artificial Neural Network; RF, Random Forest; GBM, Gradient Boosting Machine; KNN, K Nearest Neighbor; Adaboost, Adaptive Boosting; SVM, Support Vector Machine; Catboost, Categorical Boosting

Fig. 4
figure 4

Decision Curves for PCE, China-PAR and Machine Learning-based Models. Abbreviations: PCE, Pooled Cohort Equations; ReChina-PAR, Recalibrated China-PAR; ANN, Artificial Neural Network; RF, Random Forest; GBM, Gradient Boosting Machine; KNN, K Nearest Neighbor; Adaboost, Adaptive Boosting; SVM, Support Vector Machine; Catboost, Categorical Boosting

Table 4 Net reclassification improvement (NRI) in the test set

Variable importance

The leading predictors of the ANN ASCVD model are shown in Fig. 5 and a complete table of feature importance is available in Table 3. Age, SBP, R Area in V2, Max R Amplitude, and I.T Area (Full) in V2 were the most significant features to predict ASCVD.

Fig. 5
figure 5

Radar Plot for the Ten Most Important Predictors of ASCVD. As the values of feature importances were spread over a wide range (more orders of magnitude), base-10 logarithmic transformation was performed to facilitate plotting


In a community-based general population which included 11,956 adults from Northeast China, we developed seven ML models based on different algorithms. After extensive evaluation, the ANN model was chosen as the best model. The ANN model includes 30 predictors which can accurately and efficiently predict 5-year ASCVD in individuals with no history of CVD. Compared to the traditional regression models (China-PAR and PCE), the ANN model showed higher discrimination, better calibration, net benefit, and improved NRI in predicting ASCVD. Besides, our study provided a ranking of candidate variables by their significance in predicting ASCVD, which may help ASCVD risk stratification and management.

Early detection of high-risk individuals is the most effective approach to reduce the escalating incidence of ASCVD across multiple countries, while significant improvements of available prediction models were absent [34]. The PCE integrates several cardiovascular risk predictors to assess an individual’s 10-year risk of ASCVD and to guide treatment decisions. Disparities on distribution of cardiovascular risk factors existed between Asian and Western populations [35], clinical decisions may be influenced by over- or underestimation of risk. PCE provided moderate discrimination in the Korean KHS cohort, absolute 10-year ASCVD risk was overestimated by 56.5% for men and underestimated by 27.9% for women [8]. In the CHERRY Study in Southeastern China, PCE overestimated the risk by 63% in men and underestimated the risk by 34% in women [9]. In the “stroke belt” of Northern China, Fangshan Cohort Study found PCE showed an underestimate of 76.2% for men and 88.2% for women with poor calibration [10].

The China-PAR model was derived from multiple contemporary Chinese cohorts, and external validation studies of the model are limited. In the CHERRY study, the China-PAR model underestimated by 20% in men and 40% in women [9]. In the Fangshan Cohort, the China-PAR model overestimated by 29.4% risk in women [10]. When PCE and China-PAR models were applied in our cohort, we found that PCE overestimated by 63.8% in men and inversely underestimated the risk by 10.3% in women. Meanwhile, the China-PAR model underestimated the risk by 55.5% in men and 52.4% in women. After recalibration, the recalibrated PCE overestimated the risk by 144.8% in men and 129.9% in women, and the recalibrated China-PAR model inversely overestimated the risk by 98.3% in men and 64.7% in women. All PCE and China-PAR models had poor calibration, despite good discrimination. A potential reason for the differences in diverse Chinese populations is the regional disparity [36]. Residents in Northeast China tend to have a diet with high sodium and fat, which leads to the high prevalence of ASCVD up to 12.6% in this area [37].

These contemporary ASCVD risk calculators are parsimonious models based on a limited number of clinical risk variables, the potential influence of intricate and hidden interactions between weaker predictors may be overlooked. With the extension of artificial intelligence, ML algorithms have emerged as highly effective methods for resolving medical prediction puzzles in large-scale datasets and to allow guideline-directed management based on risk assessment [38]. Same as previous studies [25, 39,40,41], when compared with conventional models, our ANN-based ASCVD prediction model exhibited improved prediction performance (Table S4). The AUC of the ANN model was + 0.023, + 0.02, + 0.021, + 0.02 compared to that of PCE (White), China-PAR, Recalibrated PCE, and Recalibrated China-PAR (p = 0.08, 0.12, 0.12, 0.12), while calibration was significantly better (HL χ2 = 9.1 vs. 37.3, 67.6, 126.6, 18.6). In addition, DCA that accounts for the influence of false-negative (undertriage) and false-positive (overtriage), suggests an increased net clinical benefit with use of the ANN model as compared to traditional models if the ideal risk threshold for medical consultation lies between 5 and 10%. NRI also highlight the ability of ANN model to augment ASCVD prediction and provide a better risk stratification strategy for patients.

The observed incremental gains compared to traditional methods demonstrate the potential value of machine learning algorithms. The major advantage of ML algorithms over linear models is their capacity to capture the complex underlying interactions of myriad features and improve ex-sample predictions [42]. Althought the ML algorithms are inherently complex and difficult to interpret, ML models remain attractive due to their more accurate predictive power and the capacity to assimilate and evaluate large amounts of complex healthcare data.


Our study has several limitations. First, the ML algorithm cannot assess the independent effects of each variable on events and may be difficult to identify specific treatments to reduce individual risk. However, as the volume of data increases, ML algorithms allow for more in-depth prospective studies to identify the causative factors and interaction mechanisms. Second, longer follow-up is needed considering the chronic, progressive course of ASCVD. Third, we excluded 84 variables with > 10% missing data that might have predictive value, and the imputation of missing data might bias the analysis. However, the imputation by Multiple Imputation (MI) is known to be a precise method for imputation. Fourth, external validation studies are required to demonstrate the accuracy of the model’s predictions in diverse populations. Ultimately, the models were established using the initial follow-up blood pressure and glucose, but it may have changed during the follow-up period and this was not taken into account in the model.


In this contemporary cohort of Northeast China, we observed that the PCE and China-PAR models provided adequate discrimination but poor calibration in predicting 5-year ASCVD risk. However, the ANN-based model incorporating 30 clinical variables outperformed PCE and China-PAR, even after recalibration. Our study highlights the limitations of traditional risk prediction models for ASCVD and demonstrates the potential of machine learning algorithms in improving risk prediction accuracy. Further studies are required to demonstrate the benefits of ML algorithms and to enhance the clinician's triage decision making.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.



Atherosclerotic cardiovascular disease


Coronary Heart Disease


Cardiovascular Disease


Pooled Cohort Equations


Prediction for ASCVD Risk in China


Total Cholesterol


High-density Lipoprotein






Left Ventricular Ejection Fraction


Machine learning


WHO Quality of Life-BREF


Patient Health Questionnaire-9


Northeast China Rural Cardiovascular Health Study


Body Mass Index


Blood pressure


Myocardial Infarction


Recursive Feature Elimination


Mean Area under the Receiver Operating Characteristic Curve


Artificial Neural Network


Random Forest


Gradient Boosting Machine


K Nearest Neighbors


Adaptive Boosting


Support Vector Machine


Categorical Boosting


Matthews Correlation Coefficient


Decision-curve Analysis


  1. Roth GA, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1736–88.

    Article  Google Scholar 

  2. Xia S, Du X, Guo L, Du J, Arnott C, Lam CSP, et al. Sex Differences in Primary and Secondary Prevention of Cardiovascular Disease in China. Circulation. 2020;141(7):530–9.

    Article  PubMed  Google Scholar 

  3. Zhao D, Liu J, Wang M, Zhang X, Zhou M. Epidemiology of cardiovascular disease in China: current features and implications. Nat Rev Cardiol. 2019;16(4):203–12.

    Article  PubMed  Google Scholar 

  4. Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the Management of Blood Cholesterol: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;139(25):e1082–143.

    PubMed  Google Scholar 

  5. Whelton PK, Carey RM, Aronow WS, Casey DE Jr, Collins KJ, Dennison Himmelfarb C, et al. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA Guideline for the Prevention, Detection, Evaluation, and Management of High Blood Pressure in Adults: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Hypertension. 2018;71(6):1269–324.

    Article  CAS  PubMed  Google Scholar 

  6. Williams B, Mancia G, Spiering W, Agabiti Rosei E, Azizi M, Burnier M, et al. 2018 ESC/ESH Guidelines for the management of arterial hypertension. Eur Heart J. 2018;39(33):3021–104.

    Article  PubMed  Google Scholar 

  7. Joint Task Force for Guideline on the Assessment and Management of Cardiovascular Risk in China. Guideline on the assessment and management of cardiovascular risk in China. Zhonghua Yu Fang Yi Xue Za Zhi. 2019;53(1):13–35.

    Google Scholar 

  8. Jung KJ, Jang Y, Oh DJ, Oh BH, Lee SH, Park SW, et al. The ACC/AHA 2013 pooled cohort equations compared to a Korean Risk Prediction Model for atherosclerotic cardiovascular disease. Atherosclerosis. 2015;242(1):367–75.

    Article  CAS  PubMed  Google Scholar 

  9. Liu X, Shen P, Zhang D, Sun Y, Chen Y, Liang J, et al. Evaluation of Atherosclerotic Cardiovascular Risk Prediction Models in China. JACC: Asia. 2022;2(1):33–43.

    PubMed  PubMed Central  Google Scholar 

  10. Tang X, Zhang D, He L, Wu N, Si Y, Cao Y, et al. Performance of atherosclerotic cardiovascular risk prediction models in a rural Northern Chinese population: Results from the Fangshan Cohort Study. Am Heart J. 2019;211:34–44.

    Article  PubMed  Google Scholar 

  11. Yang X, Li J, Hu D, Chen J, Li Y, Huang J, et al. Predicting the 10-Year Risks of Atherosclerotic Cardiovascular Disease in Chinese Population: The China-PAR Project (Prediction for ASCVD Risk in China). Circulation. 2016;134(19):1430–40.

    Article  PubMed  Google Scholar 

  12. Jorgensen PG, Jensen JS, Marott JL, Jensen GB, Appleyard M, Mogelvang R. Electrocardiographic changes improve risk prediction in asymptomatic persons age 65 years or above without cardiovascular disease. J Am Coll Cardiol. 2014;64(9):898–906.

    Article  PubMed  Google Scholar 

  13. Denes P, Larson JC, Lloyd-Jones DM, Prineas RJ, Greenland P. Major and minor ECG abnormalities in asymptomatic women and risk of cardiovascular events and mortality. JAMA. 2007;297(9):978–85.

  14. Shah AJ, Vaccarino V, Janssens AC, Flanders WD, Kundu S, Veledar E, et al. An Electrocardiogram-Based Risk Equation for Incident Cardiovascular Disease From the National Health and Nutrition Examination Survey. JAMA Cardiol. 2016;1(7):779–86.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Tereshchenko LG, Henrikson CA, Sotoodehnia N, Arking DE, Agarwal SK, Siscovick DS, et al. Electrocardiographic deep terminal negativity of the P wave in V(1) and risk of sudden cardiac death: the Atherosclerosis Risk in Communities (ARIC) study. J Am Heart Assoc. 2014;3(6): e001387.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Samad MD, Ulloa A, Wehner GJ, Jing L, Hartzel D, Good CW, et al. Predicting Survival From Large Echocardiography and Electronic Health Record Datasets: Optimization With Machine Learning. JACC Cardiovasc Imaging. 2019;12(4):681–9.

    Article  PubMed  Google Scholar 

  17. Jensen MT, Sogaard P, Gustafsson I, Bech J, Hansen TF, Almdal T, et al. Echocardiography improves prediction of major adverse cardiovascular events in a population with type 1 diabetes and without known heart disease: the Thousand & 1 Study. Diabetologia. 2019;62(12):2354–64.

    Article  CAS  PubMed  Google Scholar 

  18. Goecks J, Jalili V, Heiser LM, Gray JW. How Machine Learning Will Transform Biomedicine. Cell. 2020;181(1):92–101.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Motwani M, Dey D, Berman DS, Germano G, Achenbach S, Al-Mallah MH, et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis. Eur Heart J. 2017;38(7):500–7.

    PubMed  Google Scholar 

  20. Norrish G, Ding T, Field E, Ziolkowska L, Olivotto I, Limongelli G, et al. Development of a Novel Risk Prediction Model for Sudden Cardiac Death in Childhood Hypertrophic Cardiomyopathy (HCM Risk-Kids). JAMA Cardiol. 2019;4(9):918–27.

    Article  PubMed  PubMed Central  Google Scholar 

  21. D’Ascenzo F, De Filippo O, Gallone G, Mittone G, Deriu MA, Iannaccone M, et al. Machine learning-based prediction of adverse events following an acute coronary syndrome (PRAISE): a modelling study of pooled datasets. The Lancet. 2021;397(10270):199–207.

    Article  Google Scholar 

  22. Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial Intelligence in Precision Cardiovascular Medicine. J Am Coll Cardiol. 2017;69(21):2657–64.

    Article  PubMed  Google Scholar 

  23. Ahmad T, Lund LH, Rao P, Ghosh R, Warier P, Vaccaro B, et al. Machine learning methods improve prognostication, identify clinically distinct phenotypes, and detect heterogeneity in response to therapy in a large cohort of heart failure patients. J Am Heart Assoc. 2018;7(8):e008081.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Mortazavi BJ, Downing NS, Bucholz EM, Dharmarajan K, Manhapra A, Li SX, et al. Analysis of Machine Learning Techniques for Heart Failure Readmissions. Circ Cardiovasc Qual Outcomes. 2016;9(6):629–40.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Ward A, Sarraju A, Chung S, Li J, Harrington R, Heidenreich P, et al. Machine learning and atherosclerotic cardiovascular disease risk prediction in a multi-ethnic population. NPJ Digit Med. 2020;3:125.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Nakanishi R, Slomka PJ, Rios R, Betancur J, Blaha MJ, Nasir K, et al. Machine Learning Adds to Clinical and CAC Assessments in Predicting 10-Year CHD and CVD Deaths. JACC Cardiovasc Imaging. 2021;14(3):615–25.

    Article  PubMed  Google Scholar 

  27. Tison GH, Zhang J, Delling FN, Deo RC. Automated and Interpretable Patient ECG Profiles for Disease Detection, Tracking, and Discovery. Circ Cardiovasc Qual Outcomes. 2019;12(9): e005289.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Patel AP, Wang M, Kartoun U, Ng K, Khera AV. Quantifying and Understanding the Higher Risk of Atherosclerotic Cardiovascular Disease Among South Asian Individuals: Results From the UK Biobank Prospective Cohort Study. Circulation. 2021;144(6):410–22.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Gerds TA, Cai T, Schumacher M. The performance of risk prediction models. Biom J. 2008;50(4):457–79.

    Article  PubMed  Google Scholar 

  30. Chicco D, Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics. 2020;21(1):6.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Lemeshow SHDJ. A review of goodness of fit statistics for use in the development of logistic regression models Am J Epidemiol. 1982;115(1):92–106.

    CAS  PubMed  Google Scholar 

  32. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.

  33. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74.

    Article  PubMed  PubMed Central  Google Scholar 

  34. McConnachie A, Walker A, Robertson M, Marchbank L, Peacock J, Packard CJ, et al. Long-term impact on healthcare resource utilization of statin treatment, and its cost effectiveness in the primary prevention of cardiovascular disease: a record linkage study. Eur Heart J. 2014;35(5):290–8.

    Article  CAS  PubMed  Google Scholar 

  35. Volgman AS, Palaniappan LS, Aggarwal NT, Gupta M, Khandelwal A, Krishnan AV, et al. Atherosclerotic Cardiovascular Disease in South Asians in the United States: Epidemiology, Risk Factors, and Treatments: A Scientific Statement From the American Heart Association. Circulation. 2018;138(1):e1–34.

    Article  PubMed  Google Scholar 

  36. Zhou M, Wang H, Zhu J, Chen W, Wang L, Liu S, et al. Cause-specific mortality for 240 causes in China during 1990–2013: a systematic subnational analysis for the Global Burden of Disease Study 2013. Lancet. 2016;387(10015):251–72.

    Article  PubMed  Google Scholar 

  37. Li X, Wu C, Lu J, Chen B, Li Y, Yang Y, et al. Cardiovascular risk factors in China: a nationwide population-based cohort study. The Lancet Public Health. 2020;5(12):e672–81.

    Article  PubMed  Google Scholar 

  38. Al’Aref SJ, Anchouche K, Singh G, Slomka PJ, Kolli KK, Kumar A, et al. Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging. Eur Heart J. 2019;40(24):1975–86.

    Article  CAS  PubMed  Google Scholar 

  39. Cho SY, Kim SH, Kang SH, Lee KJ, Choi D, Kang S, et al. Pre-existing and machine learning-based models for cardiovascular risk prediction. Sci Rep. 2021;11(1):8886.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Hathaway QA, Yanamala N, Budoff MJ, Sengupta PP, Zeb I. Deep neural survival networks for cardiovascular risk prediction: The Multi-Ethnic Study of Atherosclerosis (MESA). Comput Biol Med. 2021;139:104983.

    Article  CAS  PubMed  Google Scholar 

  41. Siva Kumar S, Al-Kindi S, Tashtish N, Rajagopalan V, Fu P, Rajagopalan S, et al. Machine learning derived ECG risk score improves cardiovascular risk assessment in conjunction with coronary artery calcium scoring. Front Cardiovasc Med. 2022;9:976769.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Shameer K, Johnson KW, Glicksberg BS, Dudley JT, Sengupta PP. Machine learning in cardiovascular medicine: are we there yet? Heart. 2018;104(14):1156–64.

    Article  PubMed  Google Scholar 

Download references


The authors thank healthcare workers and countless participants throughout the Liaoning provinces in the NCRCHS for their great efforts and collaboration.


This work was supported by the Science and Technology Program of Liaoning Province, China (Grant #2020JH1/10300002). The funding body did not play any role in the design of the study, data collection, analysis, interpretation, or writing of the manuscript.

Author information

Authors and Affiliations



Fan and Du are joint first authors and contributed equally to the manuscript. Sun has full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Sun, Fan, Du, and Zhou designed the study and analysis, interpreted the results, drafted the manuscript and revised it critically for important intellectual content. Zhou, Zhang, and Shi led the data collection and laboratory testing. Sun, Fan, Du, and Fu revised the manuscript. All authors reviewed and revised the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Corresponding author

Correspondence to Yingxian Sun.

Ethics declarations

Ethics approval and consent to participate

This study conformed to the ethical guidelines of the 1975 Declarations of Helsinki and was approved by the Ethics Committee of China Medical University (Shenyang, China, AF-SDP-07–1, 0–01); ID: 2018194. Every participant provided written informed consent before enrolment.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Health Survey Form for Rural Residents in Liaoning Province (Version 2012).

Additional file 2: Table S1.

List of Candidate Variables Used in this Study. Table S2. Results of Hyper-parameter Tuning with Different Algorithms in ML-based ASCVD – Models. Table S3. Estimation of Five-year ASCVD Risk Using the PCE (White) and China-PAR Models. Table S4. Comparison with Similar Studies. Figure S1. Methodology Flow Chart of Data Analysis.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, Z., Du, Z., Fu, J. et al. Comparing the performance of machine learning and conventional models for predicting atherosclerotic cardiovascular disease in a general Chinese population. BMC Med Inform Decis Mak 23, 134 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: