Table 3 Model performance
ML algorithm | Parameters | Evaluation metrics | Without class decompositions (%) | With one vs. one class decomposition (%) | With one vs. rest class decomposition (%) |
|---|---|---|---|---|---|
Decision tree | criterion = 'entropy',max_features = 'sqrt',min_samples_split = 12,random_state = 0,max_depth = 30, max_leaf_nodes = 600 | Accuracy | 79.38 | 89.88 | 89.09 |
precision | 79.09 | 89.81 | 89.01 | ||
Recall | 79.21 | 89.77 | 88.98 | ||
F1_score | 79.03 | 89.71 | 88.96 | ||
Cross-validation | 68.48 | 84.27 | 83.17 | ||
ROC | 95.6 | 95.6 | 95.6 | ||
Random forest | criterion = 'entropy', max_features = 'sqrt', min_samples_split = 3, n_estimators = 500, random_state = 0, max_depth = 20, max_leaf_nodes = 400, n_jobs = -1 | Accuracy | 91.34 | 94.4 | 94.4 |
Precision | 91.32 | 94.36 | 94.37 | ||
Recall | 91.28 | 94.35 | 94.35 | ||
F1_score | 91.25 | 94.34 | 94.34 | ||
Cross-validation | 81.23 | 89.37 | 88.18 | ||
ROC | 99 | 99 | 99.43 | ||
Cat boost | depth = 10, iterations = 300, l2_leaf_reg = 1, learning_rate = 0.15 | Accuracy | 97.08 | 97.44 | 97.595 |
Precision | 97.09 | 97.438 | 97.596 | ||
Recall | 97.05 | 97.418 | 97.574 | ||
F1_score | 97.06 | 97.422 | 97.58 | ||
Cross-validation | 95.94 | 96.478 | 96.482 | ||
ROC | 99.9 | 99.94 | 99.9 | ||
Extreme gradient Boost | max_depth = 3, learning_rate = 0.1, n_estimators = 100, silent = True, objective = 'binary: logistic’ booster = 'gbtree', n_jobs = 1, nthread = None | Accuracy | 94.26 | 95.21 | 94.54 |
Precision | 94.27 | 95.20 | 94.53 | ||
Recall | 94.20 | 95.16 | 94.48 | ||
F1_score | 94.20 | 95.16 | 94.48 | ||
Cross-validation | 88.86 | 91.73 | 89.72 | ||
ROC | 99.53 | 99.53 | 99.54 |