Table 3 Performance evaluation of the selected ML algorithms
Classifiers | MLP | KNN | DT (j48) | Pattern recognition network | XG Boost | Probabilistic neural network | SVM (kernel = RBF) | SVM (kernel = linear) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Full feature | Selected Feature | Full feature | Selected Feature | Full feature | Selected Feature | Full feature | Selected Feature | Full feature | Selected Feature | Full feature | Selected Feature | Full feature | Selected Feature | Full feature | Selected Feature | ||
Mean Accuracy | 0.67 | 0.77 | 0.62 | 0.68 | 0.73 | 0.83 | 0.62 | 0.68 | 0.69 | 0.79 | 0.62 | 0.69 | 0.69 | 0.85 | 0.69 | 0.83 | |
95% confidence interval | (0.66, 0.68) | (0.76, 0.781) | (0.59, 0.66) | (0.671, 0.71) | (0.71, 0.75) | (0.834, 0.848) | (0.611, 0.64) | (0.68, 0.7) | (0.68, 0.7) | (0.77, 0.81) | (0.62, 0.63) | (0.691, 0.71) | (0.69, 0.71) | (0.82, 0.85) | (0.69, 0.71) | (0.82, 0.84) | |
Standard deviation | 0.01 | 0.09 | 0.05 | 0.02 | 0.02 | 0.01 | 0.02 | 0.01 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 0.02 | 0.01 | 0.01 | |
Mean Specificity | 0.68 | 0.76 | 0.62 | 0.66 | 0.74 | 0.81 | 0.62 | 0.68 | 0.68 | 0.76 | 0.62 | 0.68 | 0.69 | 0.85 | 0.69 | 0.82 | |
95% confidence interval | (0.67, 0.71) | (0.75, 0.77) | (0.58, 0.66) | (0.651, 0.71) | (0.731, 0.75) | (0.80, 0.82) | (0.61, 0.64) | (0.67, 0.691) | (0.67, 0.69) | (0.75, 0.77) | (0.62, 0.63) | (0.68, 0.7) | (0.68, 0.7) | (0.85, 0.86) | (0.68, 0.7) | (0.816, 0.83) | |
Standard deviation | 0.02 | 0.01 | 0.07 | 0.02 | 0.01 | 0.09 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | |
Mean Sensitivity | 0.71 | 0.72 | 0.62 | 0.70 | 0.74 | 0.83 | 0.61 | 0.70 | 0.70 | 0.78 | 0.62 | 0.71 | 0.71 | 0.86 | 0.70 | 0.83 | |
95% confidence interval | (0.71, 0.73) | (0.71, 0.74) | (0.57, 0.68) | (0.68, 0.73) | (0.73, 0.752) | (0.83, 0.85) | (0.591, 0.64) | (0.69, 0.72) | (0.69, 0.72) | (0.78, 0.79) | (0.61, 0.65) | (0.7, 0.73) | (0.7, 0.73) | (0.86, 0.87) | (0.7, 0.72) | (0.82, 0.84) | |
Standard deviation | 0.01 | 0.01 | 0.08 | 0.03 | 0.02 | 0.09 | 0.03 | 0.02 | 0.02 | 0.01 | 0.03 | 0.03 | 0.02 | 0.01 | 0.01 | 0.01 | |
Mean area under the curve | 0.70 | 0.76 | 0.62 | 0.69 | 0.75 | 0.83 | 0.62 | 0.69 | 0.69 | 0.76 | 0.62 | 0.70 | 0.70 | 86.1% | 0.70 | 0.83 | |
95% confidence interval | (0.69, 0.71) | (0.751, 0.774) | (0.610, 0.630) | (0.671, 0.712) | (0.731, 0.76) | (0.83, 0.85) | (0.61, 0.63) | (0.68, 0.7) | (0.68, 0.71) | (0.75, 0.778) | (0.62, 0.64) | (0.69, 0.71) | (0.69, 0.71) | (0.85, 0.86) | (0.69, 0.71) | (0.82, 0.84) | |
Standard deviation | 0.01 | 0.09 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.014 | 0.01 | |
Mean F1-score | 0.70 | 0.76 | 0.61 | 0.68 | 0.73 | 0.83 | 0.62 | 0.69 | 0.69 | 0.77 | 0.62 | 0.70 | 0.72 | 0.87 | 0.69 | 0.82 | |
95% confidence interval | (0.69, 0.71) | (0.751, 0.772) | (0.61, 0.63) | (0.671, 0.71) | (0.72, 0.74) | (0.83, 0.851) | (0.611, 0.63) | (0.68, 0.7) | (0.68, 0.71) | (0.76, 0.78) | (0.61, 0.64) | (0.69, 0.71) | (0.69, 0.71) | (0.86, 0.88) | (0.69, 0.71) | (0.821, 0.84) | |
Standard deviation | 0.01 | 0.08 | 0.01 | 0.03 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.01 | 0.01 | |
Kappa Statistic (KS) | 0.7201 | 76.2% | 0.612 | 0.681 | 0.701 | 83.2% | 0.622 | 0.671 | 0.621 | 78.2% | 0.602 | 0.718 | 0.752 | 0.861 | 0.681 | 0.831 | |
(0.71, 0.73) | (0.75, 0.771) | (0.61, 0.63) | (0.66, 0.69) | (0.70, 0.71) | (0.828, 0.85) | (0.59, 0.63) | (0.66, 0.69) | (0.61, 0.63) | (0.77, 0.79) | (0.58, 0.62) | (0.68, 0.73) | (0.74, 0.76) | (0.85, 0.86) | (0.67, 0.70) | (0.82, 0.84) | ||
0.01 | 0.01 | 0.01 | 0.02 | 0.00 | 0.08 | 0.07 | 0.01 | 0.02 | 0.01 | 0.05 | 0.04 | 0.01 | 0.01 | 0.01 | 0.01 | ||