Monte Carlo cross-validation for a study with binary outcome and limited sample size

Table 3 Comparison of the 12 ML methods with regards to performance metrics (accuracy and F1) and the statistical test with assumption check by using data from the ADNI study

Method	Normality	Levene test	Paired t-test		Accuracy		F1
			Test statistic	p Value	MCCV	CV	MCCV	CV
LDA	0.102	0.962	0.009	0.000	0.649	0.639	0.654	0.648
GLM	0.798	0.443	0.009	0.000	0.651	0.642	0.658	0.651
LOG	0.898	0.129	0.005	0.002	0.606	0.600	0.612	0.605
BAY	0.249	0.457	0.008	0.000	0.667	0.659	0.669	0.665
CART	0.473	0.754	0.004	0.054	0.610	0.606	0.614	0.612
TREE	0.632	0.193	0.010	0.000	0.641	0.630	0.643	0.635
KNN	0.129	0.086	0.008	0.000	0.621	0.612	0.625	0.616
RF	0.733	0.475	0.011	0.000	0.635	0.624	0.639	0.633
LVQ	0.095	0.828	0.010	0.000	0.629	0.619	0.632	0.625
SVM-L	0.591	0.288	0.009	0.000	0.654	0.646	0.661	0.652
SVM-P	0.475	0.435	0.011	0.000	0.621	0.610	0.626	0.619
SGB	0.609	0.946	0.006	0.000	0.651	0.645	0.656	0.648

ISSN: 1472-6947