Table 3 Optimal hyper-parameters across each of 5 “outer” folds in nested cross-validation
Parameter | Fold 1 | Fold 2 | Fold 3 | Fold 4 | Fold 5 |
|---|---|---|---|---|---|
Linear SVM | |||||
Cost | 0.1 | 25 | 25 | 25 | 25 |
Radial SVM | |||||
Cost | 25 | 50 | 50 | 50 | 50 |
Gamma | 0.1 | 0.1 | 0.1 | 0.1 | 0.1 |
Single-layer NN | |||||
Size of hidden layer | 15 | 20 | 20 | 20 | 1 |
Maximum # iterations | 100 | 200 | 200 | 200 | 100 |
Decay | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 |
Triple-layer DNN | |||||
Size of 3 hidden layers | 20, 20, 20 | 20, 20, 20 | 20, 20, 20 | 20, 20, 20 | 20, 20, 20 |
Learning rate | 1 | 1 | 1 | 1 | 1 |
Momentum | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
Number of epochs | 20 | 20 | 20 | 20 | 20 |
XG Boost | |||||
Max depth | 20 | 20 | 6 | 20 | 6 |
Eta | 0.9 | 0.9 | 0.01 | 0.9 | 0.01 |
Nrounds | 200 | 200 | 50 | 200 | 50 |
Gamma | 10 | 10 | 0 | 10 | 0 |
Min. child weight | 10 | 10 | 0 | 10 | 0 |
Ratio of columns per tree | 1.0 | 1.0 | 0.1 | 1.0 | 0.1 |