Table 3 Real data experiments. CI stands for confidence interval and rf and lr stand for random forest and logistic regression, respectively
UCI dataset | Diabetic Retinopathy | Sepsis Survival |
|---|---|---|
total number of cases on test set | 575 | 42,405 |
number of negative cases on test set | 275 | 39,418 |
number of positive cases on test set | 300 | 2,987 |
logistic regr. \(\widehat{tpAUC}_{lr}\) (original estimator) | 0.216667 | 0.134550 |
logistic regr. \(\widehat{tpAUC}_{lr}\) (proposed estimator) | 0.214788 | 0.123094 |
random forest \(\widehat{tpAUC}_{rf}\) (original estimator) | 0.138909 | 0.124622 |
random forest \(\widehat{tpAUC}_{rf}\) (proposed estimator) | 0.137091 | 0.121070 |
\(\widehat{\Delta tpAUC}\) (original estimator) | 0.077758 | 0.009929 |
\(\widehat{\Delta tpAUC}\) (proposed estimator) | 0.077697 | 0.002024 |
95% Confidence Interval (original estimator) | (0.076342, 0.079173) | (0.009889, 0.009968) |
95% Confidence Interval (proposed estimator) | (0.076326, 0.079068) | (0.002019, 0.002029) |
95% CI computation time (original estimator) | 5.13 minutes | 5.12 days |
95% CI computation time (proposed estimator) | 3.24 seconds | 46.69 seconds |