Skip to main content

Table 3 Performance of the 5 base models before and after calibration

From: Applying probability calibration to ensemble methods to predict 2-year mortality in patients with DLBCL

 

AUC

ECE

MCE

P

NB

0.804 (0.773–0.839)

14.206 (11.646–16.880)

38.900 (31.675–46.575)

< 0.001(< 0.001- < 0.001)

NB-Platt

0.804 (0.773–0.839)

9.966 (8.216–11.942)

23.500 (19.100–29.925)

0.250 (0.069–0.427)

NB-IsoReg

0.749 (0.686–0.792)

12.027 (8.577–15.908)

37.450 (25.000–53.550)

0.009(< 0.001–0.275)

NB-RPR

0.794 (0.767–0.830)

9.514 (7.761–11.503)

23.800 (17.900–30.625)

0.257 (0.072–0.536)

Logit

0.803 (0.773–0.835)

9.517 (7.909–11.415)

24.400 (18.875–31.900)

0.226 (0.055–0.486)

Logit-Platt

0.803 (0.773–0.835)

10.475 (8.588–12.490)

25.200 (19.450–33.200)

0.185 (0.057–0.409)

Logit-IsoReg

0.752 (0.697–0.794)

10.897 (8.234–15.700)

33.350 (21.720–50.820)

0.026(< 0.001–0.342)

Logit-RPR

0.801 (0.771–0.832)

9.784 (8.266–11.712)

24.600 (19.050–31.025)

0.244 (0.093–0.479)

RF

0.800 (0.769–0.828)

13.569 (11.453–15.771)

36.000 (30.580–41.630)

< 0.001(< 0.001- < 0.001)

RF-Platt

0.800 (0.769–0.828)

12.122 (9.999–14.068)

28.700 (22.500–35.800)

0.101 (0.022–0.284)

RF-IsoReg

0.777 (0.748–0.811)

8.871 (6.431–11.491)

28.600 (19.575–41.075)

0.185 (0.005–0.606)

RF-RPR

0.788 (0.753–0.816)

10.070 (7.989–11.932)

26.550 (20.200–33.275)

0.198 (0.042–0.464)

SVM

0.792 (0.762–0.821)

13.225 (11.390–15.112)

32.100 (25.420–39.970)

0.014 (0.001–0.102)

SVM-Platt

0.792 (0.762–0.821)

11.514 (9.373–13.996)

27.100 (21.650–34.170)

0.133 (0.026–0.352)

SVM-IsoReg

0.743 (0.676–0.784)

11.744 (8.644–15.926)

31.350 (21.975–47.525)

0.034(< 0.001–0.331)

SVM-RPR

0.788 (0.751–0.816)

10.893 (8.880–13.453)

26.300 (20.300–34.320)

0.140 (0.025–0.418)

FNN

0.813 (0.780–0.845)

9.211 (7.075–10.391)

23.500 (17.675–28.925)

0.329 (0.113–0.585)

FNN-Platt

0.813 (0.780–0.845)

9.990 (7.925–11.745)

23.600 (17.975–30.025)

0.243 (0.074–0.496)

FNN-IsoReg

0.731 (0.663–0.776)

14.090 (9.810–19.200)

46.000 (28.400–63.400)

< 0.001(< 0.001–0.079)

FNN-RPR

0.809 (0.776–0.838)

9.910 (7.540–11.020)

23.500 (17.600–28.900)

0.287 (0.103–0.531)

  1. In each cell M (P25 - P75): M is the median, P25 is the 25th percentile and P75 is the 75th percentile of 300 tests. For each individual model, the best performance in each column is in bold and the secondary best performance in each column is underlined
  2. NB Naïve Bayes, Logit Logistic regression, RF Random forest, SVM Support vector machine, FNN Feedforward neural network, Platt Platt scaling, IsoReg Isotonic regression, RPR Shape-restricted polynomial regression. “-Platt”, “-IsoReg” and “-RPR” represent performing probability calibration using corresponding method
Back to article page

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us