BMC Medical Informatics and Decision Making

Table 2 Model performance measured by area under the receiver operating characteristic (AUROC-) and area under precision recall (AUPR-) curves [95% CI]

From: Impact of the Covid-19 pandemic on the performance of machine learning algorithms for predicting perioperative mortality

training set	validation parameter	validation set
		pre-pandemic	1^st wave	after 1^st wave	2^nd wave	after 2^nd wave
		06/2014–03/2020	04/2020–05/2020	06/2020–09/2020	10/2020–05/2021	06/2021–10/2021
		XGBoost
pre-pandemic (model 1)	AUROC	0.951 [0.941–0.962]	0.914 [0.871–0.957]	0.931 [0.909–0.953]	0.944 [0.929–0.959]	0.944 [0.927–0.961]
pre-pandemic (model 1)	AUPR	0.144 [0.140–0.149]	0.074 [0.064–0.086]	0.150 [0.141–0.159]	0.177 [0.171–0.184]	0.118 [0.111–0.125]
pre-pandemic + 1^st wave (model 2)	AUROC	0.923 [0.907–0.940]		0.907 [0.870–0.943]	0.942 [0.924–0.959]	0.937 [0.917–0.958]
pre-pandemic + 1^st wave (model 2)	AUPR	0.142 [0.138–0.147]		0.052 [0.041–0.066]	0.174 [0.169–0.179]	0.136 [0.129–0.144]
whole set (model 3)	AUROC	0.941 [0.927–0.954]
whole set (model 3)	AUPR	0.168 [0.164–0.173]
		Deep Learning
pre-pandemic (model 1)	AUROC	0.942 [0.921–0.962]	0.914 [0.855–0.975]	0.907 [0.861–0.953]	0.958 [0.945–0.971]	0.899 [0.854–0.945]
pre-pandemic (model 1)	AUPR	0.187 [0.182–0.192]	0.074 [0.064–0.085]	0.160 [0.151–0.169]	0.193 [0.186–0.200]	0.145 [0.138–0.153]
pre-pandemic + 1^st wave (model 2)	AUROC	0.877 [0.850–0.905]		0.747 [0.608–0.887]	0.912 [0.888–0.935]	0.884 [0.838–0.930]
pre-pandemic + 1^st wave (model 2)	AUPR	0.080 [0.076–0.083]		0.041 [0.032–0.054]	0.106 [0.101–0.112]	0.073 [0.068–0.079]
whole set (model 3)	AUROC	0.885 [0.862–0.908]
whole set (model 3)	AUPR	0.089 [0.085–0.092]

Back to article page

ISSN: 1472-6947

Contact us

General enquiries: journalsubmissions@springernature.com