Skip to main content

Table 3 Influence of signal-to-noise difference (SND) characteristics on the performance metrics of the detection algorithms and the fusion methods

From: Using decision fusion methods to improve outbreak detection in disease surveillance

  Sensitivity per outbreak Sensitivity per day Specificity PPV NPV Cases required Proportion of delay Time to detection AUC AMOC AUWROC
Positive SND: scenario with a SND = 65.4
 CUSUM 1 0.74 0.83 0.29 0.97 0.17 0.25 4 0.89 0.90 0.81
 C1 1 0.25 0.99 0.69 0.93 0.08 0.15 2 0.59 0.92 0.57
 C2 1 0.38 0.99 0.77 0.94 0.08 0.14 2 0.66 0.92 0.64
 C3 1 0.54 0.96 0.61 0.95 0.09 0.19 2 0.77 0.90 0.72
 Farrington 1 0.42 1.00 0.99 0.95 0.18 0.21 2 0.84 0.93 0.79
 EWMA 1 0.58 0.97 0.67 0.96 0.14 0.22 2 0.76 0.90 0.70
 Majority voting 1 0.56 1.00 0.98 0.96 0.10 0.17 2 0.78 0.91 0.73
 Weighted majority voting 1 0.53 1.00 0.99 0.96 0.13 0.22 2 0.77 0.89 0.71
 Logistic regression 1 0.59 0.99 0.92 0.96 0.09 0.16 2 0.84 0.94 0.80
 CARTa 1 0.56 1.00 0.99 0.96 0.12 0.19 2 0.83 0.92 0.78
 Bayesian Networks 1 0.56 1.00 1.00 0.96 0.12 0.19 2 0.90 0.93 0.84
Quasi-null SND: scenario with a SND = −1.4
 CUSUM 1 0.61 1.00 0.93 0.96 0.49 0.38 5 0.86 0.88 0.77
 C1 1 0.17 0.99 0.64 0.92 0.24 0.27 4 0.55 0.89 0.53
 C2 1 0.28 0.99 0.75 0.93 0.24 0.26 4 0.61 0.89 0.58
 C3 1 0.39 0.97 0.56 0.94 0.36 0.31 5 0.72 0.86 0.66
 Farrington 1 0.27 1.00 1.00 0.93 0.35 0.34 4 0.80 0.91 0.74
 EWMA 1 0.51 0.94 0.46 0.95 0.20 0.24 4 0.76 0.90 0.70
 Majority voting 1 0.42 1.00 0.99 0.94 0.25 0.28 4 0.71 0.86 0.65
 Weighted majority voting 1 0.38 1.00 1.00 0.94 0.34 0.33 4 0.50 0.50 0.50
 Logistic regression 1 0.70 0.99 0.93 0.97 0.22 0.27 4 0.86 0.88 0.77
 CARTa 1 0.68 1.00 0.93 0.97 0.25 0.27 4 0.84 0.86 0.75
 Bayesian Networks 1 0.70 0.99 0.94 0.97 0.23 0.27 4 0.86 0.88 0.77
Negative SND: scenario with a SND = −89.2
 CUSUM 0.29 0.03 1.00 0.96 0.91 0.87 0.77 11 0.65 0.82 0.59
 C1 0.51 0.05 0.99 0.25 0.91 0.73 0.64 5 0.52 0.86 0.49
 C2 0.60 0.07 0.98 0.30 0.91 0.70 0.60 5 0.55 0.86 0.51
 C3 0.78 0.16 0.96 0.27 0.92 0.62 0.50 6 0.59 0.82 0.54
 Farrington 0.67 0.09 0.99 0.46 0.92 0.64 0.55 5 0.60 0.87 0.56
 EWMA 0.98 0.18 0.95 0.25 0.92 0.47 0.37 5 0.59 0.87 0.54
 Majority voting 0.60 0.07 0.99 0.45 0.92 0.71 0.61 5 0.53 0.69 0.51
 Weighted majority voting 0.53 0.06 1.00 0.69 0.91 0.75 0.65 5 0.55 0.72 0.52
 Logistic regression 0.29 0.03 1.00 0.96 0.91 0.87 0.77 11 0.60 0.81 0.55
 CARTa 0.29 0.03 1.00 0.96 0.91 0.87 0.77 11 0.59 0.77 0.54
 Bayesian Networks 0.51 0.06 1.00 0.96 0.91 0.80 0.68 7 0.60 0.81 0.55
  1. aCART Classification and Regression Trees, PPV Positive Predictive Values, NPV Negative Predictive Values, AUC Area Under the ROC (Receiver Operating Characteristic) Curve, Cases required proportion of cases needed for outbreak detection, Proportion of delay = 1 – timeliness score, that is: 1- (sum of time to detection) / outbreak duration, AMOC Activity Monitor Operating Characteristic, AUWROC Area Under Weighted ROC. Positive SND: scenario generated with a daily incidence of 1 for the baseline and an outbreak magnitude of 100 (SND = 65.4), Quasi-null SND scenario generated with an daily incidence of 1 for the baseline and an outbreak magnitude of 30 (SND = −1.4), Negative SND scenario generated with a daily incidence of 3 for the baseline and an outbreak magnitude of 10 (SND = −8)