Table 2 Proportion of alerts (mean and standard error) given either a suggestion of no action (i.e. no concern indicated either directly following review of the data or after the simulated call) or action (concern indicated with either follow-up, medication adjustment or sending a community nurse) together with the percentage of raw agreement (i.e. the average proportion of rater-pairs agreeing of the total possible rater-pair agreements) with bootstrapped 95% confidence intervals. The Fleiss kappa statistic excluded cases in which not all clinicians provided a rating (total: arm A, 281 cases with 8 raters; arm P, 147 + 85 cases with 4 + 4 raters)
Indicated response | Arm A | Arm P | |
|---|---|---|---|
No action | Proportion of alerts | 50.5 ± 6.2% | 40.6 ± 7.1% |
Agreement across participants on response | 66.8% (63.4–69.8) | 57.5% (52.1–62.4) | |
Action (concern indicated) | Proportion of alerts | 49.5 ± 6.2% | 59.4 ± 7.1% |
Agreement across participants on response | 66.4% (62.7–69.6) | 72.8% (69.0–76.3) | |
Fleiss’s Kappa statistic | .334 (.329–.340) | .305 (.291–.319) |