Contextual property detection in Dutch diagnosis descriptions for uncertainty, laterality and temporality

Table 2 Interrater reliability and kappa scores for the internal (n = 980) and multicenter (n = 996) validation set

Property (n)/dataset	Interrater reliability (%)^a		Kappa score
Property (n)/dataset	Internal validation set (n = 980)	Multicenter validation set (n = 996)	Internal validation set (n = 980)	Multicenter validation set (n = 996)
Laterality	98.0 (N_ESK = 245, N_FJP = 233)	98.1 (N_ESK = 288, N_FJP = 293)	0.94	0.95
Temporality	97.7 (N_ESK = 163, N_FJP = 157)	98.1 (N_ESK = 96, N_FJP = 85)	0.91	0.88
Uncertainty	96.1 (N_ESK = 135, N_FJP = 107)	97.5 (N_ESK = 98, N_FJP = 87)	0.82	0.85
Removal of uncertainty	98.3 (N_ESK = 11, N_FJP = 14)	99.3 (N_ESK = 8, N_FJP = 11)	0.36	0.63

Descending on Kappa scores
^aN_ESK is the sum of the records in the corresponding property according to annotator ESK and N_FJP is the sum of the records in the corresponding property according to annotator FJP

ISSN: 1472-6947