Skip to main content

Table 2 Inter-annotator agreement measured in Cohen’s Κ between two human annotators in ESS (266 doubly annotated reports) & NHS Tayside (100 doubly annotated reports) for different phenotypes (document-level annotations)

From: A validated natural language processing algorithm for brain imaging phenotypes from radiology reports in UK electronic health records

 

Κ ESS

Κ NHS Tayside

Atrophy

 

0.95

0.97

Small vessel disease

0.98

0.97

Stroke

 Underspecified

0.99

 Intracerebral haemorrhage

Deep, old

1.00

Deep, recent

1.00

Lobar, old

1.00

1.00

Lobar, recent

1.00

0.80

Underspecified

1.00

0.88

 Ischaemic stroke

Cortical, old

0.97

1.00

Cortical, recent

0.98

Deep, old

0.97

1.00

Deep, recent

1.00

1.00

Underspecified

0.95

0.71

Haemorrhagic transformation

1.00

Other intracranial haemorrhage

 Subdural hematoma

1.00

1.00

 Subarachnoid haemorrhage

Aneurysmal

Other

0.99

0.66

 Microbleeds

Deep

1.00

Lobar

1.00

Underspecified

1.00

Tumours

 Glioma

 

 Meningioma

1.00

 Metastasis

0.85

 Other

0.98

0.39