Skip to main content

Table 2 Inter-annotator agreement across several rounds of corpus annotation as measured by Cohen’s Kappa

From: Identification of research hypotheses and new knowledge from scientific literature

  

Round 1

Round 2

Round 3

Research Hypothesis

EU-ADR

0.486

0.724

0.761

 

GENIA-MK

0.593

0.859

0.855

New Knowledge

EU-ADR

0.627

0.825

0.842

 

GENIA-MK

0.772

0.895

0.895

  1. We show that agreement increased throughout the annotation process as we discussed difficult cases with annotators. We undertook regular meetings with the annotators to quickly resolve any disagreements