Algorithm | FMA | FMA | MetaMap | MetaMap |
---|---|---|---|---|
Vocabulary | Read/OXMIS | Read/OXMIS | Read/OXMIS | Full Read |
Test set | Death | General | General | General |
Number of texts | 1000 | 1000 | 1000 | 1000 |
Number of words | 7534 | 25981 | 25981 | 25981 |
Positive diagnoses detected in free text | ||||
True positives | 683 | 346 | 286 | 273 |
False positives | 11 | 32 | 126 | 18 |
False negatives | 52 | 101 | 161 | 174 |
Precision, % | 98.4 (97.2, 99.2) | 91.5 (88.3, 94.1) | 69.4 (64.7, 73.8) | 93.8 (90.4, 96.3) |
Recall, % | 92.9 (90.8, 94.7) | 77.4 (73.2, 81.2) | 64.0 (59.3, 68.4) | 61.1 (56.4, 65.6) |
F-score | 0.96 | 0.84 | 0.67 | 0.74 |
Strictly defined precision for positive diagnoses (best term and correct attribute) | ||||
Number strictly correct | 625 | 315 | 260 | 247 |
Precision strict, % | 90.1 (87.6, 92.2) | 83.3 (79.2, 86.9) | 63.1 (58.2, 67.8) | 84.9 (80.2, 88.8) |
Precision of non-diagnosis positive concepts | ||||
True positives | 84 | 304 | 295 | 453 |
False positives | 2 | 22 | 55 | 41 |
Precision, % | 97.7 (91.9, 99.7) | 93.3 (90.0, 95.7) | 84.3 (80.0, 87.9) | 91.7 (88.9, 94.0) |
Overall precision of positive concepts detected (diagnostic and non-diagnostic) | ||||
True positives | 767 | 650 | 581 | 726 |
False positives | 13 | 54 | 181 | 59 |
Precision, % | 98.3 (97.2, 99.1) | 92.3 (90.1, 94.2) | 76.2 (73.1, 79.2) | 92.5 (90.4, 94.2) |
Precision of negative concepts detected | ||||
True positives | 5 | 57 | 0 | 92 |
False positives | 5 | 18 | 0 | 33 |
Precision, % | 50.0 (18.7, 81.3) | 76.0 (64.7, 85.1) | 73.6 (65.0, 81.1) | |
Texts for which algorithm suggested a better Read term than the original term | ||||
Percentage of texts | 0 | 1.2 | 0.5 | 0.6 |
Dates and durations | ||||
True positives | 116 | 96 | ||
False positives | 15 | 10 | ||
False negative | 25 | 22 | ||
Precision, % | 88.5 (81.8, 93.4) | 90.6 (83.3, 95.4) | ||
Recall, % | 82.3 (74.9, 88.2) | 81.4 (73.1, 87.9) | ||
F-score | 0.85 | 0.86 | ||
Test results and quantitative measurements | ||||
True positives | 105 | |||
False positives | 11 | |||
False negatives | 18 | |||
Precision, % | 90.5 (83.7, 95.2) | |||
Recall, % | 85.4 (77.9, 91.1) | |||
F-score | 0.89 |