Table 4 Overview of number of HPO IDs found in all experiments and associated hallucinations
From: An evaluation of GPT models for phenotype concept recognition
Model | Prompt | Total found | Unique | Hall’s | Hall’s (%) | Total found | Unique | Hall’s | Hall’s (%) |
|---|---|---|---|---|---|---|---|---|---|
BASE | 2,773 | 497 | 783 | 358 | |||||
3.5 | 1 | 978 | 408 | 0 | 0 | 167 | 126 | 0 | 0 |
2 | 460 | 237 | 0 | 0 | 53 | 45 | 0 | 0 | |
3 | 7780 | 1546 | 11 | 1 | 755 | 432 | 0 | 0 | |
4 | 6698 | 1980 | 14 | 1 | 996 | 484 | 0 | 0 | |
5 | 1095 | 841 | 3 | 0 | 351 | 242 | 0 | 0 | |
6 | 771 | 491 | 4 | 1 | 442 | 316 | 0 | 0 | |
7 | 2551 | 681 | 2 | 0 | 709 | 319 | 1 | 0 | |
4 | 1 | 1617 | 634 | 1 | 0 | 666 | 390 | 0 | 0 |
2 | 1605 | 633 | 2 | 0 | 699 | 397 | 0 | 0 | |
3 | 1534 | 728 | 2 | 0 | 625 | 366 | 1 | 0 | |
4 | 2003 | 855 | 7 | 1 | 839 | 479 | 3 | 1 | |
5 | 1408 | 636 | 4 | 1 | 1287 | 648 | 3 | 0 | |
6 | 977 | 483 | 4 | 1 | 432 | 288 | 2 | 1 | |
7 | 3469 | 938 | 0 | 0 | 676 | 313 | 1 | 0 | |