Deep learning approach to detection of colonoscopic information from unstructured reports

Table 7 Comparison of LSTM and BioBERT variations

Model	Loss function ^a & optimizer ^b	Precision ^c	Recall ^c	F1 score ^c
LSTM	CCE + ADAM	0.5267	0.5297	0.5282
LSTM	CCE + NADAM	0.5258	0.5285	0.5271
LSTM	CCE + RMS	0.5266	0.5297	0.5281
LSTM	KL + ADAM	0.5255	0.5286	0.5270
LSTM	KL + NADAM	0.5258	0.5287	0.5273
LSTM	KL + RMS	0.5260	0.5278	0.5269
LSTM	POISSON + ADAM	0.5255	0.5274	0.5264
LSTM	POISSON + NADAM	0.5245	0.5267	0.5256
LSTM	POISSON + RMSProp	0.5229	0.5258	0.5244
Bi-LSTM	CCE + ADAM	0.5880	0.6761	0.6290
Bi-LSTM	CCE + NADAM	0.5971	0.7056	0.6460
Bi-LSTM	CCE + RMSProp	0.5884	0.6763	0.6293
Bi-LSTM	KL + ADAM	0.5881	0.6768	0.6294
Bi-LSTM	KL + NADAM	0.5957	0.7039	0.6445
Bi-LSTM	KL + RMSProp	0.5884	0.6767	0.6295
Bi-LSTM	POISSON + ADAM	0.5873	0.6756	0.6284
Bi-LSTM	POISSON + NADAM	0.5949	0.7021	0.6433
Bi-LSTM	POISSON + RMSProp	0.5869	0.6758	0.6282
Bi-LSTM-CRF	CRF + ADAM	0.9828	0.9842	0.9835
Bi-LSTM-CRF	CRF + NADAM	0.9825	0.9851	0.9838
Bi-LSTM-CRF	CRF + RMSProp	0.9844	0.9853	0.9848
BioBERT	CCE + ADAM	0.9824	0.9821	0.9822
BioBERT-CRF	CRF + ADAM	0.9810	0.9815	0.9812

^aLoss functions; CCE = categorical cross-entropy, KL = Kullback–Leibler divergence, POISSON = Poisson distribution
^bOptimizers; ADAM = Adaptive Moment Estimation, NADAM = Nesterov-accelerated Adaptive Moment Estimation, RMSProp = Root Mean Square Propagation
^cThe best results are marked in bold

ISSN: 1472-6947