Skip to main content

Table 3 Description of Errors in NLP Performance, by Error Type

From: Development and validation of a pragmatic natural language processing approach to identifying falls in older adults in the emergency department

Note Texta Error Assessment
False Positive
 “Chief Complaint Patient presents with Other weakness Fall” Chief complaint was in error. Visit was for causes unrelated to a fall (diarrhea and weakness related to c. diff and sepsis). No fall mentioned in the note.
 “He was pulling a frozen chicken package out of freezer when it fell and struck his left lower leg anterior aspect” Something else fell
 “Had a fall on [DATE] with left knee laceration with minimal blood loss and no other injuries” Note mentions prior fall (~ 3 weeks before) in HPI section, only as context for reason for visit (low hemoglobin counts)
 “She has had multiple UTIs in the fall 2016” Refers to the season “Fall”. Algorithm excludes for this, but only when written as “Fall of 20XX”
 “PMH: Past Medical History...- s/p fall - ” Algorithm mistakenly pulled this from past history section
 “Patient has a known left-sided ovarian mass which her primary care physician and her husband falling.” Transcription error in note (physician likely meant to write “following”)
 “Pt states she fell apart after her divorce, that there is nothing that makes her happy” Use of an idiom that contained the word fall
 “The patient is unable to confirm falling”...
 “Son unaware and unable to confirm if patient fell or not”
Uncertainty about whether fall occurred; phrase “unable to confirm” was not included in algorithm for exclusion
 “Negative for falls” Negation term not included in algorithm
 “Negative for back pain, falls and neck pain” Negation term not included in algorithm
False Negative
 “...presents to the ED with complaints of left wrist pain sustained after a FOOSH injury earlier today while playing tennis.” Use of acronym that describes the incident (‘Fall On OutStretched Hand’), rather than the word ‘fall’)
 “Patient reports earlier this morning she was up in her bathroom attempting to use the toilet, sat down, missed the toilet and landed on her right hip.” Word(s) other than “fall” used to describe fall incident in the note
 “She was noted to have a syncopal event. She went down to the ground and struck her head...”; “...she syncopized at her aerobics class today. She does not remember the events immediately before or after.,, She did hit her head.” Word(s) other than “fall” used to describe fall incident in the note
 “He says that he intermittently woke up on the floor feeling unusual… it took him several minutes to realize that he had hit his head and that he had a large hematoma on the back left side of his head” Word(s) other than “fall” used to describe fall incident in the note
 “...presents with CT angiogram showing ‘occlusion’. Patient fell 1 month ago, had wound to her R knee. This opened into a sore, she has been managing this with her physician.” Fall was not recent, but contributed to the cause of the ED visit.
  1. aEllipses added in place of deleted text