- Research
- Open access
- Published:
Using a decision tree algorithm to distinguish between repeated supra-therapeutic and acute acetaminophen exposures
BMC Medical Informatics and Decision Making volume 23, Article number: 102 (2023)
Abstract
Background
This study aimed to compare clinical and laboratory characteristics of supra-therapeutic (RSTI) and acute acetaminophen exposures using a predictive decision tree (DT) algorithm.
Methods
We conducted a retrospective cohort study using the National Poison Data System (NPDS). All patients with RSTI acetaminophen exposure (n = 4,522) between January 2012 and December 2017 were included. Additionally, 4,522 randomly selected acute acetaminophen ingestion cases were included. After that, the DT machine learning algorithm was applied to differentiate acute acetaminophen exposure from supratherapeutic exposures.
Results
The DT model had accuracy, precision, recall, and F1-scores of 0.75, respectively. Age was the most relevant variable in predicting the type of acetaminophen exposure, whether RSTI or acute. Serum aminotransferase concentrations, abdominal pain, drowsiness/lethargy, and nausea/vomiting were the other most important factors distinguishing between RST and acute acetaminophen exposure.
Conclusion
DT models can potentially aid in distinguishing between acute and RSTI of acetaminophen. Further validation is needed to assess the clinical utility of this model.
Introduction
Acetaminophen is a commonly used antipyretic and analgesic drug available over the counter [1]. In adults, the safe dose of acetaminophen is between 325 and 650 mg every 4–6 h, with a maximum of 4 g per day [2]. Overdose of acetaminophen can be severe, leading to liver failure or even death [3]. Acetaminophen poisoning is one of the most common causes of acute liver failure in the US and worldwide [4]. Although most patients with acetaminophen poisoning have mild-to-moderate consequences, 29% of patients with acetaminophen-induced liver failure require a liver transplant and have a mortality rate of 28% [5].
Most commonly, acetaminophen overdose occurs after ingesting acetaminophen, within 8 h or less, in amounts that can cause toxicity (e.g., > 4 g/24 hours). Although hepatotoxicity may occur from acute or repeated acetaminophen ingestion, mortality from acetaminophen poisoning is uncommon (death occurred in 0.3% of cases treated with N-acetylcysteine). Chronic exposure is termed repeated supra-therapeutic ingestion (RSTI) to separate it from regular therapeutic use. In adults, RSTI of acetaminophen occurs when the intake of acetaminophen exceeds 8 h, resulting in a cumulative dose of more than 200 mg/kg/day (or 10 g/day, whichever is less) within 24 h or more than 150 mg/kg/day (or 6 g/day, whichever is less) within 48 h [6, 7]. In addition, for children under six years of age, RSTI is the repeated consumption of acetaminophen of more than 100 mg/kg/day for 72 h [7].
Contrasting with acute exposure cases with known ingestion time, in RSTI, the acetaminophen nomogram-based approach is not applicable. There are few studies and guidelines vary in what constitutes RSTI and when to start N-acetylcysteine. Typically, N-acetylcysteine is recommended in patients with alanine aminotransferase (ALT) greater than 50 IU/L or serum acetaminophen concentration greater than 20 mg/L (132 µmol /L) [8, 9]. Daly et al. studied 249 RSTI of acetaminophen and found that patients with AST < 50 U/L and acetaminophen concentrations less than 10 mg/L had a lower risk of hepatotoxicity [10, 11]. As management of RSTI and acute acetaminophen ingestion may differ, it is crucial to differentiate between these two overdose presentations.
Machine learning (ML) is a new approach that has recently been used in medicine for disease diagnosis, treatment decisions and prognosis [12]. ML uses algorithms and mathematical methods to identify patterns and relationships between variables and to contribute to the predictability of the target variable [13]. Supervised and unsupervised learning are two main ML approaches to classifying patients and creating risk detection models [14]. DT models are one of the most commonly used ML algorithms to classify medical data to inform appropriate medical decision making [15,16,17,18]. This study uses a DT model to help distinguish between RSTI and acute acetaminophen ingestion using clinical and laboratory characteristics from the National Poison Data System (NPDS).
Methods
Study design
This was a retrospective cohort study using NPDS, a de-identified national repository for poison control center data used by the American Association of Toxic Control Centers (AAPCC). The AAPCC maintains case records of self-reported information collected by callers during exposure management and poison information calls managed by the 55 poison control centers across the United States. Poison control centers submit de-identified case data to the NPDS after providing poison exposure management. As soon as information regarding this file is available, it is uploaded to the NDPS and recorded by poison center staff.
The NPDS includes clinical effects and interactions with the agent, duration of effect, end-organ effects, chronicity, demographic data (age, sex, and weight), administration sites, clinical findings, exposure information, and categorical laboratory findings. Each of these was included in the model. In addition, we defined “related to the exposure” to be appropriate when: (1) the timing of clinical effect is consistent with the exposure recorded (clinical effects recorded matches the half-life of toxicant in the body); (2) the severity of the clinical effect is consistent with the reported exposure; (3) the clinical effect is consistent with the expected toxicity of the substance; and (4) a physician assessed the relationship’s clinical significance.
Last, correlational analyses do not necessarily indicate causality. Clinical effects are not related to exposure if they existed before the exposure and did not increase or worsen as a result of the exposure or if the effect can be attributed to an alternative cause. All methods were carried out in accordance with relevant guidelines and regulations.
While most cases are “closed” within a few hours of the initial contact, some exposures are followed to obtain the patient’s medical outcome and may remain open for months. Follow-up calls provide a proven mechanism for monitoring the appropriateness of management recommendations, enabling continual updates of case information, augmenting patient guidelines, providing poison prevention education, and obtaining final medical outcomes to make the data collected as accurate and complete as possible.
Study population
All patients with RSTI of acetaminophen (n = 4,522) were included between January 2012 and December 2017. In addition, 4,522 randomly selected acute acetaminophen ingestion cases were included. Criteria for exclusion were missing demographic data and clinical findings. We assumed that duplicated data was not included.
We obtained de-identified data from NPDS. Based on the Colorado Multiple Institutional Review Board (COMIRB) on Human Subjects Protection guidelines and procedures, the analysis of NPDS data for this research did not meet the criteria for human subjects research according to the 45 Code of Federal Regulations (CFR) 46.101(b). Therefore, this study was determined to be exempt (COMIRB#: 22-1088).
ML-based approach
We used a DT model using Statistical Package for the Social Sciences (SPSS) 26 and Python (version 3.9) for distinguishing between RSTI and acute acetaminophen ingestion using clinical and laboratory characteristics extracted from the NPDS. The DT model uses split criteria to divide an end node and set a tree with a rate for each predictor variable [19]. It divides the data into binary parts and builds a binary tree based on them so that two edges come out of each inner node, and the resulting trees are pruned. This algorithm is used to create regression and classification trees. A DT model evaluates the variable that best divides the data [20] using the Gini index criterion to decide how to select tree nodes. The Gini Cost Function shows how pure the nodes are, wherein the purity of the node refers to the degree to which the training data assigned to each node is combined. The division continues until the node has the minimum number of training samples or exceeds the maximum depth of the tree. The root node is the most critical variable that starts the decision tree graph. It is the variable that best divides the data in a DT approach. Intermediate nodes are those in which variables are estimated but not the ultimate nodes in which predictions are formed.
Similar to other ML methods, standard performance measures were used to measure the performance of the DT model, including precision, recall, accuracy, F1 score, and confusion matrices. A confusion matrix was generated with all the required information to measure specificity, accuracy, and sensitivity [13].
Statistical analysis
Analysis was performed using Python 3.9 and SPSS 26. Chi-square and student t-tests were used to compare data between groups. A P-value of < 0.05 was determined to be statistically significant.
Results
In total, 4,522 patients with RSTI of acetaminophen were included. The baseline and clinical characteristics of patients were compared between two groups, and their results are presented in Table 1. The DT model produced in this study was 29 nodes in size, with 17 leaves and four levels (Fig. 1).
Age, followed by serum aminotransferase concentrations, abdominal pain, and drowsiness/lethargy, were the most important factors in distinguishing between RSTI and acute acetaminophen exposures (Fig. 1). The rules derived from the DT are shown in Table 2. In Fig. 2, we evaluated important signs and symptoms that affect the DT model performance (i.e., feature importance) in the classification task. Serum aminotransferase concentrations, abdominal pain, nausea/vomiting, and drowsiness/lethargy contributed most to classifying acute and RSTI, respectively.
The DT model had an accuracy, precision, recall, and F1-score of 0.75 each in the cross-validation method (Table 3). Confusion matrices are shown in Table 4. The model could correctly predict 3,361 cases of acute acetaminophen poisoning and 3,381 RSTIs. Figure 3 shows the ROC curve for the DT model (AUC = 0.81).
Discussion
We assessed the efficacy and accuracy of a DT model to distinguish acute and RSTI of acetaminophen using retrospective large-scale NPDS dataset analysis. Our DT model had acceptable characteristics and could correctly predict 3,361 cases of acute acetaminophen poisoning and 3,381 RST exposures. These findings show promise in using ML to help differentiate acute ingestions from RSTIs.
In our DT model, the most relevant predictive variable was age. This result is unsurprising given that elderly patients are more likely to use acetaminophen as an analgesic due to various ailments, raising the possibility of poisoning due to RSTI. Our study found that most RSTI exposures tend to produce aminotransferase concentrations between 100 and 1,000 IU/L or greater than 1,000 IU/L. Prior studies have found that all cases with RSTI who developed hepatotoxicity were found to have abnormal ALT (more than 50 IU/L) [8] and more likely to have encephalopathy requiring renal replacement therapy, mechanical ventilation, or death [21]. The risk of liver failure has also been seen in patients with RSTI compared to acute overdose cases [22]. This may be partly due to the delayed treatment of RSTI patients as compared to acute acetaminophen exposure cases, which typically undergo treatment within eight hours of exposure [23, 24].
Our data also showed that acidosis is another important variable in RSTI exposures. Anion gap metabolic acidosis can occur in patients with acetaminophen overdose-induced hepatotoxicity and occasionally precedes hepatic injury [25]. In addition, studies showed that electrolyte abnormalities due to an acetaminophen overdose correlate with metabolic acidosis [26, 27].
We also found that patients with RSTI of acetaminophen had increased creatinine concentrations and prolonged PT/INR compared to the acute group. Other studies have shown that patients with RSTI have increased creatinine concentrations, and serum INR had a significant association with hepatotoxicity. All patients who died or had a liver transplant showed increased creatinine concentrations [23]. It is possible that people with RSTI had an acute illness (e.g., hepatitis, dehydration) causing the acute kidney injury and minor elevation in transaminases. The dataset does not include information about the prior or concomitant illness that likely caused the kidney injury.
Our findings also showed that gastrointestinal manifestations, including abdominal pain and nausea/vomiting, might help differentiate acute and RSTI of acetaminophen exposures. Previous literature characterized nausea, vomiting, anorexia, diaphoresis, malaise, pallor, and lethargy during the first 24 h post-acute ingestion [28, 29]. Many authors reported gastrointestinal manifestations in acetaminophen poisoning cases at presentation or during N-acetylcysteine administration [30,31,32,33,34]. The prevalence of gastrointestinal manifestations in hospitalized patients with acute acetaminophen overdose was 76.6% [35]. A latency time of more than 8 h and an ingestion dose of 10 g can predict a higher prevalence of gastrointestinal manifestations. In addition, patients with gastrointestinal manifestations suffered more liver and kidney damage [35].
Interestingly, we found that the patients with RSTI had more diaphoresis and erythema than acute acetaminophen exposure. It is possible that N-acetylcysteine could be contributory, but more investigation is required to understand this finding.
Our study has limitations. The retrospective design of the current study may have resulted in biases due to confounding factors. Future studies will be required to assess and validate our models’ performance. Other limitations include insufficient documentation during poison center calls or possible transcription errors. Since NPDS provides data that has been fully anonymized, we assumed duplicate data was not included but could not determine precisely.
Also, this study included cases of acetaminophen exposure between January 2012 and December 2017. To enhance predictive accuracy, future studies are needed to include data from recent years, larger data sets, and more sophisticated modeling methods considering different variables such as the acetaminophen formulation taken and co-ingested drugs. The doses of acetaminophen and time to N-acetylcysteine administration are the most important considerations that cannot be determined from NPDS data. There is no way to determine an acetaminophen concentration, no determination of time from ingestion to time to N-acetylcysteine, and most importantly, no way to determine the duration of exposure other than the standard definitions. The NPDS provides the coded abnormalities in the course of hospitalization, not at arrival at the hospital. So, this point limits the generalizability of ML applications at the patient’s arrival and specific course of hospitalization. In addition, the NPDS-coded data are limited and do not provide enough information regarding the time course of poisoning. Future studies considering other data sources that include AST/ALT and acetaminophen concentrations at presentation would help to address the question. Additionally, data regarding RSTI management, including antidotes, would be useful to collect to be able to evaluate the optimal treatment to provide.
Conclusion
A DT model can assist in distinguishing acute and RSTI of acetaminophen exposures. The most important distinguishing factors are age, serum aminotransferase concentrations, abdominal pain, nausea/vomiting, and drowsiness/lethargy. Clinical validations will be necessary before use in clinical settings.
Data Availability
The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.
References
McGill MR, Jaeschke H. Metabolism and disposition of acetaminophen: recent advances in relation to hepatotoxicity and diagnosis. Pharm Res. 2013;30(9):2174–87.
Schilling A, Corey R, Leonard M, et al. Acetaminophen: old drug, new warnings. Cleve Clin J Med. 2010;77(1):19–27.
Lancaster EM, Hiatt JR, Zarrinpar A. Acetaminophen hepatotoxicity: an updated review. Arch Toxicol. 2015;89(2):193–9.
Larson AM, Polson J, Fontana RJ, et al. Acetaminophen-induced acute liver failure: results of a United States multicenter, prospective study. Hepatology. 2005;42(6):1364–72.
Bunchorntavakul C, Reddy KR. Acetaminophen-related hepatotoxicity. Clin Liver Dis. 2013;17(4):587–607.
Dart RC, Erdman AR, Olson KR, et al. Acetaminophen poisoning: an evidence-based consensus guideline for out-of-hospital management. Clin Toxicol. 2006;44(1):1–18.
Bond G, Caravati E, Dart R et al. Guidelines for the Management of Acetaminophen Overdose. available at: https://www.tylenolprofessional.com/sites/tylenol_hcp_us/files/acetaminphen_overdose_treatment_info.pdf. 2015.
Egan H, Isbister GK, Robinson J, et al. Retrospective evaluation of repeated supratherapeutic ingestion (RSTI) of paracetamol. Clin Toxicol. 2019;57(8):703–11.
Chiew AL, Fountain JS, Graudins A, et al. Summary statement: new guidelines for the management of paracetamol poisoning in Australia and New Zealand. Med J Aust. 2015;203(5):215–8.
Daly FF, O’Malley GF, Heard K, et al. Prospective evaluation of repeated supratherapeutic acetaminophen (paracetamol) ingestion. Ann Emerg Med. 2004;44(4):393–8.
Wong A, Gunja N, McNulty R, et al. Analysis of an 8-hour acetylcysteine infusion protocol for repeated supratherapeutic ingestion (RSTI) of paracetamol. Clin Toxicol. 2018;56(3):199–203.
Badger J, LaRose E, Mayer J, et al. Machine learning for phenotyping opioid overdose events. J Biomed Inform. 2019;94:103185.
Sidey-Gibbons JA, Sidey-Gibbons CJ. Machine learning in medicine: a practical introduction. BMC Med Res Methodol. 2019;19(1):1–18.
Deo RC. Machine learning in medicine. Circulation. 2015;132(20):1920–30.
Amirabadizadeh A, Nakhaee S, Mehrpour O. Risk assessment of elevated blood lead concentrations in the adult population using a decision tree approach. Jun. 2020;26:1–8.
Ramezankhani A, Hadavandi E, Pournik O, et al. Decision tree-based modelling for identification of potential interactions between type 2 diabetes risk factors: a decade follow-up in a Middle East prospective cohort study. BMJ open. 2016;6(12):e013336.
Podgorelec V, Kokol P, Stiglic B, et al. Decision trees: an overview and their use in medicine. J Med Syst. 2002;26(5):445–63.
Mehrpour O, Hoyte C, Goss F et al. Decision tree algorithm can determine the outcome of repeated supratherapeutic ingestion (RSTI) exposure to acetaminophen: review of 4500 national poison data system cases. Drug Chem Toxicol. 2022:1–7.
Castanares-Zapatero D, Dinant V, Ruggiano I, et al. Pattern of paracetamol poisoning: influence on outcome and complications. Toxics. 2018;6(4):58.
Tayefi M, Saberi-Karimian M, Esmaeili H, et al. Evaluating of associated risk factors of metabolic syndrome by using decision tree. Comp Clin Pathol. 2018;27(1):215–23.
Craig DG, Bates CM, Davidson JS, et al. Staggered overdose pattern and delay to hospital presentation are associated with adverse outcomes following paracetamol-induced hepatotoxicity. Br J Clin Pharmacol. 2012;73(2):285–94.
Rivera-Penera T, Gugig R, Davis J, et al. Outcome of acetaminophen overdose in pediatric patients and factors contributing to hepatotoxicity. J Pediatr. 1997;130(2):300–4.
Alhelail MA, Hoppe JA, Rhyee SH, et al. Clinical course of repeated supratherapeutic ingestion of acetaminophen. Clin Toxicol. 2011;49(2):108–12.
Greene S, Dargan PI, Jones AL. Acute poisoning: understanding 90% of cases in a nutshell. Postgrad Med J. 2005;81(954):204–16.
Fenves AZ, Kirkpatrick HM, Patel VV, et al. Increased anion gap metabolic acidosis as a result of 5-oxoproline (pyroglutamic acid): a role for acetaminophen. Clin J Am Soc Nephrol. 2006;1(3):441–7.
KhabazianZadeh F, Kazemi T, Nakhaee S, et al. Acetaminophen poisoning-induced heart injury: a case-based review. DARU J Pharm Sci. 2019;27(2):839–51.
Ralapanawa U, Jayawickreme KP, Ekanayake EMM, et al. A study on paracetamol cardiotoxicity. BMC Pharmacol Toxicol. 2016;17(1):1–8.
Saccomano SJ. Acute acetaminophen toxicity in adults. Nursing2020 Crit Care. 2019;14(5):10–7.
Yoon E, Babar A, Choudhary M, et al. Acetaminophen-induced hepatotoxicity: a comprehensive update. J Clin translational Hepatol. 2016;4(2):131.
Tobias JD, Gregory DF, Deshpande JK. Ondansetron to prevent emesis following N-acetylcysteine for acetaminophen intoxication. Pediatr Emerg Care. 1992;8(6):345–6.
Clark RF, Chen R, Williams SR, et al. The use of ondansetron in the treatment of nausea and vomiting associated with acetaminophen poisoning. J Toxicology: Clin Toxicol. 1996;34(2):163–7.
Scharman EJ. Use of ondansetron and other antiemetics in the management of toxic acetaminophen ingestions. J Toxicology: Clin Toxicol. 1998;36(1–2):19–25.
Wright RO, Wright R, Anderson AC, et al. Effect of metoclopramide dose on preventing emesis after oral administration of N-acetylcysteine for acetaminophen overdose. J Toxicology: Clin Toxicol. 1999;37(1):35–42.
Miller MA, Navarro M, Bird SB, et al. Antiemetic use in acetaminophen poisoning: how does the route of N-acetylcysteine administration affect utilization? J Med Toxicol. 2007;3(4):152–6.
Zyoud SeH, Awang R, Sulaiman SAS, et al. Association between gastrointestinal manifestations following acetaminophen poisoning and outcome in 291 acetaminophen poisoning patients. Pharmacoepidemiol Drug Saf. 2010;19(5):511–7.
Acknowledgements
Not applicable.
Funding
There is no funding source.
Author information
Authors and Affiliations
Contributions
CH, SN, BM, FG, and OM contributed to the manuscript’s conception, design, and preparation. CH, OM conducted the data collection and contributed to acquisition and interpretation. In addition, CH, SN, BM, FG, and OM contributed substantially to drafting the manuscript and revising it critically for important intellectual content. All authors have read and approved the final version of the manuscript.
Corresponding author
Ethics declarations
Ethical approval and consent to participate
All methods were carried out in accordance with relevant guidelines and regulations. We obtained de-identified data from NPDS. This study was reviewed by Colorado Multiple Institutional Review Board on Human Subjects Protection and determined to be exempt (COMIRB#: 22-1088).
Consent for publication
Not Applicable.
Disclaimers
AAPC supports the NPDS, which includes the data of all self-reporting patients collected by callers during poison information calls and exposure management by the country’s PCCs. However, NPDS data do not reflect the total amount of exposure to a particular substance. Additional exposure to PCC can therefore be under-reported. Accordingly, NPDS data should not confirm poisoning or overdoses, and the AAPCC does not fully verify the accuracy of each report. Therefore, the results based on NPDS data do not necessarily reflect the AAPCC’s opinion.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Mehrpour, O., Hoyte, C., Nakhaee, S. et al. Using a decision tree algorithm to distinguish between repeated supra-therapeutic and acute acetaminophen exposures. BMC Med Inform Decis Mak 23, 102 (2023). https://doi.org/10.1186/s12911-023-02188-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12911-023-02188-2