Skip to main content

Using a decision tree algorithm to distinguish between repeated supra-therapeutic and acute acetaminophen exposures

Abstract

Background

This study aimed to compare clinical and laboratory characteristics of supra-therapeutic (RSTI) and acute acetaminophen exposures using a predictive decision tree (DT) algorithm.

Methods

We conducted a retrospective cohort study using the National Poison Data System (NPDS). All patients with RSTI acetaminophen exposure (n = 4,522) between January 2012 and December 2017 were included. Additionally, 4,522 randomly selected acute acetaminophen ingestion cases were included. After that, the DT machine learning algorithm was applied to differentiate acute acetaminophen exposure from supratherapeutic exposures.

Results

The DT model had accuracy, precision, recall, and F1-scores of 0.75, respectively. Age was the most relevant variable in predicting the type of acetaminophen exposure, whether RSTI or acute. Serum aminotransferase concentrations, abdominal pain, drowsiness/lethargy, and nausea/vomiting were the other most important factors distinguishing between RST and acute acetaminophen exposure.

Conclusion

DT models can potentially aid in distinguishing between acute and RSTI of acetaminophen. Further validation is needed to assess the clinical utility of this model.

Peer Review reports

Introduction

Acetaminophen is a commonly used antipyretic and analgesic drug available over the counter [1]. In adults, the safe dose of acetaminophen is between 325 and 650 mg every 4–6 h, with a maximum of 4 g per day [2]. Overdose of acetaminophen can be severe, leading to liver failure or even death [3]. Acetaminophen poisoning is one of the most common causes of acute liver failure in the US and worldwide [4]. Although most patients with acetaminophen poisoning have mild-to-moderate consequences, 29% of patients with acetaminophen-induced liver failure require a liver transplant and have a mortality rate of 28% [5].

Most commonly, acetaminophen overdose occurs after ingesting acetaminophen, within 8 h or less, in amounts that can cause toxicity (e.g., > 4 g/24 hours). Although hepatotoxicity may occur from acute or repeated acetaminophen ingestion, mortality from acetaminophen poisoning is uncommon​​ (death occurred in 0.3% of cases treated with N-acetylcysteine). Chronic exposure is termed repeated supra-therapeutic ingestion (RSTI) to separate it from regular therapeutic use. In adults, RSTI of acetaminophen occurs when the intake of acetaminophen exceeds 8 h, resulting in a cumulative dose of more than 200 mg/kg/day (or 10 g/day, whichever is less) within 24 h or more than 150 mg/kg/day (or 6 g/day, whichever is less) within 48 h [6, 7]. In addition, for children under six years of age, RSTI is the repeated consumption of acetaminophen of more than 100 mg/kg/day for 72 h [7].

Contrasting with acute exposure cases with known ingestion time, in RSTI, the acetaminophen nomogram-based approach is not applicable. There are few studies and guidelines vary in what constitutes RSTI and when to start N-acetylcysteine. Typically, N-acetylcysteine is recommended in patients with alanine aminotransferase (ALT) greater than 50 IU/L or serum acetaminophen concentration greater than 20 mg/L (132 µmol /L) [8, 9]. Daly et al. studied 249 RSTI of acetaminophen and found that patients with AST < 50 U/L and acetaminophen concentrations less than 10 mg/L had a lower risk of hepatotoxicity [10, 11]. As management of RSTI and acute acetaminophen ingestion may differ, it is crucial to differentiate between these two overdose presentations.

Machine learning (ML) is a new approach that has recently been used in medicine for disease diagnosis, treatment decisions and prognosis [12]. ML uses algorithms and mathematical methods to identify patterns and relationships between variables and to contribute to the predictability of the target variable [13]. Supervised and unsupervised learning are two main ML approaches to classifying patients and creating risk detection models [14]. DT models are one of the most commonly used ML algorithms to classify medical data to inform appropriate medical decision making [15,16,17,18]. This study uses a DT model to help distinguish between RSTI and acute acetaminophen ingestion using clinical and laboratory characteristics from the National Poison Data System (NPDS).

Methods

Study design

This was a retrospective cohort study using NPDS, a de-identified national repository for poison control center data used by the American Association of Toxic Control Centers (AAPCC). The AAPCC maintains case records of self-reported information collected by callers during exposure management and poison information calls managed by the 55 poison control centers across the United States. Poison control centers submit de-identified case data to the NPDS after providing poison exposure management. As soon as information regarding this file is available, it is uploaded to the NDPS and recorded by poison center staff.

The NPDS includes clinical effects and interactions with the agent, duration of effect, end-organ effects, chronicity, demographic data (age, sex, and weight), administration sites, clinical findings, exposure information, and categorical laboratory findings. Each of these was included in the model. In addition, we defined “related to the exposure” to be appropriate when: (1) the timing of clinical effect is consistent with the exposure recorded (clinical effects recorded matches the half-life of toxicant in the body); (2) the severity of the clinical effect is consistent with the reported exposure; (3) the clinical effect is consistent with the expected toxicity of the substance; and (4) a physician assessed the relationship’s clinical significance.

Last, correlational analyses do not necessarily indicate causality. Clinical effects are not related to exposure if they existed before the exposure and did not increase or worsen as a result of the exposure or if the effect can be attributed to an alternative cause. All methods were carried out in accordance with relevant guidelines and regulations.

While most cases are “closed” within a few hours of the initial contact, some exposures are followed to obtain the patient’s medical outcome and may remain open for months. Follow-up calls provide a proven mechanism for monitoring the appropriateness of management recommendations, enabling continual updates of case information, augmenting patient guidelines, providing poison prevention education, and obtaining final medical outcomes to make the data collected as accurate and complete as possible.

Study population

All patients with RSTI of acetaminophen (n = 4,522) were included between January 2012 and December 2017. In addition, 4,522 randomly selected acute acetaminophen ingestion cases were included. Criteria for exclusion were missing demographic data and clinical findings. We assumed that duplicated data was not included.

We obtained de-identified data from NPDS. Based on the Colorado Multiple Institutional Review Board (COMIRB) on Human Subjects Protection guidelines and procedures, the analysis of NPDS data for this research did not meet the criteria for human subjects research according to the 45 Code of Federal Regulations (CFR) 46.101(b). Therefore, this study was determined to be exempt (COMIRB#: 22-1088).

ML-based approach

We used a DT model using Statistical Package for the Social Sciences (SPSS) 26 and Python (version 3.9) for distinguishing between RSTI and acute acetaminophen ingestion using clinical and laboratory characteristics extracted from the NPDS. The DT model uses split criteria to divide an end node and set a tree with a rate for each predictor variable [19]. It divides the data into binary parts and builds a binary tree based on them so that two edges come out of each inner node, and the resulting trees are pruned. This algorithm is used to create regression and classification trees. A DT model evaluates the variable that best divides the data [20] using the Gini index criterion to decide how to select tree nodes. The Gini Cost Function shows how pure the nodes are, wherein the purity of the node refers to the degree to which the training data assigned to each node is combined. The division continues until the node has the minimum number of training samples or exceeds the maximum depth of the tree. The root node is the most critical variable that starts the decision tree graph. It is the variable that best divides the data in a DT approach. Intermediate nodes are those in which variables are estimated but not the ultimate nodes in which predictions are formed.

Similar to other ML methods, standard performance measures were used to measure the performance of the DT model, including precision, recall, accuracy, F1 score, and confusion matrices. A confusion matrix was generated with all the required information to measure specificity, accuracy, and sensitivity [13].

Statistical analysis

Analysis was performed using Python 3.9 and SPSS 26. Chi-square and student t-tests were used to compare data between groups. A P-value of < 0.05 was determined to be statistically significant.

Results

In total, 4,522 patients with RSTI of acetaminophen were included. The baseline and clinical characteristics of patients were compared between two groups, and their results are presented in Table 1. The DT model produced in this study was 29 nodes in size, with 17 leaves and four levels (Fig. 1).

Table 1 Characteristics of patients with acute exposure vs. supra-therapeutic ingestions (RSTI)
Fig. 1
figure 1

DT diagram for predicting repeated supra-therapeutic ingestion (RSTI) versus acute acetaminophen exposure. Values shown are in percentages

Age, followed by serum aminotransferase concentrations, abdominal pain, and drowsiness/lethargy, were the most important factors in distinguishing between RSTI and acute acetaminophen exposures (Fig. 1). The rules derived from the DT are shown in Table 2. In Fig. 2, we evaluated important signs and symptoms that affect the DT model performance (i.e., feature importance) in the classification task. Serum aminotransferase concentrations, abdominal pain, nausea/vomiting, and drowsiness/lethargy contributed most to classifying acute and RSTI, respectively.

Table 2 The 17 rules extracted from the DT model to distinguish between acute and repeated supra-therapeutic (RST) acetaminophen exposures
Fig. 2
figure 2

Feature importance based on the DT model

The DT model had an accuracy, precision, recall, and F1-score of 0.75 each in the cross-validation method (Table 3). Confusion matrices are shown in Table 4. The model could correctly predict 3,361 cases of acute acetaminophen poisoning and 3,381 RSTIs. Figure 3 shows the ROC curve for the DT model (AUC = 0.81).

Table 3 Characteristics of the DT model in the acute and RSTI acetaminophen poisoning
Table 4 Confusion matrix of DT in acute and RSTI acetaminophen poisoning
Fig. 3
figure 3

Receiver Operating Curve (ROC) for the DT model

Discussion

We assessed the efficacy and accuracy of a DT model to distinguish acute and RSTI of acetaminophen using retrospective large-scale NPDS dataset analysis. Our DT model had acceptable characteristics and could correctly predict 3,361 cases of acute acetaminophen poisoning and 3,381 RST exposures. These findings show promise in using ML to help differentiate acute ingestions from RSTIs.

In our DT model, the most relevant predictive variable was age. This result is unsurprising given that elderly patients are more likely to use acetaminophen as an analgesic due to various ailments, raising the possibility of poisoning due to RSTI. Our study found that most RSTI exposures tend to produce aminotransferase concentrations between 100 and 1,000 IU/L or greater than 1,000 IU/L. Prior studies have found that all cases with RSTI who developed hepatotoxicity were found to have abnormal ALT (more than 50 IU/L) [8] and more likely to have encephalopathy requiring renal replacement therapy, mechanical ventilation, or death [21]. The risk of liver failure has also been seen in patients with RSTI compared to acute overdose cases [22]. This may be partly due to the delayed treatment of RSTI patients as compared to acute acetaminophen exposure cases, which typically undergo treatment within eight hours of exposure [23, 24].

Our data also showed that acidosis is another important variable in RSTI exposures. Anion gap metabolic acidosis can occur in patients with acetaminophen overdose-induced hepatotoxicity and occasionally precedes hepatic injury [25]. In addition, studies showed that electrolyte abnormalities due to an acetaminophen overdose correlate with metabolic acidosis [26, 27].

We also found that patients with RSTI of acetaminophen had increased creatinine concentrations and prolonged PT/INR compared to the acute group. Other studies have shown that patients with RSTI have increased creatinine concentrations, and serum INR had a significant association with hepatotoxicity. All patients who died or had a liver transplant showed increased creatinine concentrations [23]. It is possible that people with RSTI had an acute illness (e.g., hepatitis, dehydration) causing the acute kidney injury and minor elevation in transaminases. The dataset does not include information about the prior or concomitant illness that likely caused the kidney injury.

Our findings also showed that gastrointestinal manifestations, including abdominal pain and nausea/vomiting, might help differentiate acute and RSTI of acetaminophen exposures. Previous literature characterized nausea, vomiting, anorexia, diaphoresis, malaise, pallor, and lethargy during the first 24 h post-acute ingestion [28, 29]. Many authors reported gastrointestinal manifestations in acetaminophen poisoning cases at presentation or during N-acetylcysteine administration [30,31,32,33,34]. The prevalence of gastrointestinal manifestations in hospitalized patients with acute acetaminophen overdose was 76.6% [35]. A latency time of more than 8 h and an ingestion dose of 10 g can predict a higher prevalence of gastrointestinal manifestations. In addition, patients with gastrointestinal manifestations suffered more liver and kidney damage [35].

Interestingly, we found that the patients with RSTI had more diaphoresis and erythema than acute acetaminophen exposure. It is possible that N-acetylcysteine could be contributory, but more investigation is required to understand this finding.

Our study has limitations. The retrospective design of the current study may have resulted in biases due to confounding factors. Future studies will be required to assess and validate our models’ performance. Other limitations include insufficient documentation during poison center calls or possible transcription errors. Since NPDS provides data that has been fully anonymized, we assumed duplicate data was not included but could not determine precisely.

Also, this study included cases of acetaminophen exposure between January 2012 and December 2017. To enhance predictive accuracy, future studies are needed to include data from recent years, larger data sets, and more sophisticated modeling methods considering different variables such as the acetaminophen formulation taken and co-ingested drugs. The doses of acetaminophen and time to N-acetylcysteine administration are the most important considerations that cannot be determined from NPDS data. There is no way to determine an acetaminophen concentration, no determination of time from ingestion to time to N-acetylcysteine, and most importantly, no way to determine the duration of exposure other than the standard definitions. The NPDS provides the coded abnormalities in the course of hospitalization, not at arrival at the hospital. So, this point limits the generalizability of ML applications at the patient’s arrival and specific course of hospitalization. In addition, the NPDS-coded data are limited and do not provide enough information regarding the time course of poisoning. Future studies considering other data sources that include AST/ALT and acetaminophen concentrations at presentation would help to address the question. Additionally, data regarding RSTI management, including antidotes, would be useful to collect to be able to evaluate the optimal treatment to provide.

Conclusion

A DT model can assist in distinguishing acute and RSTI of acetaminophen exposures. The most important distinguishing factors are age, serum aminotransferase concentrations, abdominal pain, nausea/vomiting, and drowsiness/lethargy. Clinical validations will be necessary before use in clinical settings.

Data Availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. McGill MR, Jaeschke H. Metabolism and disposition of acetaminophen: recent advances in relation to hepatotoxicity and diagnosis. Pharm Res. 2013;30(9):2174–87.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Schilling A, Corey R, Leonard M, et al. Acetaminophen: old drug, new warnings. Cleve Clin J Med. 2010;77(1):19–27.

    Article  PubMed  Google Scholar 

  3. Lancaster EM, Hiatt JR, Zarrinpar A. Acetaminophen hepatotoxicity: an updated review. Arch Toxicol. 2015;89(2):193–9.

    Article  CAS  PubMed  Google Scholar 

  4. Larson AM, Polson J, Fontana RJ, et al. Acetaminophen-induced acute liver failure: results of a United States multicenter, prospective study. Hepatology. 2005;42(6):1364–72.

    Article  CAS  PubMed  Google Scholar 

  5. Bunchorntavakul C, Reddy KR. Acetaminophen-related hepatotoxicity. Clin Liver Dis. 2013;17(4):587–607.

    Article  PubMed  Google Scholar 

  6. Dart RC, Erdman AR, Olson KR, et al. Acetaminophen poisoning: an evidence-based consensus guideline for out-of-hospital management. Clin Toxicol. 2006;44(1):1–18.

    Article  CAS  Google Scholar 

  7. Bond G, Caravati E, Dart R et al. Guidelines for the Management of Acetaminophen Overdose. available at: https://www.tylenolprofessional.com/sites/tylenol_hcp_us/files/acetaminphen_overdose_treatment_info.pdf. 2015.

  8. Egan H, Isbister GK, Robinson J, et al. Retrospective evaluation of repeated supratherapeutic ingestion (RSTI) of paracetamol. Clin Toxicol. 2019;57(8):703–11.

    Article  CAS  Google Scholar 

  9. Chiew AL, Fountain JS, Graudins A, et al. Summary statement: new guidelines for the management of paracetamol poisoning in Australia and New Zealand. Med J Aust. 2015;203(5):215–8.

    Article  PubMed  Google Scholar 

  10. Daly FF, O’Malley GF, Heard K, et al. Prospective evaluation of repeated supratherapeutic acetaminophen (paracetamol) ingestion. Ann Emerg Med. 2004;44(4):393–8.

    Article  PubMed  Google Scholar 

  11. Wong A, Gunja N, McNulty R, et al. Analysis of an 8-hour acetylcysteine infusion protocol for repeated supratherapeutic ingestion (RSTI) of paracetamol. Clin Toxicol. 2018;56(3):199–203.

    Article  CAS  Google Scholar 

  12. Badger J, LaRose E, Mayer J, et al. Machine learning for phenotyping opioid overdose events. J Biomed Inform. 2019;94:103185.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Sidey-Gibbons JA, Sidey-Gibbons CJ. Machine learning in medicine: a practical introduction. BMC Med Res Methodol. 2019;19(1):1–18.

    Article  Google Scholar 

  14. Deo RC. Machine learning in medicine. Circulation. 2015;132(20):1920–30.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Amirabadizadeh A, Nakhaee S, Mehrpour O. Risk assessment of elevated blood lead concentrations in the adult population using a decision tree approach. Jun. 2020;26:1–8.

    Google Scholar 

  16. Ramezankhani A, Hadavandi E, Pournik O, et al. Decision tree-based modelling for identification of potential interactions between type 2 diabetes risk factors: a decade follow-up in a Middle East prospective cohort study. BMJ open. 2016;6(12):e013336.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Podgorelec V, Kokol P, Stiglic B, et al. Decision trees: an overview and their use in medicine. J Med Syst. 2002;26(5):445–63.

    Article  PubMed  Google Scholar 

  18. Mehrpour O, Hoyte C, Goss F et al. Decision tree algorithm can determine the outcome of repeated supratherapeutic ingestion (RSTI) exposure to acetaminophen: review of 4500 national poison data system cases. Drug Chem Toxicol. 2022:1–7.

  19. Castanares-Zapatero D, Dinant V, Ruggiano I, et al. Pattern of paracetamol poisoning: influence on outcome and complications. Toxics. 2018;6(4):58.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Tayefi M, Saberi-Karimian M, Esmaeili H, et al. Evaluating of associated risk factors of metabolic syndrome by using decision tree. Comp Clin Pathol. 2018;27(1):215–23.

    Article  Google Scholar 

  21. Craig DG, Bates CM, Davidson JS, et al. Staggered overdose pattern and delay to hospital presentation are associated with adverse outcomes following paracetamol-induced hepatotoxicity. Br J Clin Pharmacol. 2012;73(2):285–94.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Rivera-Penera T, Gugig R, Davis J, et al. Outcome of acetaminophen overdose in pediatric patients and factors contributing to hepatotoxicity. J Pediatr. 1997;130(2):300–4.

    Article  CAS  PubMed  Google Scholar 

  23. Alhelail MA, Hoppe JA, Rhyee SH, et al. Clinical course of repeated supratherapeutic ingestion of acetaminophen. Clin Toxicol. 2011;49(2):108–12.

    Article  CAS  Google Scholar 

  24. Greene S, Dargan PI, Jones AL. Acute poisoning: understanding 90% of cases in a nutshell. Postgrad Med J. 2005;81(954):204–16.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Fenves AZ, Kirkpatrick HM, Patel VV, et al. Increased anion gap metabolic acidosis as a result of 5-oxoproline (pyroglutamic acid): a role for acetaminophen. Clin J Am Soc Nephrol. 2006;1(3):441–7.

    Article  CAS  PubMed  Google Scholar 

  26. KhabazianZadeh F, Kazemi T, Nakhaee S, et al. Acetaminophen poisoning-induced heart injury: a case-based review. DARU J Pharm Sci. 2019;27(2):839–51.

    Article  CAS  Google Scholar 

  27. Ralapanawa U, Jayawickreme KP, Ekanayake EMM, et al. A study on paracetamol cardiotoxicity. BMC Pharmacol Toxicol. 2016;17(1):1–8.

    Article  Google Scholar 

  28. Saccomano SJ. Acute acetaminophen toxicity in adults. Nursing2020 Crit Care. 2019;14(5):10–7.

    Article  Google Scholar 

  29. Yoon E, Babar A, Choudhary M, et al. Acetaminophen-induced hepatotoxicity: a comprehensive update. J Clin translational Hepatol. 2016;4(2):131.

    Google Scholar 

  30. Tobias JD, Gregory DF, Deshpande JK. Ondansetron to prevent emesis following N-acetylcysteine for acetaminophen intoxication. Pediatr Emerg Care. 1992;8(6):345–6.

    Article  CAS  PubMed  Google Scholar 

  31. Clark RF, Chen R, Williams SR, et al. The use of ondansetron in the treatment of nausea and vomiting associated with acetaminophen poisoning. J Toxicology: Clin Toxicol. 1996;34(2):163–7.

    CAS  Google Scholar 

  32. Scharman EJ. Use of ondansetron and other antiemetics in the management of toxic acetaminophen ingestions. J Toxicology: Clin Toxicol. 1998;36(1–2):19–25.

    CAS  Google Scholar 

  33. Wright RO, Wright R, Anderson AC, et al. Effect of metoclopramide dose on preventing emesis after oral administration of N-acetylcysteine for acetaminophen overdose. J Toxicology: Clin Toxicol. 1999;37(1):35–42.

    CAS  Google Scholar 

  34. Miller MA, Navarro M, Bird SB, et al. Antiemetic use in acetaminophen poisoning: how does the route of N-acetylcysteine administration affect utilization? J Med Toxicol. 2007;3(4):152–6.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Zyoud SeH, Awang R, Sulaiman SAS, et al. Association between gastrointestinal manifestations following acetaminophen poisoning and outcome in 291 acetaminophen poisoning patients. Pharmacoepidemiol Drug Saf. 2010;19(5):511–7.

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

There is no funding source.

Author information

Authors and Affiliations

Authors

Contributions

CH, SN, BM, FG, and OM contributed to the manuscript’s conception, design, and preparation. CH, OM conducted the data collection and contributed to acquisition and interpretation. In addition, CH, SN, BM, FG, and OM contributed substantially to drafting the manuscript and revising it critically for important intellectual content. All authors have read and approved the final version of the manuscript.

Corresponding author

Correspondence to Omid Mehrpour.

Ethics declarations

Ethical approval and consent to participate

All methods were carried out in accordance with relevant guidelines and regulations. We obtained de-identified data from NPDS. This study was reviewed by Colorado Multiple Institutional Review Board on Human Subjects Protection and determined to be exempt (COMIRB#: 22-1088).

Consent for publication

Not Applicable.

Disclaimers

AAPC supports the NPDS, which includes the data of all self-reporting patients collected by callers during poison information calls and exposure management by the country’s PCCs. However, NPDS data do not reflect the total amount of exposure to a particular substance. Additional exposure to PCC can therefore be under-reported. Accordingly, NPDS data should not confirm poisoning or overdoses, and the AAPCC does not fully verify the accuracy of each report. Therefore, the results based on NPDS data do not necessarily reflect the AAPCC’s opinion.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mehrpour, O., Hoyte, C., Nakhaee, S. et al. Using a decision tree algorithm to distinguish between repeated supra-therapeutic and acute acetaminophen exposures. BMC Med Inform Decis Mak 23, 102 (2023). https://doi.org/10.1186/s12911-023-02188-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-023-02188-2

Keywords