Skip to main content
Download PDF

Prediction of inappropriate pre-hospital transfer of patients with suspected cardiovascular emergency diseases using machine learning: a retrospective observational study

BMC Medical Informatics and Decision Making volume 23, Article number: 56 (2023) Cite this article

Abstract

Background

This study aimed to develop a prediction model for transferring patients to an inappropriate hospital for suspected cardiovascular emergency diseases at the pre-hospital stage, using variables obtained from an integrated nationwide dataset, and to assess the performance of this model.

Methods

We integrated three nationwide datasets and developed a two-step prediction model utilizing a machine learning algorithm. Ninety-eight clinical characteristics of patients identified at the pre-hospital stage and 13 hospital components were used as input data for the model. The primary endpoint of the model was the prediction of transfer to an inappropriate hospital.

Results

A total of 94,256 transferred patients in the public pre-hospital care system matched the National Emergency Department Information System data of patients with a pre-hospital cardiovascular registry created in South Korea between July 2017 and December 2018. Of these, 1,770 (6.26%) patients failed to be transferred to a capable hospital. The area under the receiver operating characteristic curve of the final predictive model was 0.813 (0.800–0.825), and the area under the receiver precision-recall curve was 0.286 (0.265–0.308).

Conclusions

Our prediction model used machine learning to show favorable performance in transferring patients with suspected cardiovascular disease to a capable hospital. For our results to lead to changes in the pre-hospital care system, a digital platform for sharing real-time information should be developed.

Peer Review reports

Background

Cardiovascular symptoms are one of the most common reasons for patients visiting the emergency department (ED) [1]. Cardiovascular emergency conditions, such as acute myocardial infarction (AMI), are life-threatening conditions that must be recognized immediately to avoid treatment delays. Missing true cardiovascular emergencies can result in mortality and morbidity cases, which burdens healthcare services [1, 2]. Studies have reported that early recognition of cardiovascular emergency conditions at the pre-hospital stage and its rapid treatment in the hospital are crucial to improving survival rates [3,4,5,6,7,8]. Therefore, evaluating the possibility of cardiovascular emergency conditions and selecting a suitable hospital for treatment at the pre-hospital stage are critical. In this context, several studies have been conducted to predict cardiovascular emergency conditions with patient information acquired at the pre-hospital stage. However, none have focused on predicting a capable hospital for treating patients with cardiovascular emergencies [9,10,11,12,13]. At the pre-hospital stage, providers should select a hospital for patient transfer by sharing hospital information about patient treatment capacity while predicting cardiovascular emergencies with information obtained from patients. However, training all pre-hospital care providers to make accurate assessments is challenging. Thus, developing and validating data-based practical assessment tools at the pre-hospital stage is necessary. While efforts to predict critical events using machine learning in the medical field exist, few studies have used machine learning-based practical tools to help pre-hospital care providers decide which hospital to transfer a patient to [14,15,16]. Clinical tools using machine learning may prove useful at the pre-hospital stage, where advanced human resources are insufficient, rather than in hospitals, where the expertise of physicians is available.

This study was conducted to develop a prediction model using machine learning to assess the inappropriate hospital transfer of patients suspected of having cardiovascular emergency conditions at the pre-hospital stage using variables obtained from an integrated nationwide dataset.

Methods

Study design and setting

The present study was a retrospective observational study using three nationwide datasets from the National Fire Agency and National Emergency Medical Center in South Korea. The study protocol was approved, and informed consent was waived by the Institutional Review Board of Severance Hospital, South Korea (approval number 4-2020-0110). The present study adhered to the ethical standards of the Declaration of Helsinki.

In South Korea, the National Fire Agency, which consists of 18 provincial fire departments, oversees the public pre-hospital care system. When the pre-hospital care providers transport an emergency patient from the scene to an ED, they must complete a transfer record in compliance with field first aid standard protocol according to the National Fire Agency. This protocol covers standardized first aid procedures at the scene and guidelines for selecting a transfer hospital. This protocol is updated annually under the supervision of medical advisors. According to the Rescue and Fire Emergency Medical Service Act, all providers at the National Fire Agency are required to receive 40 h of mandatory annual training in medical skills and knowledge [17].

In South Korea, three levels of EDs are designated by the Ministry of Health and Welfare based on its human resources, emergency equipment, and availability of medical service specialists. By law, level-1 and level-2 EDs must be staffed 24 h a day with board-certified emergency physicians [18]. EDs rated at these two levels are evaluated annually by the Ministry of Health and Welfare in accordance with the Emergency Medical Service Act, to confirm whether they can provide high-level emergency medical care. The designation of levels 1 and 2 can change according to this result. In 2017, the numbers of sites for levels 1 and 2 were 36 and 119, respectively; in 2018, they were 36 and 118, respectively [19].

Selection of participants

Patients aged > 15 years who had been transferred to EDs by the public pre-hospital care system between July 2017 and December 2018 were enrolled. Of these, patients whose Pre-Hospital Cardiovascular Registry (PHCR) had been activated were selected for inclusion, and those who could not match hospital stage information were excluded.

Data collection and processing

The datasets were the Pre-Hospital Run Sheets (PHRS) and PHCR, both managed by the National Fire Agency; and the National Emergency Department Information System (NEDIS), operated by the National Emergency Medical Center in South Korea. The collection, processing, and integration methods of the three datasets are described in detail in Additional File 1.

Model development

The prediction model for transferring patients to an inappropriate hospital was developed by integrating the three datasets (PHRS, PHCR, and NEDIS). The dataset was divided into train and test sets in a 7:3 ratio, using an iterative stratification method, to ensure a similar positive ratio in every class label. The train dataset was then split into 10 folds. This model comprised two structures: the patient class prediction and the matching of hospital factors with this classification (Fig. 1). First, in the patient class prediction step, we trained a three-layer multilayer perceptron model with dropout and batch normalization using 98 variables that reflect the patient characteristics at the pre-hospital stage. Since a single patient can be categorized into multiple classes (multi-label classification problem), the loss function was constructed using each class’s binary cross-entropy loss. Accordingly, the model was designed to capture the correlation of patient class labels and simultaneously predict these correctly. Additional File 1 shows the 13 patient class labels according to the examination and treatment codes patients received in the ED. Second, for the final model to predict a patient’s transfer to an inappropriate hospital, we trained the XGBoost model. This model was developed by connecting the patient class prediction results with the hospital classification data. The model was trained to identify hospital factors that were highly associated with patients in each patient class not being transferred to an appropriate hospital. The corresponding hospital factors comprised five classes: management quality, resource availability, ED crowding, hospital occupancy, and distance between the scene and the hospital [see Additional File 1]. One of the hospital classes, resource availability, was defined as when a real-time signal of emergency resources was sent to the NEDIS from the transferred hospital. Relative crowding was defined as the number of patients in the ED compared to the average number of patients at a given time in the ED. Similar to the patient class prediction method, the final model was developed and selected by 10-fold cross-validation.

Fig. 1
figure 1

Flow chart of model development

Full size image

For model interpretation, we adopted the SHapley Additive exPlanation (SHAP) [20]. SHAP can explain any machine learning model’s output by calculating each feature’s impact on model prediction based on game theory. Through this process, we can understand: (1) the feature that is most important for model prediction, (2) the positive or negative direction of feature impact, and (3) the relationship between importance score (SHAP value) and feature value. Applying SHAP to our developed models, we used the Deep Explainer module, which enables the fast approximation of SHAP values in the deep learning model, and the Tree Explainer module, which optimizes the SHAP algorithm for tree ensemble methods such as XGBoost [20].

Outcome measurement

The primary outcome was the transfer of patients to an inappropriate hospital. In the NEDIS, the disposition of patients visiting the ED was classified into four categories: admission, discharge, transfer to another hospital, and death. Transfer to an inappropriate hospital was defined as death or transfer to another hospital at the initial ED.

Statistical analysis

All statistical analyses, model fitting, and validation were conducted using the R statistical package (www.R-project.org). The significance criterion was set as two-sided, and P values < 0.05 were considered statistically significant. Model performance was evaluated by calculating the sensitivity, specificity, accuracy, positive predictive value, negative predictive value, the area under the receiver operating characteristic curve (AUROC), and the area under the receiver precision-recall curve (AUPRC). The AUROC encompasses the area under the curve drawn with 1-specificity as the X-axis and sensitivity as the Y-axis. The AUPRC is the area under the curve summarized as the trade-off between the true positive rate and the positive predictive value for a predictive model using different probability thresholds. The AUROC and the AUPRC scores were calculated in the validation set in every epoch. We chose the model showing the highest AUROC and AUPRC scores in the validation fold. The cut-off threshold for calculating the avoidable failure of transfer to the optimal hospital was selected as the point that maximizes Youden’s J-score in the final model.

Results

Baseline characteristics of study participants

During the study period, 94,267 public pre-hospital care system-transferred patients were matched with NEDIS data of patients created with the PHCR. Our final study participants were 94,256 patients after excluding missing data on the transport time and transfer distance to the hospital. These patients were categorized into train and test sets. Of the enrolled patients in the train set, 4,039 patients (6.1%) were transferred to an inappropriate hospital. Of the 28,277 patients enrolled in the test set, 1,770 (6.3%) failed to be transferred to an appropriate hospital (Fig. 2). The baseline characteristics of the study participants are summarized in Additional File 1.

Fig. 2
figure 2

Flow chart of patient selection. (ED, emergency department)

Full size image

Model prediction

The proportions of patient classes in the test set are presented in Table 1. Table 2 shows the performance of our model that predicts 13 subclasses and transfers to an inappropriate hospital using information from the pre-hospital stage of all patients. The prediction performance rates of patient classes that had undergone cardiopulmonary resuscitation for in-hospital arrest (Class 1) or intubation (Class 2), central catheterization (Class 3), and percutaneous coronary intervention after transport (Class 5) were high: 0.865, 0.882, 0.847, and 0.929, respectively. The AUROC of the final model for predicting transfer to an inappropriate hospital was 0.813, whereas the AUPRC was 0.286.

Table 1 The proportions of patient classes in the test set
Full size table
Table 2 The model’s performance for patient classes and transfer to an inappropriate hospital
Full size table

Pre-hospital stage variables with high SHAP values to predict each patient class are presented in Additional File 2. Dyspnea was included in the top ten features that predicted all classes for which urgent intervention was required, except for patients for whom coronary intervention was predicted (Class 5). Patients with less than 94% peripheral oxygen saturation at the pre-hospital stage were predicted to receive more intubation, central line catheterization, and cardiopulmonary resuscitation in the ED. TIMI scores, excepting cardiac enzyme and three-lead electrocardiogram monitoring, were high-impact features in predicting the class expected to undergo percutaneous coronary intervention (PCI) in the model. Figure 3 shows the top features in the prediction model for transfer to an inappropriate hospital. In the case of patients who were expected to receive cardiopulmonary resuscitation (Class 1), intubation (Class 2), and central line catheterization (Class 3), the probability of transfer to an inappropriate hospital was high. For patients predicted to be in classes 2, 3, and 5 according to the patient class prediction, we confirmed that if their annual hospital admission rate to their transferred hospital was high, then failure to transfer to an appropriate hospital would decrease. The model’s calibration and fairness analysis results are presented in Additional File 3.

Fig. 3
figure 3

Top features in the final prediction model for transfer to inappropriate hospital. (ED, emergency department; SHAP, SHapley Additive exPlanation)

Full size image

Discussion

Since a cardiovascular emergency condition is a life-threatening disease that causes a poor prognosis if not treated in a timely manner, a patient suspected of this condition at the pre-hospital stage should be transferred to a hospital where prompt, definite care is available [21,22,23]. Since it is impossible to continuously build qualified hospitals in all communities [24, 25], developing a wide-ranging system covering emergency treatment capabilities in the community is crucial [26]. Accurate distribution of patients to suitable hospitals at the pre-hospital stage is vital. When patients with cardiovascular emergency conditions are transferred to a hospital with insufficient capacity, the outcome can be fatal. Conversely, emergency resources are needlessly saturated when patients without life-threatening conditions are transferred to a higher-level hospital. Saturation of emergency resources can obstruct the immediate resuscitation of critical patients and increase the possibility of transfer to other hospitals for definite care [27,28,29]. Most studies that have performed cardiovascular emergency prediction based on pre-hospital settings focus on diagnosing an urgent condition or fatal disease using the patient’s features [9,10,11,12,13, 30]. However, the ultimate role of the pre-hospital care system is not only the prediction of a patient’s critical condition but also a timely transfer to an optimal hospital that can resolve this condition. Therefore, our model is designed to be a practical tool for pre-hospital care providers’ use in the field. Individual hospital factors should be considered in addition to patients’ features, to determine the optimal hospital at the pre-hospital stage. Our study presented a two-step model that can predict hospitals suitable for the patient class by considering hospital factors: the real-time crowding status of hospitals and EDs; distance from the scene; and treatment capacity, including PCI, of the hospital. Hospital capability can change depending on its real-time capacity or quality, even for patients with the same features; the present modeling results developed by incorporating these variable factors showed a favorable performance.

Our study results confirmed that patients in classes for which intubation or central line catheterization procedures were predicted found it more difficult to be transferred to an appropriate hospital. Interestingly, the present prediction model showed that information on the annual admission rate for the preceding year of these patient classes in the hospital to which they were transferred increased their probability of being transferred to a capable hospital. In addition, patients predicted to undergo percutaneous transluminal coronary angioplasty had a higher probability of being transferred to an appropriate hospital when annual admission rate information was combined. This is presumably because the lower ED could not supply enough medical resources for resuscitation; thus, the patient died or was transferred to another hospital for final treatment. These findings can be interpreted such that, among several hospital factors, information related to management quality for patients needing urgent intervention was the most beneficial for selecting an appropriate hospital. As the patient class prediction step in our model was designed to predict treatment to be received at the ED, rather than predicting the patient’s diagnosis, the management quality of the hospital for this treatment could be used to determine the hospital for transfer.

Our experimental modeling was developed as a practical tool for pre-hospital care providers working in the field. Pre-hospital care providers cannot manually collect and analyze on-site information in real-time to determine the hospital for transfer, considering a patient’s condition. The prediction model’s feasibility can be guaranteed only when an algorithm can perform accurate predictions by rapidly processing real-time unstructured data that are automatically collected from the pre-hospital field [31]. To identify a hospital for transfer, a shared platform for hospital information on management quality provided by various medical institutions to pre-hospital care providers is needed. Therefore, a machine learning model that can rapidly extract prediction values by inputting multitudes of information should be loaded on the digital platform.

Although various clinically useful prediction tools developed using machine learning have been introduced, the lack of explanation, the so-called black box, is a factor that makes clinical application difficult [32, 33]. Our study attempted to verify the clinical relevance of the model using explainable machine learning called SHAP and analyze the evidence for its outcome clinically. Through SHAP, we identified clinical variables from the pre-hospital stage that play an important role in predicting patient class and hospital factors that influence the final model. These explanations can convince pre-hospital care providers to use this prediction model developed by machine learning. In addition, the XGBoost used in our study can reportedly perform better in classification and regression problems involving tabular data organized as rows and columns, which are the most common data types in traditional statistical modeling [34, 35]. As the datasets used in this study consist of tubular data, we used XGBoost as an algorithm for the prediction model.

Our study has several limitations that should be considered when interpreting its findings. Since our prediction model was developed based on a retrospective observational design, a potential bias is possible. Particularly, patients who would have died at any hospital or patients who were transferred for palliative care were also defined as inappropriate transfers, which lack clinical relevance. Our study dataset and design precluded the identification of these cases. Next, although we used three prospectively collected nationwide datasets, these were not completely merged because their matching keys could not be used to ensure anonymity. As this study was performed on a fire-department-based pre-hospital care system in South Korea, the prediction model was not tested according to a specific period or geographical area, and its generalizability is not guaranteed. Finally, our prediction model has not been validated in real-world scenarios. Therefore, its usefulness should be validated through additional prospective studies in the real pre-hospital care system.

Conclusions

Our prediction model using machine learning showed favorable performance in transferring patients with suspected cardiovascular disease to an appropriate hospital. For our results to be used as a basis to improve the pre-hospital care system in the real world, a digital platform for sharing real-time information should be developed, and additional prospective studies on its use are warranted.

Data availability

The datasets generated during the current study are available from the corresponding author upon reasonable request and with permission of National Fire Agency and National Emergency Medical Center in South Korea.

Abbreviations

AMI:

Acute Myocardial Infarction

AUPRC:

Area Under the Receiver Precision-Recall Curve

AUROC:

Area Under the Receiver Operating Characteristic Curve

EDs:

Emergency Departments

EMTs:

Emergency Medical Technicians

NEDIS:

National Emergency Department Information System

PCI:

Percutaneous Coronary Intervention

PHCR:

Pre-Hospital Cardiovascular Registry

PHRS:

Pre-Hospital Run Sheets

SHAP:

SHapley Additive explanation

TIMI:

Thrombolysis in Myocardial Infarction

References

  1. Anjum S. Systematic approach to acute cardiovascular emergencies. 2018. https://www.intechopen.com/chapters/60421. Accessed 04 Jan 2023.

  2. Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, Ezzati M, Shibuya K, Salomon JA, Abdalla S, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the global burden of Disease Study 2010. Lancet. 2012;380(9859):2197–223.

    Article  PubMed  Google Scholar 

  3. Bradley EH, Herrin J, Wang Y, Barton BA, Webster TR, Mattera JA, et al. Strategies for reducing the door-to-balloon time in acute myocardial infarction. N Engl J Med. 2006;355(22):2308–20.

    Article  CAS  PubMed  Google Scholar 

  4. Diercks DB, Kontos MC, Chen AY, Pollack CV Jr, Wiviott SD, Rumsfeld JS, et al. Utilization and impact of pre-hospital electrocardiograms for patients with acute ST-segment elevation myocardial infarction: data from the NCDR (National Cardiovascular Data Registry) ACTION (Acute Coronary treatment and intervention outcomes Network) Registry. J Am Coll Cardiol. 2009;53(2):161–6.

    Article  PubMed  Google Scholar 

  5. Jollis JG, Roettig ML, Aluko AO, Anstrom KJ, Applegate RJ, Babb JD, et al. Implementation of a statewide system for coronary reperfusion for ST-segment elevation myocardial infarction. JAMA. 2007;298(20):2371–80.

    Article  CAS  PubMed  Google Scholar 

  6. Kraft PL, Newman S, Hanson D, Anderson W, Bastani A. Emergency physician discretion to activate the cardiac catheterization team decreases door-to-balloon time for acute ST-elevation myocardial infarction. Ann Emerg Med. 2007;50(5):520–6.

    Article  PubMed  Google Scholar 

  7. Kurz MC, Babcock C, Sinha S, Tupesis JP, Allegretti J. The impact of emergency physician-initiated primary percutaneous coronary intervention on mean door-to-balloon time in patients with ST-segment-elevation myocardial infarction. Ann Emerg Med. 2007;50(5):527–34.

    Article  PubMed  Google Scholar 

  8. Park YH, Kang GH, Song BG, Chun WJ, Lee JH, Hwang SY, et al. Factors related to prehospital time delay in acute ST-segment elevation myocardial infarction. J Korean Med Sci. 2012;27(8):864–9.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Frisch A, Heidle KJ, Frisch SO, Ata A, Kramer B, Colleran C, et al. Factors associated with advanced cardiac care in prehospital chest pain patients. Am J Emerg Med. 2018;36(7):1182–7.

    Article  PubMed  Google Scholar 

  10. Stopyra JP, Harper WS, Higgins TJ, Prokesova JV, Winslow JE, Nelson RD, et al. Prehospital modified HEART score predictive of 30-Day adverse cardiac events. Prehosp Disaster Med. 2018;33(1):58–62.

    Article  PubMed  Google Scholar 

  11. Wibring K, Herlitz J, Christensson L, Lingman M, Bång A. Prehospital factors associated with an acute life-threatening condition in non-traumatic chest pain patients - a systematic review. Int J Cardiol. 2016;219:373–9.

    Article  PubMed  Google Scholar 

  12. Rawshani N, Rawshani A, Gelang C, Herlitz J, Bång A, Andersson JO, et al. Association between use of pre-hospital ECG and 30-day mortality: a large cohort study of patients experiencing chest pain. Int J Cardiol. 2017;248:77–81.

    Article  PubMed  Google Scholar 

  13. Karam N, Bataille S, Marijon E, Giovannetti O, Tafflet M, Savary D, et al. Identifying patients at risk for prehospital sudden cardiac arrest at the early phase of myocardial infarction: the e-MUST study (evaluation en Médecine d’Urgence des Stratégies Thérapeutiques des infarctus du myocarde). Circulation. 2016;134(25):2074–83.

    Article  PubMed  Google Scholar 

  14. Cho KJ, Kwon O, Kwon JM, Lee Y, Park H, Jeon KH, et al. Detecting patient deterioration using artificial intelligence in a rapid response system. Crit Care Med. 2020;48(4):e285–e9.

    Article  PubMed  Google Scholar 

  15. Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, et al. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019;139:84–91.

    Article  PubMed  Google Scholar 

  16. Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7(13).

  17. Ro YS, Shin SD, Lee YJ, Lee SC, Song KJ, Ryoo HW, et al. Effect of dispatcher-assisted cardiopulmonary resuscitation program and location of out-of-hospital cardiac arrest on survival and neurologic outcome. Ann Emerg Med. 2017;69(1):52–61.

    Article  PubMed  Google Scholar 

  18. Kim EN, Kim MJ, You JS, Shin HJ, Park IC, Chung SP, et al. Effects of an emergency transfer coordination center on secondary overtriage in an emergency department. Am J Emerg Med. 2019;37(3):395–400.

    Article  PubMed  Google Scholar 

  19. Kim JH, Han SG, Cho A, Shin HJ, Baek SE. Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study. BMC Med Inform Decis Mak. 2021;21(1):311.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2(1):56–67.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Żurowska-Wolak M, Piekos P, Jąkała J, Mikos M. The effects of prehospital system delays on the treatment efficacy of STEMI patients. Scand J Trauma Resusc Emerg Med. 2019;27(1):39.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Foo CY, Bonsu KO, Nallamothu BK, Reid CM, Dhippayom T, Reidpath DD, et al. Coronary intervention door-to-balloon time and outcomes in ST-elevation myocardial infarction: a meta-analysis. Heart. 2018;104(16):1362–9.

    Article  PubMed  Google Scholar 

  23. Grzybowski M, Zalenski RJ, Ross MA, Bock B. A prediction model for prehospital triage of patients with suspected cardiac ischemia. J Electrocardiol. 2000;33:253–8.

    Article  PubMed  Google Scholar 

  24. Kim JH, Kim MJ, You JS, Song MK, Cho SI. Do emergency physicians improve the appropriateness of emergency transfer in rural areas? J Emerg Med. 2018;54(3):287–94.

    Article  PubMed  Google Scholar 

  25. Holland CM, Lovasik BP, Howard BM, McClure EW, Samuels OB, Barrow DL. Interhospital transfer of neurosurgical patients: implications of timing on hospital course and clinical outcomes. Neurosurgery. 2017;81(3):450–7.

    Article  PubMed  Google Scholar 

  26. Javat D, Heal C, Banks J, Buchholz S, Zhang Z. Regional to tertiary inter-hospital transfer versus in-house percutaneous coronary intervention in acute coronary syndrome. PLoS ONE. 2018;13(6):e0198272.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Sorensen MJ, von Recklinghausen FM, Fulton G, Burchard KW. Secondary overtriage: the burden of unnecessary interfacility transfers in a rural trauma system. JAMA Surg. 2013;148(8):763–8.

    Article  PubMed  Google Scholar 

  28. Bible JE, Kadakia RJ, Kay HF, Zhang CE, Casimir GE, Devin CJ. How often are interfacility transfers of spine injury patients truly necessary? Spine J. 2014;14(12):2877–84.

    Article  PubMed  Google Scholar 

  29. Lee SJ, Choi A, Ryoo HW, Pak YS, Kim HC, Kim JH. Changes in clinical characteristics among febrile patients visiting the emergency department before and after the COVID-19 outbreak. Yonsei Med J. 2021;62(12):1136–44.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Al-Zaiti S, Besomi L, Bouzid Z, Faramand Z, Frisch S, Martin-Gill C, et al. Machine learning-based prediction of acute coronary syndrome using only the pre-hospital 12-lead electrocardiogram. Nat Commun. 2020;11(1):3966.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Kim H, Kim S-W, Park E, Kim JH, Chang H. The role of fifth-generation mobile technology in prehospital emergency care: an opportunity to support paramedics. Health Policy and Technology. 2020;9(1):109–14.

    Article  Google Scholar 

  32. Schwartz JM, George M, Rossetti SC, Dykes PC, Minshall SR, Lucas E, et al. Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study. JMIR Hum Factors. 2022;9(2):e33960.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Muralitharan S, Nelson W, Di S, McGillion M, Devereaux PJ, Barr NG, et al. Machine learning-based early warning systems for clinical deterioration: systematic scoping review. J Med Internet Res. 2021;23(2):e25187.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal, Canada: Curran Associates Inc.; 2018. p. 6639–49.

  35. Shwartz-Ziv R, Armon A. Tabular data: deep learning is not all you need. Inf Fusion. 2022;81(C):84–90.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work was funded by a grant from the 2019 IT Promotion fund (Development of AI based Precision Medicine Emergency system) of the government of Korea (Ministry of Science and ICT) and also from AI Ambulance project for digital healthcare of precision medicine SW development through the National Fund of the Korea government (Ministry of Science and ICT). The funding body had no rule in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Author information

Authors and Affiliations

  1. Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea

    Ji Hoon Kim & Min Joung Kim

  2. Department of Preventive Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea

    Bomgyeol Kim & Hyeon Chang Kim

  3. AITRICS, 28 Hyoryeong-ro 77-gil, Seocho-gu, Seoul, 06627, Republic of Korea

    Heejung Hyun

  4. Department of Cardiology, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun- gu, Seoul, 03722, Republic of Korea

    Hyuk-Jae Chang

Authors
  1. Ji Hoon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bomgyeol Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Joung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heejung Hyun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hyeon Chang Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hyuk-Jae Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JHK, MJK, HJC, and HCK conceptualized and designed the study; JHK and HH collected data; HH, BKK, and JHK analyzed and interpreted the data; JHK and BKK drafted the article; HCK revised the article critically for important intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hyeon Chang Kim.

Ethics declarations

Ethics approval and consent to participate

The study protocol was approved, and informed consent was waived by the Institutional Review Board of Severance Hospital, South Korea (approval number 4-2020-0110). The present study adhered to the ethical standards of the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests. AITRICS is the institution of one of the co-authors and had no rule in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary Material 2

Supplementary Material 3

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J.H., Kim, B., Kim, M.J. et al. Prediction of inappropriate pre-hospital transfer of patients with suspected cardiovascular emergency diseases using machine learning: a retrospective observational study. BMC Med Inform Decis Mak 23, 56 (2023). https://doi.org/10.1186/s12911-023-02149-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-023-02149-9

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us