Rapid response nursing triage outcomes for COVID-19: factors associated with patient’s participation in triage recommendations
BMC Medical Informatics and Decision Making volume 23, Article number: 47 (2023)
COVID-19 is an ongoing global health crisis with prevention and treatment recommendations rapidly changing. Rapid response telephone triage and advice services are critical in providing timely care during pandemics. Understanding patient participation with triage recommendations and factors associated with patient participation can assist in developing sensitive and timely interventions for receiving the treatment to prevent adverse health effects of COVID-19.
This cohort study aimed to assess patient participation (percentage of patients who followed nursing triage suggestions from the COVID hotline) and identify factors associated with patient participation in four quarterly electronic health records from March 2020 to March 2021 (Phase 1: 14 March 2020–6 June 2020; Phase 2: 17 June 2020–16 September 2020; Phase 3: 17 September 2020–16 December 2020; Phase 4: 17 December 2020–16 March 2021). All callers who provided their symptoms (including asymptomatic with exposure to COVID) and received nursing triage were included in the study. Factors associated with patient participation were identified using multivariable logistic regression analyses, including demographic variables, comorbidity variables, health behaviors, and COVID-19-related symptoms.
The aggregated data included 9849 encounters/calls from 9021 unique participants. Results indicated: (1) 72.5% of patient participation rate; (2) participants advised to seek emergency department care had the lowest patient participation rate (43.4%); (3) patient participation was associated with older age, a lower comorbidity index, a lack of unexplained muscle aches, and respiratory symptoms. The absence of respiratory symptoms was the only factor significantly associated with patient participation in all four phases (OR = 0.75, 0.60, 0.64, 0.52, respectively). Older age was associated with higher patient participation in three out of four phases (OR = 1.01–1.02), and a lower Charlson comorbidity index was associated with higher patient participation in phase 3 and phase 4 (OR = 0.83, 0.88).
Public participation in nursing triage during the COVID pandemic requires attention. This study supports using a nurse-led telehealth intervention and reveals crucial factors associated with patient participation. It highlighted the importance of timely follow-up in high-risk groups and the benefit of a telehealth intervention led by nurses serving as healthcare navigators during the COVID-19 pandemic.
The novel coronavirus (SARS-CoV-2; 2019-nCoV; COVID-19) is an urgent and ongoing global health crisis. As of 18 January 2022, California had about 6.8 million confirmed cases, with 77,306 deaths due to COVID-19 complications and a 21.1% test-positive rate . To prepare for a sudden surge in patients needing critical medical care and to reduce the possibility of overwhelming health systems, healthcare providers must work together to manage the influx of potential patients at an early stage. This includes managing the influx at an early stage, and identifying patients in the community early through triage hotlines and clinics to provide appropriate and timely care for patients during the COVID-19 pandemic . Nurses are uniquely positioned to lead community-based triage efforts to identify and recommend care for patients during the ongoing COVID-19 pandemic.
The outbreak of COVID-19 has impacted the usual face-to-face interactions between nurses and their patients. To provide high-quality clinical care and increase access to healthcare systems, many hospitals have increased the use of virtual and other telehealth systems to triage and care for patients. One of these technologies, Telephone Triage and Advice Services (TTAS), allows patients to speak to a nurse over the telephone to receive assessment and health advice . Nurses can quickly obtain detailed travel and exposure histories and act as first responders for patients at elevated risk . Automated screening algorithms may be built into the intake process, and local epidemiologic information can standardize screening and practice patterns across hospitals. Respiratory symptoms, which may be early signs of COVID-19, are most commonly evaluated using the TTAS approach.
Current research has revealed that TTAS are safe, promote care access, and increase service utilization [5,6,7]. During the ongoing pandemic, TTAS capabilities may have the potential to address increasing demands placed on overwhelmed health systems . The COVID-19 pandemic has created and reinforced the opportunity and advantage for nurses and healthcare providers to provide assessment and recommendations via phone, especially when face-to-face interaction is limited and a large number of patients seek care recommendations. TTAS also enables access to care for people living in remote or underserved communities and those with underlying comorbid conditions with limited resources. COVID-19 is an ongoing public health challenge and mitigating public transmission with adherence to COVID-19 measures is needed. TTAS is run and led by nurses, serving as critical healthcare navigators who provide an essential function in the fight against COVID-19, offering patient education and triaging patients to the appropriate level of care. Patient participation is a patient’s subsequent action adherence to the health providers’ recommendation. Nevertheless, there is little evidence that documents the impact of a nurse-led COVID-19 rapid response telephone triage center on patient participation with the recommendations, and few studies have examined factors associated with patient participation with the COVID-19 triage recommendations [8, 9].
Understanding patient participation with triage recommendations and factors associated with patient participation (such as age, sex, health conditions, and symptoms) can assist in developing sensitive and timely interventions for receiving the treatment needed to prevent adverse health effects of COVID-19. Lessons revealed in this study can inform healthcare providers and policymakers in making recommendations for managing an early response to public health concerns.
This study aimed to (a) assess patient participation rate with COVID-19 nursing triage recommendations in a large healthcare center in the United States and (b) identify factors (demographic data such as age, gender, comorbidity, obesity status, health behaviors such as smoking, drinking, and illicit drug used and symptoms) associated with patient participation with nurse recommendations at a University-Affiliated Medical Center. In this study, patient participation is defined as a patient’s subsequent action adherence to the telephone triage recommendation within 14 days of the initial triage call in the three-month phase.
A cohort study design was used to follow the patients over the two weeks after initiation of the triage hotline and advised them to follow the recommendations based on risk factors and reported symptoms. This study included patients from a large University-affiliated Medical Center in Northern California participating in the COVID-19 nursing triage hotline between 14 March 2020 and 21 March 2021. This study included a convenience sample of individuals who called the hotline, reported symptoms (including asymptomatic responses from patients who had exposure to COVID-19), and received nursing triage. Callers who called the hotline for COVID prevention or other COVID-related information without providing their symptom information for triage are omitted. This study utilized extensive data from the Medical Center’s electronic health records system. Study participants' consent was not required to use data from the electronic health records. This study has been approved by the Institutional Ethics Committee on Human Research of the University of California, San Francisco (20-31034).
A nurse triage hotline was established in a large University-affiliated medical center, Rapid Response Nursing Triage Hotline for COVID-19 called RN TO COVID, led by nurses who serve as health care navigators. Patients at this University-affiliated health center were advised to contact the COVID-19 hotline with questions or if they were experiencing COVID-19-related symptoms. All calls were taken by a triage nurse, and patients were asked about their symptoms, exposures, and comorbidities. The triage nurse then made care recommendations based on pre-determined triage protocols created by current clinical practice guidelines and COVID-19 symptomatology. Patients without symptoms but who had exposure to COVID-19 were asked to monitor their conditions and call back if they developed symptoms. Patients received four possible recommendations when they reported symptoms: self-care (SC), video acute care clinic visit (VACC within 24 h), respiratory symptom clinic (RSC in-person care within 12 h), and emergency department care (ED immediate care). A recent publication provides information on this triage protocol .
This study utilized extensive data from the Medical Center’s electronic health record system (Epic), available to hospital clinicians and researchers. Data were analyzed in four phases, based on quarterly periods of data collection from March 2020 to March 2021 (Phase 1: 14 March 2020–6 June 2020; Phase 2: 17 June 2020–16 September 2020; Phase 3: 17 September 2020–16 December 2020; Phase 4: 17 December 2020–16 March 2021). Individuals were included if they called the COVID-19 hotline, provided their symptoms, and received telephone triage decisions during the study period. This study used only initial call data per patient every three months. A response to the triage recommendation was determined by a patient’s action documented in the electronic medical record within 14 days of the triage call. Patients’ participation in the recommendation (VACC, RSC, and ED) were obtained in the electronic medical record system at the University-affiliated hospital system. All data extracted were de-identified. R v4.1.0  and STATA SE v16.0  were used for data cleaning and analysis.
Dependent variable: patient participation in COVID-19 triage recommendation is defined as the patient’s subsequent action adherence to the telephone triage recommendation within 14 days of the initial triage call in the three-month phase. Independent variables include demographic characteristics (age, gender), comorbidity (Charlson comorbidity index and obesity status), health behaviors (smoking, drinking, and illicit drug use), and symptoms (fever, unexplained muscle aches, eye-nose-throat (ENT) symptoms, eye symptom-(i.e., eye redness and/or discharge, respiratory symptom, upper respiratory infection (URI) symptom, gastrointestinal (GI symptom), and altered mental status).
Charlson comorbidity index (CCI) is a weighted index as a continuous variable to predict the risk of death within one year of hospitalization for patients with specific comorbid conditions. Higher scores indicate a more severe condition and a poorer prognosis and have been used consistently in the literature . Body Mass Index (BMI) was defined as the body mass (kg) divided by the square of the height (m2) and was calculated using self-reported weight and height. We defined obesity as having a BMI equal to or above 30 . This was a dichotomous variable. Health behaviors included smoking, drinking, and illicit drug use as binary variables (0 = never, 1 = quit or currently use). COVID-related symptoms included fever, unexplained muscle aches, ENT, eye symptoms, respiratory symptoms, URI symptoms, GI symptoms, and altered mental status, which were included in the hotline nursing triage (0 = No, 1 = Yes).
Descriptive analyses were used to describe demographic characteristics and patient participation. Chi-square tests were performed to examine differences in the prevalence of obesity, health behaviors, and COVID-19-related symptoms between patients who followed the triage recommendation and those who did not follow the recommendation. Multivariable logistic regression analyses were performed to determine the odds ratios (ORs) and 95% confidence intervals (CIs) between demographic variables (gender and age), comorbidity variables (CCI and BMI), health behaviors (smoking, drinking, and illicit drug used), and COVID-related symptoms and patient participation based on four different pandemic phases and all four phases. All statistical analyses were performed with STATA SE v16.0 with a significance value set at p < 0.05 and a Bonferroni corrected (for 15 tests) significance level of 0.0033 for logistic regression models.
The analysis included 9849 encounters/calls from 9021 unique participants (mean age = 45.4 years, SD = 21.1). Overall, about 39.5% of patients self-identified as male, ranging from 38.8% in phase 2 to 40.3% in phase 4. On average, 21.8% of patients were obese, ranging from 19.5% in phase 3 and 24% in phase 1. In terms of health behaviors, in the overall sample, 30.1%, 66.1%, and 23.8% of patients reported smoking, drinking, and using illicit drugs, respectively (Table 1). The most commonly reported symptoms included URI symptoms (37.6%), respiratory symptoms (30%), unexplained muscle aches (17.8%), and GI symptoms (13%, Table 1).
On average, of the total cohort, about 3.4% of patients (n = 331) were directed to the ED, 14.2% to RSC (n = 1401), 18.3% to Self-care (n = 1798), 21.1% to VACC (n = 2082), and 43% to self-monitor (n = 4238, Table 2).
The overall patient participation rate for the following triage recommendations was 72.5%; in the four phases, patient participation rates were 69%, 68.7%, 77.8%, and 72.5%, respectively. Significant differences were found among the four recommendations across the four study phases. Participants advised to seek ED care had the lowest patient participation (43.4%), in comparison with participants advised to seek care at a video acute care clinic (57.6%), a respiratory screening center (71.6%), or to administer self-care (67.7%) (Table 3).
In all 4 phases of aggregated data, older age, a lower Charlson comorbidity index (CCI), and lack of unexplained muscle aches and respiratory symptoms were associated with higher patient participation (Pseudo R2 = − 0.02; p < 0.001; Table 4). The absence of respiratory symptoms was the only factor significantly associated with higher patient participation in all periods [OR = 0.75, 0.60, 0.64, 0.52; 95% CI = (0.57, 0.91), (0.56, 0.85), (0.45, 0.75), (0.39, 0.70), respectively]. Older age was associated with higher patient participation in three out of four phases [OR = 1.01–1.02; 95% CI = (1.01, 1.02)], and a lower Charlson comorbidity index (CCI) was associated with higher patient participation in phase 3 and phase 4 [OR = 0.83, 0.88; 95% CI = (0.76, 0.90), (0.80, 0.97)] (Table 4).
This study aimed to assess the utilization of the COVID-19 hotline and patient participation with COVID-19 nursing triage recommendations and identify factors associated with patient participation in the early COVID-19 pandemic. More than 9,800 patients called the nurse-led triage COVID hotline during the first 12 months of the pandemic. The top three reported symptoms reported by patients include URI symptoms (37.6%), respiratory symptoms (30%), and unexplained muscle aches (17.8%). Of patients who reported symptoms, 21.1% were advised to VACC, 18.3% to Self-care, 14.2% to RSC, and 3.4% of patients to ED. In this study, the average patient participation rate of nursing triage recommendations was 72.5%, ranging from 68.7 to 77.8%. We found that participants advised to seek ED care had the lowest patient participation (43.4%). The absence of respiratory symptoms, older age, and lower Charlson comorbidity index are associated with higher patient participation in the recommendations.
The triage nurses’ recommendations were based on the triage protocol, including symptoms and other high-risk factors (such as age and other chronic health conditions). The COVID-19 hotline was heavily utilized by people seeking nursing triage and recommendations in the community. Our study found that about one-third of patients reported URI or respiratory symptoms. These respiratory-related symptoms reported by patients are consistent with studies suggesting that COVID-19 primarily targets the respiratory system. The virus can enter the respiratory tract through the mucous membranes of the mouth, nose, and eyes [15,16,17]. A recent systematic review and meta-analysis study on the prevalence of symptoms in adults from nine countries found that the most common clinical presentation of severe COVID-19 was fever (78%), followed by cough (57%) and fatigue (31%) . Current evidence suggests that individuals infected with COVID-19 experience various symptoms, from mild to severe conditions, and symptoms typically occur 2–14 days after virus exposure . The time gap between exposure to COVID and the onset of symptoms may be responsible for the low prevalence of some symptoms in our study since some callers reached the hotline after being exposed to COVID but without symptoms. As the virus mutates and evolves, the continued assessment of the symptoms reported by people with COVID-19 is critical in understanding the symptom presentation and influencing how patients perceive the need to follow the nurses’ recommendations and receive treatment when necessary. The average patient participation rate in nursing triage recommendations was 72.5%, ranging from 68.7 to 77.8%. Patients advised ED care had the lowest patient participation rate (43.4%), which is very concerning as the progression of severe symptoms can occur quickly and increase the risk for further health deterioration and delayed necessary hospitalization. Our findings are consistent with national data indicating that ED visits were lower between April and June 2020 . Delayed ED care increases the risk of not receiving appropriate care and not getting an accurate diagnosis, and prolonged hospitalization increases mortality [19, 20]. A current study in Sweden found the overall 60-day mortality rate as 17.4% during the first year of the pandemic . Studies have found high mortality rates ranging from 35 to 62% for critically ill patients with coronavirus [21,22,23]. Some possible rationales for non-adherence to the recommendation to ED care could include that individuals were concerned about the long waiting time in the ED or the likelihood of being infected with COVID while staying in the ED. Additionally, during the stay-at-home order, individuals may have had limited assistance at home, thus impacting their ability to seek acute ED care, and others may have been concerned about the cost of ED care. Timely treatment is essential in reducing the mortality associated with COVID-19. Future studies will need to explore the individual perception of the severity of illness and individuals’ rationale for not going to ED when advised.
Our study found that common factors associated with patient participation in the triage recommendation include the absence of respiratory symptoms and older and lower comorbidity. As the most common symptoms reported by people with COVID-19 include fever and dry cough , individuals without respiratory and lower comorbidity may be more likely to be asked to perform self-care or VACC, which is easier to follow through. Consistent with previous studies, individuals with a medical history and old age have an increased risk for infection and poorer COVID-19 outcomes due to weaker immune systems [24,25,26]. In the early pandemic, although there was limited knowledge of COVID-19 and its impact on health, there were several public health campaigns and regular announcements related to this newly discovered virus on media. These public announcements discussed the incidence of COVID-19 and shared information on the high-risk group for COVID-19 infection and mortality (such as older age and individuals with existing health conditions). The media coverage of COVID-19 influence individuals’ perception of their risk and decision on behavior changes (i.e., wearing a face mask, keeping social distance, frequent handwashing, getting medical attention) and vaccination intention [27, 28]. As older age has been reported to increase the risk for COVID-19-related hospitalization and mortality, individuals with older age may be more aware of their risk for COVID-19 and the health effects of COVID-19; thus, they are more likely to follow the nurses’ recommendations. To increase patient participation in nurses’ recommendations, public health organizations may need to engage various communities in health literacy champions that are culturally and linguistically appropriate to increase the understanding of the current recommendations for public health issues. Future studies may also need to explore the role of media on COVID-19 behavior change and medical care-seeking behaviors.
The results of this study provide recommendations for healthcare providers and policymakers regarding the management of the ongoing COVID-19 response and future public health emergencies. First, the utilization and participation rate of the nurse-based telephone triage system was moderate to high. A nurse-based telephone triage system based on the most up-to-date evidence can be quickly established to respond to emergency health events. The timeliness of the response is feasible and critical in providing timely and appropriate care. Second, timely follow-up with the patient regarding the recommendations, understanding the patient’s hesitancy in following the recommendation, and adapting policy in real-time are important factors in addressing the ongoing COVID-19 pandemic, as well as improving patient health outcomes. Third, patients advised to seek emergency department care had the lowest patient participation (43.4%), revealing the importance of timely follow-up for high-risk groups. Future clinical pathways or practice guidelines may need to consider factors (respiratory symptoms, younger age, and higher Charlson comorbidity index) in clinical decision-making.
This study is one of the first to explore the utilization of the nurse-led triage COVID-19 hotline. The results of the study should be interpreted with caution. Some limitations of the study include convenient sampling, the use of only electronic medical record data within a university-affiliated hospital, and self-reported data. The fewer report of some symptoms, such as altered mental conditions and fevers, may cause the models to be unstable, and the results must be interpreted carefully. Moreover, the percentage of variance explained by the model and ORs for age is small and other variables such as living alone or with family, history of illness, and previous relationships with healthcare workers could be included in future analyses. Another limitation is that we were unable to truly validate whether patients have performed self-care. We assumed that patients who did not use other services (such as VCC, RSC, and ED) within 14 days performed self-care. Future studies may reach out to patients concerning their self-care behaviors.
Future studies may need to consider other potential factors (such as symptom severity, medical history, knowledge about COVID-19, exposure to COVID-19, social determinants of health variables, access to treatment, and social support) related to patient participation in recommendations made by nurses and matching groups. Future work may have broader generalizability if other settings were included and a mixed-method design was used to examine barriers and facilitators following triage recommendations.
The average patient participation rate was 72.5%, with the ED being the lowest rate. The absence of respiratory symptoms, older age, and lower Charlson comorbidity index are associated with higher patient participation. Public participation in nursing triage during the COVID pandemic requires attention. This study supports using a nurse-led telehealth intervention, where nurses serve as critical healthcare navigators, providing assessment and care recommendations during the COVID-19 pandemic. While many adhered to nurse triage recommendations, those determined to be most critical did not; this is a concerning finding that warrants future study.
Availability of data and materials
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to them containing personal information.
Telephone triage and advice services
Video acute care clinic visit
Respiratory symptom clinic (RSC in-person care within 12 h) and emergency
Emergency department care
Eye-nose-throat (ENT) symptoms
Upper respiratory infection
Charlson comorbidity index
Body mass index (BMI)
California State Government. Tracking COVID-19 in California. 2022. [Cited 2022 1/18]. Available from: https://covid19.ca.gov/state-dashboard/.
Eggleton EJ. Simple, fast and affordable triaging pathway for COVID-19. Postgrad Med J. 2021;97(1145):192–5.
Purc-Stephenson RJ, Thrasher C. Nurses’ experiences with telephone triage and advice: a meta-ethnography. J Adv Nurs. 2010;66(3):482–94.
Bolina AF, Bomfim E, Lopes-Junior LC. Frontline nursing care: the COVID-19 pandemic and the brazilian health system. SAGE Open Nurs. 2020;6:2377960820963771.
Chambers D, et al. Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review. BMJ Open. 2019;9(8): e027743.
Lake R, et al. The quality, safety and governance of telephone triage and advice services: an overview of evidence from systematic reviews. BMC Health Serv Res. 2017;17(1):614.
McLean S, et al. The impact of telehealthcare on the quality and safety of care: a systematic overview. PLoS ONE. 2013;8(8):e71238.
Caponnetto V, et al. The COVID-19 pandemic as an opportunity to improve health care through a nurse-coordinated multidisciplinary model in a headache specialist center: the implementation of a telemedicine protocol. Telemed J E Health. 2022;28(7):1016–22.
Raesi R, et al. The impact of education through nurse-led telephone follow-up (telenursing) on the quality of life of COVID-19 patients. J Egypt Public Health Assoc. 2021;96(1):30.
Judson TJ, et al. Rapid design and implementation of an integrated patient self-triage and self-scheduling tool for COVID-19. J Am Med Inform Assoc. 2020;27(6):860–6.
Team RC. R: a language and environment for statistical computing. 2013.
StataCorp. Stata statistical software: release 17. College Station, TX: StataCorp LP; 2021.
Nanda S, et al. Body mass index, multi-morbidity, and COVID-19 risk factors as predictors of severe COVID-19 outcomes. J Prim Care Community Health. 2021;12:21501327211018560.
Wu S, et al. Metabolic health has greater impact on diabetes than simple overweight/obesity in Mexican Americans. J Diabetes Res. 2016;2016:4094876.
Grant MC, et al. The prevalence of symptoms in 24,410 adults infected by the novel coronavirus (SARS-CoV-2; COVID-19): a systematic review and meta-analysis of 148 studies from 9 countries. PLoS ONE. 2020;15(6): e0234765.
Brosnahan SB, et al. COVID-19 and respiratory system disorders: current knowledge, future clinical and translational research questions. Arterioscler Thromb Vasc Biol. 2020;40(11):2586–97.
Harrison AG, Lin T, Wang P. Mechanisms of SARS-CoV-2 transmission and pathogenesis. Trends Immunol. 2020;41(12):1100–15.
Smulowitz PB, et al. National trends in ED visits, hospital admissions, and mortality for medicare patients during the COVID-19 pandemic. Health Aff (Millwood). 2021;40(9):1457–64.
Bae SJ, et al. Comparison of the clinical process and outcomes in patients after coronavirus infection 2019 outbreak. Medicina (Kaunas). 2021;57(10):1086.
Stralin K, et al. Mortality in hospitalized COVID-19 patients was associated with the COVID-19 admission rate during the first year of the pandemic in Sweden. Infect Dis (Lond). 2022;54(2):145–51.
Bhatraju PK, et al. Covid-19 in critically Ill patients in the Seattle region—case series. N Engl J Med. 2020;382(21):2012–22.
Gupta S, et al. Factors associated with death in critically ill patients with coronavirus disease 2019 in the US. JAMA Intern Med. 2020;180(11):1436–47.
Yang X, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. 2020;8(5):475–81.
Barek MA, Aziz MA, Islam MS. Impact of age, sex, comorbidities and clinical symptoms on the severity of COVID-19 cases: a meta-analysis with 55 studies and 10014 cases. Heliyon. 2020;6(12): e05684.
Jiang Y, et al. Risk factors associated with 28-day all-cause mortality in older severe COVID-19 patients in Wuhan, China: a retrospective observational study. Sci Rep. 2020;10(1):22369.
Leijte WT, et al. Mortality and re-admission after hospitalization with COVID-19. Ned Tijdschr Geneeskd. 2020;164:D5423.
Gozzi N, et al. Collective response to media coverage of the COVID-19 pandemic on Reddit and Wikipedia: mixed-methods analysis. J Med Internet Res. 2020;22(10): e21597.
Zhou WK, et al. Effects of media reporting on mitigating spread of COVID-19 in the early phase of the outbreak. Math Biosci Eng. 2020;17(3):2693–707.
We would like to acknowledge Heather Leicester, MSPH Sr. Programmer Quality Analyst, for her assistance with data management and participants who seek care at the medical center.
This research was funded by the University of California Office of President Emergency COVID-19 Research Seed Funding.
Ethics approval and consent to participate
The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Ethics Committee on Human Research of the University of California San Francisco (20-31034). Study participants' consent was not required to use data from the electronic health records.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Chen, JL., Lin, CX., Park, M. et al. Rapid response nursing triage outcomes for COVID-19: factors associated with patient’s participation in triage recommendations. BMC Med Inform Decis Mak 23, 47 (2023). https://doi.org/10.1186/s12911-023-02139-x