Skip to main content
Download PDF

Transition to a new nursing information system embedded with clinical decision support: a mixed-method study using the HOT-fit framework

BMC Medical Informatics and Decision Making volume 22, Article number: 310 (2022) Cite this article

Abstract

Background

Nursing information systems embedded with standardized nursing language and clinical decision support have been increasingly introduced in health care settings. User experience is key to the adoption of health information technologies. Despite extensive research into the user experience with nursing information systems, few studies have focused on the interaction between user, technology and organizational attributes during its implementation. Guided by the human, organization and technology-fit framework, this study aimed to investigate nurses’ perceptions and experiences with transition to a new nursing information system (Care Direct) 2 years after its first introduction.

Methods

This is a mixed-method study using an embedded design. An online survey was launched to collect nurses’ self-reported use of the new system, perceived system effectiveness and experience of participation in system optimization. Twenty-two semi structured interviews were conducted with twenty nurses with clinical or administrative roles. The quantitative and qualitative data were merged using the Pillar Integration Process.

Results

The average score of system use behavior was 3.76 ± 0.79. Regarding perceived system effectiveness, the score of each dimension ranged 3.07–3.34 out of 5. Despite large variations in approaches to participating in system optimization, nurses had generally positive experiences with management and technical support. Eight main categories emerged from the integrated findings, which were further condensed into three themes: perceptions on system content, structure, and functionality; perceptions on interdisciplinary and cross-level cooperation; and embracing and accepting the change.

Conclusions

Effective collaboration between clinicians, administrators and technical staff is required during system promotion to enhance system usability and user experience. Clear communication of organizational missions to staff and support from top management is needed to smooth the system implementation process and achieve broader system adoption.

Peer Review reports

Background

The electronic health record (EHR) has become the mainstream of nursing documentation with the introduction of nursing information systems (NIS). Standardized nursing languages (SNLs) are a commonly understood set of terms used to describe the clinical judgments involved in nursing care [1]. Embedding SNLs into the NIS is essential for extraction, exchange and integration of nursing data across disciplines and institutions, achieving secondary utilization of nursing information [2]. Based on standardized data on patient history and nursing assessment results, clinical decision support (CDS) is made available to accomplish meaningful use of EHR and to assist health care professionals with decision-making [3]. SNL and CDS are among the top research priorities in nursing informatics [4]. Despite its potential to support and transform nursing practice by simplifying record-keeping, promoting standards-based practice, and giving timely access to information to aid decision-making [5], the introduction of NIS, does not necessarily lead to user adoption [6]. Research showed that NIS has changed the way nursing is practiced, with mixed findings identified in terms of information quality and access, documentation burden, time spent on patient care, communication and care coordination, quality of care and ultimately, nurse and patient satisfaction [7, 8], which have implications for administration decisions on the implementation of NIS.

Multiple technology acceptance theories recognize user experience as key to the adoption of health information technologies (HITs) [9]. HITs poorly adapted to the work context can cause contradictions to nursing workflows, compromising EHR usability [10]. Perceived poor EHR usability is associated with a higher level of emotional exhaustion among nurses [11], hindering system adoption [12]. Therefore, a user-centered design with the participation of frontline staff is needed from the pre-introduction to the postimplementation stage to improve system usability to facilitate user adaptation [13]. Based on the Information System Success Model and the IT-Organization Fit Model, the human, organization and technology-fit (HOT-fit) framework proposed by Yusof et al. [14] can be used to evaluate the influence of user attitude and skills, communication, leadership and an IT-favorable environment on HIT adoption. This framework has implications for anticipating and preventing implementation barriers from occurring and exploiting facilitators to successful HIT implementation [15].

Despite the increasing popularity of CDS, its provider uptake remains unsatisfactory, with a recent meta-analysis revealing the overall uptake of clinical decision support systems among 3607 providers to be as low as 34.2% [16]. Transition to a new NIS can be challenging and has been reported to cause changes in nurses’ routine practices, leading to emotional insecurity and stress [17]. A recent Dutch study showed that almost half of respondents experienced results worse than their expectations 7 months after the implementation of a structured and standardized EHR [18]. Research into the interaction between technology-related, dispositional, and contextual attributes during CDS implementation is worth exploring. However, to the best of our knowledge, no study has focused on these multi-layer interactions in the nursing context. This study reports on the implementation of a CDS-embedded commercial NIS (care direct) in a large tertiary hospital in China. Guided by HOT-fit framework, this study sought to address the following research questions: (1) What are nurses’ perceptions and attitudes toward Care Direct? (2) How does the implementation of Care Direct affect nurses' daily practice? (3) How do technology, organizational and human attributes affect user adoption of Care Direct?

Methods

Design

This is a mixed-method study using an embedded design in which qualitative data and quantitative data were collected concurrently with the former given priority. The mixed-method design was used because it enabled researchers to gain a deeper understanding of the topic of interest where different sets of data triangulated with each other, contributing to the credibility of the study findings [19]. In our study, the quantitative and qualitative data (survey findings, obervation notes and interview transcriptions) were integrated at the analysis level. This study was reported in accordance with the Good Reporting of a Mixed Methods Study (GRAMMS) checklist [20] (see Additional file 1).

Care direct

Developed in accordance with the internationally consented standard for nursing clinical decision support systems in EHRs [21], Care Direct integrated the SNL, evidence-based nursing knowledge base and big data analysis resources, with all data contents completely disassembled into the minimun data set and encoded with the SNL, meeting the national requirements of healthcare data standardization in China. An illustration of the modules in Care Direct and algorithms for documenting patient care is provided in the Additional file 2: Fig. S1.

Study context

This study was conducted in a 2000-bed tertiary general hospital in Shanghai, China. Care Direct (the new NIS) has been running in parralel with the old hospital information system since its introduction in late 2018. At the commencement of this study in November 2020, Care Direct had been in pilot use (running in parallel with the old system) in 24 general medical-surgical wards for 1–17 months. The interoperability between Care Direct and exisiting hospital information systems was constantly optimizing during this study. The timeline of Care Direct implementation is shown in Fig. 1.

Fig. 1
figure 1

The timeline of Care Direct introduction

Full size image

A project team was launched during the system design phase. A nurse manager (30 years in nursing) with previous experience of NIS implementation was appointed as the project champion. An on-site customer service personnel (2 years experience in HIT development) appointed by the vendor stationed in the hospital during workdays to respond to technical issues raised by nurses together with background technical staff as well as to communicate with nurse leaders regarding issues of system improvement. There was also a project advisor (30 years of experience in HIT development) and a technical director (11 years of experience in HIT development) who remotely connected with the team members while engaging in system development. No major changes in team members took place during the 3-year system implementation period except for the on-site customer service personnel.

Sample

For the quantitative part, a cluster sampling was used to recruit nurses from 24 pilot wards to collect their perceptions and experience with Care Direct. For semi structured interviews, purposive sampling [22] was used to recruit nurses to ensure the diversity of experience, position, professional title, educational background and level of participation during system development in our study samples. The sample size was determined based on the principle of data saturation (no new category or concept appeared). Nurses on long-term leaves of absence, employed for less than 1 year and student nurses were excluded from this study due to limited access to or recent use of the NISs.

Measure

The following instruments were used to collect nurses’ perceptions and experiences with Care Direct from various aspects: (1) human: the Nursing Information System Use Behavior Scale for nurses revised by Wen [23], (2) technological: the Clinical Nursing Information System Effectiveness Scale developed by Zhao [24] based on the Information System Success Model, and (3) organizational: a questionnaire about participation in system optimization developed based on the literature [25, 26]. For the third instrument, its construct and content validity were tested. The exploratory factor analysis extracted two principal components with eigenvalues greater than one, suggesting that the instrument can be divided into two factors: (a) nurses’ degree of participation in system development (5 items) and (b) nurses’ experience with participation in system development (8 items), explaining 75.47% of the cumulative variance. Seven experts specialized in nursing informatics were invited to evaluate the relevance of the items on a 4-point Likert scale, which resulted in a content validity index of 0.939. Revisions were made to three items according to expert opinions. The psychometric properties of the instruments in our study are shown in Table 1.

Table 1 Validity and reliability of the instruments used in this study
Full size table

Data collection

Participant observation

From January to December 2021, the primary investigator (a research nurse with no clinical responsibilities) worked with the nurses and observed their use of Care Direct. The protocol of observation was developed based on the HOT-fit framework. On the wards, the observation focused on nurses’ interaction with Care Direct on which they performed routine tasks, especially its CDS options; meanwhile, nurses’ feedbacks regarding any technical issues (technological aspect) and capability and willingness to use Care Direct (human aspect) were also collected. We also paid attention to the leader role (of the charge nurse) in encouraging and standardization of the use of Care Direct within the unit (organizational aspect). (2) The primary investigator also joined a group chat involving core team members of system implementation and attended regular biweekly group meetings addressing system-related issues and work plans on system implementation as an observer to gain insight into the cross-level and multi-disciplinary collaboration during system optimization (organizational aspect).

Semi structured interviews

From October 2021 to January 2022, personal in-depth interviews were organized by the primary investigator to collect nurses' views and experience of Care Direct to verify and supplement the findings in the survey. An interview outline was prepared according to the research questions and existing frameworks (refer to Additional file 3). The topic guide was flexibly used to adapt to the different clinical roles of and responses from the interviewees by the primary investigator. Another investigator kept a note of the tones, gestures and facial expressions of the interviewee (s). The interviews were audio-recorded and transcribed verbatim within 24 h. Data analysis and collection were carried out simultaneously, and the interview outline was iteratively modified based on the data analysis results, which is conducive to the in-depth analysis of the themes and authenticity of findings [27]. Data reached saturation at 22 person-times. Information about the participants is shown in Table 2.

Table 2 Characteristics of interviewees (n = 20)
Full size table

Questionnaire survey

An online questionnaire containing 48 required items (four items on nurses’ demographics) was used to collect nurses’ perceptions and experiences with Care Direct via Questionnaire Star (an online survey software). There were also two optional open-ended questions that prompted nurses to express their perceived advantages/disadvantages of Care Direct and expectations for its improvement. We asked the nurse manager of each block to distribute the questionnaire link to the charge nurse of each pilot ward after a clear explanation of the objective and inclusion and exclusion criteria of this study. Nurses were required to carefully read the instructions before completing the questionnaire. Respondents could submit the questionnaire only after completing all the required items, so there were no missing items; however, returned questionnaires with a response time of less than 90 s were excluded to ensure the validity of the results. A total of 384 nurses participated in the online survey, among whom 324 (84.4%) were included in the analysis. The demographics of the survey participants are shown in Table 3.

Table 3 Characteristics of survey respondents (n = 324)
Full size table

Data analysis

Quantitative Data were imported into SPSS 22.0 for statistical analysis. The measurement data were described by the mean (standard deviation) or median (quartile) depending on the normality of the distribution. The counting data were described by frequency (percentage). The rank sum test was used to compare the differences in multigroup hierarchical data. On the qualitative branch, Investigators read through the observation notes and interview transcriptions several times to familiarize themselves with the contents.

The two sets of data were merged at the interpretation level using the Pillar Integration Process (PIP) proposed by Johnson et al. [28], presented in the form of a table including five row headings, quantitative data, quantitative concepts, categories, qualitative concepts and qualitative codes (Fig. 2). The PIP is a four-step process [28]: listing, matching, checking and pillar building. The categories emerged from the PIP were further integrated into themes using inductive reasoning and a final framework was formed.

Fig. 2
figure 2

An illustration of the PIP [28]

Full size image

Trustworthiness and rigor

To enhance the trustworthiness of this study, member checking was employed to paraphrase and summarize the participants’ statement to ensure the intended meaning was accurately conveyed [29]. Interview transcripts were also returned to the interviewees within 24 h for verification, which ensured dependability. Transferability to other contexts was assured by describing the context, purposive selection of participants with different characteristics and using the PIP for conducting data analysis. Confirmability was guaranteed by various approaches to data collection and triangulation of multiple data sources guided by a biostatistician as an external auditor on study design, instrument development, data reconstruction and synthesis. To reduce the investigators’ own bias on the research process and results, constant self-reflexivity was employed during data collection and analysis to eliminate the possible interference of personal emotions and opinions with the research results [30].

Results

Quantitative results

System use behavior

The average score of system use behavior was 3.76 ± 0.79, with most respondents (53.40–73.46%) responding “agree” or “strongly agree” for each item. Refer to see Additional file 4: Table S1 for detailed results.

Perceived system effectiveness

The average scores of items in the five dimensions are as follows: system quality: 3.23 ± 1.00, information quality: 3.34 ± 0.98, service quality: 3.23 ± 17.69, user satisfaction: 3.07 ± 0.97, net benefit: 3.07 ± 1.05. Among the 23 items, the proportion of participants responding with “strongly agree” and “agree” ranged from 23.15 to 61.73%, with the median percentage being 36.73%. Refer to Additional file 4: Table S2 for the responses to each item.

Experience of participation in system development

Users' participation in NIS development was medium, with the percentage of participants responding with “very often” and “often” < 50% for all items (Table 4). There were significant differences in nurses' participation in different system improvement approaches (χ2 = 34.097, p < 0.001).

Table 4 Participating in system optimization: approaches and user experience (n = 324)
Full size table

Nurses had an overall positive experience with respect to communication with the management and technical staff during system development, with the percentage of participants responding with “strongly agree” and “agree” > 70% (Table 4). There were significant variances in nurses’ experience regarding different aspects of technical support (χ2 = 32.781, p < 0.001).

Merging qualitative and quantitative findings

The qualitative data were merged with the quantitative data via the Pilar Integration Process (Table 5).

Table 5 Merging quantitative and qualitative data using the PIP
Full size table

Integrated findings

Eight main categories emerged from the integrated findings, which were further condensed into three themes: (a) perceptions on system content, structure and functionality; (b) perceptions on interdisciplinary and cross-level cooperation; and the overarching theme (c) embracing and accepting the change.

Theme 1: perceptions on system content, structure and functionality

Record template increasing documentation burden

The documentation framework based on the nursing process in Care Direct is not conducive to reflecting the whole picture and the dynamic changes of patient conditions, which compromised the communication of information among nurses. With the large amount of information to be collected in the past history module and some items not being closely related to the patient's conditions, nurses felt it a mere formality to meet the documentation requirement, which may have caused negative experiences for the patient. Moreover, the lengthy overview containing a large amount of irrelevant information generated by Care Direct failed to meet nurses’ information needs; therefore, Care Direct was rarely referred to as a source of information exchange. The need to frequently switch between screens also added to nurses’ documentation burden and increased the risk of missing information (Fig. 3 and Table 5).

Fig. 3
figure 3

Illustration of the steps of record-keeping in care direct (A documentation of observation items for acute postoperative patients; B documentation of fluid output)

Full size image

Suboptimal information linkage

Negative experiences regarding issues with information linkage in Care Direct were frequently referred to in our study. While Care Direct achieves automatic linkages across each stage of the nursing process through CDS with a preset criteria for reassessment according to patients’ care demands, it cannot give decision support in response to changes in patient condition such as abnormal vital signs, abnormal laboratory results, and documented signs/symptoms. Although the system allows users to add nursing problems and update care plans manually, nurses would not bother to do so based on the findings from on-site observations and record review by the investigators.

Many decision support rules of Care Direct were triggered by medical diagnoses or medical orders generated in the old system, but the push timing did not conform to the clinical workflow, which required nurses to manually add or delete schedules according to work routines and actual situations. Problems with information linkages also hindered the system from generating accurate nursing records based on user actions. While Care Direct allows users to view and edit the nursing documentation, heavy clinical tasks kept nurses from spending much time on verification (Table 5).

Value of CDS

Care Direct automatically links the five steps of the nursing process in the form of decision support, as opposed to the separate status of different components in the old system. The value of system decision support is also reflected in the daily reminder for nurses. Clinical nursing is composed of numerous tasks and the schedule generated by the system can remind nurses of tasks they tend to neglect, ensuring delivery of quality care. Nevertheless, nurses had different levels of perceived system benefits. Some nurses thought that CDS could supplement their clinical reasoning to facilitate decision-making; others, however, viewed CDS as a disruption to their inherent thinking and work habits and thus were reluctant to follow the system’s recommendations (Table 5).

Theme 2: perceptions on interdisplinary and cross-level collaboration

Insufficient training on system use

Shortly after the initiation of pilot use, training on system use was organized to familiarize nurses with Care Direct. To ensure that every nurse on each pilot ward attended at least once, the training was carried out during the lunch break every weekday. The way the training was organized, however, was inconducive to the learning efficiency of participating nurses. Moreover, the content of training sessions provided limited support for nurses. Due to the lack of contexts, it failed to solve the problems encountered by nurses during daily use of the system. Lack of continuity of training was also a problem for nurses. As Care Direct was only piloted in general wards, some junior nurses who later rotated from other units missed the training sessions (Table 5).

Collaboration between nurses and technical staff

Due to interprofessional barriers between nurses and technical staff, cooperation between the two was suboptimal. Lacking understanding of the nursing workflow, technical staff had trouble understanding the requests made by nurses, leading to misalignment between user expectations and system outcomes. The survey showed that among all aspects of nurses' experience with participating in system development, their satisfaction with the speed of request solving was the lowest. Unmet needs led to nurses’ disenchantment, rendering them unwilling to provide additional feedback; instead, nurses chose to adapt to the imperfections of the system. Nevertheless, nurses did recognize efforts made by the technical staff and understood that the heavy workload they were facing hindered them from handling user requests in a timely manner (Table 5).

Leader role

Most nurses agreed that the management took their suggestions and experiences seriously. Support from management is key to user adoption. Although nursing documentation under Care Direct was temporarily out the scope of quality audit, to ensure nurses smoothly transition after it goes live, charge nurses on the wards communicated with staff nurses about the omissions identified from regular record review to determine whether these were caused by the user or technical defects, which urged nurses to establish positive system use behavior and was also conducive to system optimization (Table 5).

Cross-level collaboration

Care Direct was a commercial NIS purchased from a third party without adequate input from frontline nurses at the design phase; therefore, local adaptation was needed to improve its suitability. A feedback pathway involving the project leader with previous experience in NIS development (nurse champion), nurse managers/informatics nurses at each block, ward nurses and technical staff was established (Fig. 4); meanwhile, a project team including the core members involved in system implementation was launched. Taking a bottom-up approach, each ward reported system-related problems and improvement requests to management, which would be gathered by the nurse champion. Problems were discussed in person among nurse leaders and common problems were summarized before being put forward on the project meetings. Relevant materials were prepared by the nurse leaders and submitted to the technical staff to initiate improvements. Relying on the wisdom of nurses at all levels, the integrity of system content and function was constantly improving to streamline nursing workflow  (Refer to Additional file 5 for details of upgrades in each system module) (Table 5).

Fig. 4
figure 4

A bottom-top approach to raising requests and feedbacks

Full size image

Despite the overall positive outcomes of cross-level collaboration, it was not without barriers. Designation of responsibilities seemed to be suboptimal among the project team, rendering the project champion to face significant pressure during system implementation. Sometimes, efforts and outputs made by the project champion awaiting feedback were not responded by other team members or top management, possibly due to heavy administrative workload, which also led to her frustration (Table 5).

Overarching theme: embracing and accepting the change

Since the implementation of Care Direct was decided by the management, its use was mandatory, and competence in handling the system would be part of nursing practice. Therefore, some nurses explored system functions and gradually became proficient users. Most survey respondents agreed that they had become accustomed to the system. With routinized system use and improvement in system functionality, most nurses adapted to Care Direct and incorporated it into their daily practice. However, some participants who were lagging in HITs felt overwhelmed by the two NISs running in parallel and thus were slower to get on board with the new system (Table 5).

Overall, nurses generally took a rational position toward the benefits and hardships during their transition to a new NIS. While Care Direct was constantly adapting to the needs of users, users were also constantly adapting to Care Direct. A proposed framework demonstrating the relationships between the categories and themes is shown in Fig. 5.

Fig. 5
figure 5

The proposed framework of user adoption

Full size image

Discussion

The aim of this study was to investigate nurses' views and experiences during transition to a new NIS, focusing on the interaction of organization, technology and human attributes during system implementation. Findings from this mixed-method study revealed both positive and negative emotions related to system content, structure and functionality, interdisciplinary and cross-level collaboration. Despite nurses’ mixed emotions towards the implementation of Care Direct, they tended to integrate it into their routine workflow.

The results of this study showed that the lengthy documentation templates and suboptimal data linkage compromised the perceived usability of Care Direct. The introduction of SNL-based recording templates reduced the heterogeneity in free-text documentation, promoting data integration [2] and the quality of nursing documentation [31]. Notwithstanding their contribution to the integrity of nursing records, structured templates within the NIS failed to fully match the nurses' complex and dynamic workflow [32]. Fragmented information forced nurses to navigate between different interfaces to search for information, adding to their cognitive burden [33], as also reflected in our findings. Moreover, the restrictive nature of structured documentation framework has been criticized as compromising the accuracy of documentation [34, 35]; therefore, system-generated care plans and summaries were rarely referred to as a guidance to practice, but as a documentation requirement to serve administrative needs [36]. Considering the drawbacks regarding the clinical benefits of structured documentation framework, free-text input have been made available to supplement SNL-based documenation. Technical experts should be consulted to analyze the impact of Care Direct implementation on nurses’ information seeking and sharing practices and propose strategies to adapt it to the nursing workflow by, for instance, optimizing the relevance of the content within the summary/overview of care and shift reports generated by the system.

To ensure the clinical usefulness and user adoption of HITs, frontline nurses should take a dominant role during all phases of its implementation to voice their expectations [37]. In our study, the bottom-up feedback mechanism enabled all end users to express their concerns and expectations with Care Direct, contributing to broad participation. This hierarchical feedback system, however, deprived nurses of opportunities to sit at the same table with management and technical personnel, as demonstrated by the survey results. Our study also indicated that end users appreciated exchanging ideas with the nurses responsible for system development, which is consistent with previous findings [26]. Grounded in the clinical setting, informatics nurses acted as advocates for bedside nurses while working closely with technical staff to raise suggestions regarding usability issues with full consideration of the nursing workflow [38]. Before the introduction of Care Direct, the informatics nurse post was set up in our institution, with one nurse assigned to this position in each block, responsible for gathering and reporting system-related issues, assisting the project leader in drafting improvement strategies and connecting with technical staff.

The accessibility of technical support is an important factor affecting the implementation of HITs. In our study, contradictory findings were found regarding the perceived timeliness of technical support in the survey and interviews. In the survey, respondents provided an overall positive feedback based on the average score whereas the interviews revealed more negative experience. This was probably because some users tended to emphasize their negative experiences with system use due to its huge influence on their daily practice while selectively neglecting the positive ones. Another explanation is that they hoped their negative experience would raise concerns among investigators to guide future system improvement. A nationwide survey [26] conducted across Finnish public hospitals found that most clinicians perceived software vendors as being unresponsive to user feedback; however, technical staff had diametrically opposed views on these issues [25]. A possible explanation is that technical staff mainly interact with user representatives who are in administration positions and lack personal experience with end users’ pain points during daily use of the system, which was the case in our study, and there may be a gap between the user representative’s understanding and end users’ expectations. The lack of two-way communication between end users and technical staff is prone to negative emotions among users and the belief that their needs are not valued, leading to their disengagement with system implementation [39]. As the most direct method of information communication, in our study, on-site observation and demonstration of system use were frequently employed as approaches to identifying problems between end users and technical staff, which is also in line with previous studies [25, 26].

Support from leaders were well-received by nurses based on our investigation. During pilot use of Care Direct, the charge nurse in each unit generally took on the role of super user due to their high degree of participation in system development, and this produced moral effects to promote positive system use behavior in the whole unit. Nurse leaders play an important role in the promotion of HITs, as their support and supervision are imperative to leading nurses through resistance and doubt to achieve organizational change [40]. However, studies have reported that nurse leaders face significant obstacles in driving the implementation of HITs, such as insufficient understanding of the value of HITs due to limited informatics literacy, time constraints due to administrative tasks, and lack of support from top management, which hindered their ability to provide adequate support to nurses and make informed decisions about system improvement [41, 42]. In our study, despite the long-term commitment to HIT promotion and proactive leadership and partnership with technical staff demonstrated by the nurse champion during system implementation, barriers were encountered regarding training organization, interdisciplinary interaction and lack of engagement from top management. Therefore, implementation strategies are needed during future HIT implementation, and attention should be given to strengthen nurse leaders’ project management competencies [43].

User-perceived benefits are key to system adoption. As the end user of the new system, nurses' willingness to adopt Care Direct is largely related to their acceptance of it. Previous research [44] indicated that nurses will take the previous major change events as a reference to form their expectations for organizational change. We assumed that in the early stage of Care Direct implementation, the gap between system function and nurses’ expectations aggravated their negative response to the system, while the later acceptance came from their gradual adaptation to the system and the continuous improvement of system function. Despite their mixed feelings toward Care Direct, nurses tended to adapt to it rather than return to the original record-keeping modalities.

Finally, our study showed that nurses have mixed opinions about their perceived value of CDS. It is worth noting that the decision-making suggested by the CDS is to supplement rather than replace the professional thinking of nurses. Clinical experience is an important determinant of the perceived benefits from CDS. By using clinical intuition, senior nurses have a better grasp of the overall situation of patients and have internalized the nursing process into practice, thus having lower needs for practice guidance from the CDS [45]; fully following the clinical practice recommendations would reduce their autonomy [46]. Previous research [47, 48] showed that nurses rarely make decisions based on the CDS recommendation alone but combine subjective and objective patient information to identify the possible deviations of CDS and to reach more accurate and comprehensive decisions. Therefore, while enjoying the convenience brought by CDS, nurses still need to cultivate problem-solving skills by transforming readily available knowledge into improved care quality and patient outcomes [49].

Implications for future research/practice

As with all HIT implementation, optimization of system structure and content should be an ongoing process with continuous input from both nursing and technical staff following its implementation. Technical issues regarding both software and hardware should also be solved by actively approaching hospital informatics personnel. From a safety perspective, the system should incorporate a double check mechanism to minimize the negative consequences of erroneous data linkage. To bridge the gap between nurses and technical staff, frontline nurses, who have deepest connection with the new HIT, should be given the opportunity to participate in learning sessions on nursing informatics to provide suggestions for interface customization. Technical staff should also be invited to immerse themselves in the clinical environment to familiarize themselves with the nursing workflow so that they will better understand user requests.

Nurses’ negative emotions are a common phenomenon during their interaction with the system. To mitigate the potential detrimental consequences of system-related negative emotions, during system pilot-run, management should pay attention to nurses’ additional workload related to system use and actively build connections with nurses to obtain their feedbacks. In future studies, structured observation checklists or nurse self-rated questionnaires could be used to investigate the impact of EHR on their care delivery to come up with strategies to avoid overburdening nurses.

Strengths and limitations

This study has several strengths and limitations. First, our study recruited nurses from both clinical and administrative positions from purposive sampling where nurse leaders from the core member team shared their views on inter-disciplinary collaboration. However, insights from the project advisor and technical staff were lacking as their perspectives could complement with or possibly differ from those of nurses and could have been valuable to our study. Second, the cross-sectional nature of the quantitative branch precluded us from revealing the longitudinal changes of nurses’ perceptions and experience with system use over time; however, we were able to indirectly capture these changes from the qualitative responses. Third, this a mixed-method study guided by a theoretical framework, as reflected in the design of both quantitative and qualitative parts. Quantitative and qualitative data were innovatively triangulated with the Pilar Integration Process. However, we did not adequately excavate the possible contradiction between quantitative and qualitative results, which prevented us from drawing further inferences from the possible discordance.

Conclusions

This study builds on the existing information technology acceptance models to show that the promotion of a NIS requires effective collaboration between end users, administrators and technical personnel to enhance system usability and user experience. Optimization of system structure and content and tackling technical issues should be an ongoing process with continuous input from both nursing and technical staff. Nurse leaders can exert a positive impact on nurses to facilitate system implementation by fostering relationships across disciplines. Frontline nurses should be given the opportunity to get involved in system development so that nursing practice will truly benefit from NIS. Future HIT use training should include clinical simulation sessions to better engage and truly benefit nurses. Aligning system implementation with broader organizational goals and support from top management is needed to smooth the transition process and achieve organizational level system adoption.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. The datasets supporting the conclusions of this article are included within the article and its additional files.

Abbreviations

CDS:

Clinical decision support

EHR:

Electronic health record

HIT:

Health information technology

HOT-fit:

Human, organization and technology-fit

NIS:

Nursing information system

PIP:

Pillar integration process

SNL:

Standardized nursing language

References

  1. Jones D, Lunney M, Keenan G, Moorhead S. Standardized nursing languages: essential for the nursing workforce. Annu Rev Nurs Res. 2010;28:253.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Westra BL, Latimer GE, Matney SA, Park JI, Sensmeier J, Simpson RL, Swanson MJ, Warren JJ, Delaney CW. A national action plan for sharable and comparable nursing data to support practice and translational research for transforming health care. J Am Med Inform Assoc. 2015;22(3):600.

    Article  PubMed  Google Scholar 

  3. Ortiz DR, Maia FOM, Ortiz DCF, Peres HHC, Sousa PAF. Computerized clinical decision support system utilization in nursing: a scoping review protocol. JBI Database Syst Rev Implement Rep. 2017;15(11):2638.

    Article  Google Scholar 

  4. Peltonen LM, Topaz M, Ronquillo C, Pruinelli L, Sarmiento RF, Badger MK, Ali S, Lewis A, Georgsson M, Jeon E, et al. Nursing informatics research priorities for the future: recommendations from an international survey. Stud Health Technol Inform. 2016;225:222.

    PubMed  Google Scholar 

  5. Moore R, Stonham G. Electronic care records give nurses access to information that can transform patient care. Nurs Times. 2010;106(13):8.

    PubMed  Google Scholar 

  6. Michel-Verkerke MB. Nursing information system: a relevant substitute of the paper nursing record. Stud Health Technol Inform. 2011;169:339.

    PubMed  Google Scholar 

  7. Rouleau G, Gagnon MP, Côté J, Payne-Gagnon J, Hudson E, Dubois CA. Impact of information and communication technologies on nursing care: results of an overview of systematic reviews. J Med Internet Res. 2017;19(4):e122.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Baumann LA, Baker J, Elshaug AG. The impact of electronic health record systems on clinical documentation times: a systematic review. Health Policy. 2018;122(8):827.

    Article  PubMed  Google Scholar 

  9. Ammenwerth E. Technology acceptance models in health informatics: TAM and UTAUT. Stud Health Technol Inform. 2019;263:64.

    PubMed  Google Scholar 

  10. Farzandipour M, Nabovati E, Tadayon H, Sadeqi Jabali M. Identification and classification of usability problems in a nursing information system: a heuristic evaluation. Comput Inform Nurs. 2021;40(2):121–30.

    PubMed  Google Scholar 

  11. Melnick ER, West CP, Nath B, Cipriano PF, Peterson C, Satele DV, Shanafelt T, Dyrbye LN. The association between perceived electronic health record usability and professional burnout among US nurses. J Am Med Inform Assoc. 2021;28(8):1632.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Boonstra A, Jonker TL, van Offenbeek MAG, Vos JFJ. Persisting workarounds in electronic health record system use: types, risks and benefits. BMC Med Inform Decis Mak. 2021;21(1):183.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Brunner J, Chuang E, Goldzweig C, Cain CL, Sugar C, Yano EM. User-centered design to improve clinical decision support in primary care. Int J Med Inform. 2017;104:56.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform. 2008;77(6):386.

    Article  PubMed  Google Scholar 

  15. Kilsdonk E, Peute LW, Jaspers MW. Factors influencing implementation success of guideline-based clinical decision support systems: a systematic review and gaps analysis. Int J Med Inform. 2017;98:56.

    Article  PubMed  CAS  Google Scholar 

  16. Kouri A, Yamada J, Lam Shin Cheung J, Van de Velde S, Gupta S. Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake. Implement Sci. 2022;17(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Lee CL, Lin WT, Lin SY. Perceptions and experiences of hospital nurses during transition to an electronic handover informatics system. Comput Inform Nurs. 2019;37(11):591.

    Article  PubMed  Google Scholar 

  18. Joukes E, de Keizer NF, de Bruijne MC, Abu-Hanna A, Cornet R. Impact of electronic versus paper-based recording before EHR implementation on health care professionals’ perceptions of EHR use, data quality, and data reuse. Appl Clin Inform. 2019;10(2):199.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Sandelowski M. Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health. 2000;23(3):246.

    Article  PubMed  CAS  Google Scholar 

  20. O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92.

    Article  PubMed  Google Scholar 

  21. Müller-Staub M, de Graaf-Waar H, Paans W. An internationally consented standard for nursing process-clinical decision support systems in electronic health records. Comput Inform Nurs. 2016;34(11):493.

    Article  PubMed  Google Scholar 

  22. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42(5):533–44.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Wen X, Feng XQ, Zheng E. Reliability and validity test of Chinese version of nursing information system using behavior scale for nurses. Chin Nurs Res. 2017;31(7):810.

    Google Scholar 

  24. Zhao YX, Gu Y, Zhang XB, Liu F, Wan YM. Developed the clinical nursing information system effectiveness evaluation scale based on the new D&M model and conducted reliability and validity evaluation. Chin J Pract Nurs. 2020;36(7):544.

    Google Scholar 

  25. Martikainen S, Korpela M, Tiihonen T. User participation in healthcare IT development: a developers’ viewpoint in Finland. Int J Med Inform. 2014;83(3):189.

    Article  PubMed  Google Scholar 

  26. Martikainen S, Kaipio J, Lääveri T. End-user participation in health information systems (HIS) development: physicians’ and nurses’ experiences. Int J Med Inform. 2020;137:104117.

    Article  PubMed  Google Scholar 

  27. Morse JM, Barrett M, Mayan M, Olson K, Spiers J. Verification strategies for establishing reliability and validity in qualitative research. Int J Qual Methods. 2002;1(2):13.

    Article  Google Scholar 

  28. Johnson RE, Grove AL, Clarke A. Pillar integration process: a joint display technique to integrate data in mixed methods research. J Mixed Methods Res. 2017;13(3):301.

    Article  Google Scholar 

  29. Birt L, Scott S, Cavers D, Campbell C, Walter F. member checking: A tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. 2016;26(13):1802.

    Article  PubMed  Google Scholar 

  30. Korstjens I, Moser A. Series: practical guidance to qualitative research. Part 4: trustworthiness and publishing. Eur J Gen Pract. 2018;24(1):120.

    Article  PubMed  Google Scholar 

  31. De Groot K, Triemstra M, Paans W, Francke AL. Quality criteria, instruments, and requirements for nursing documentation: a systematic review of systematic reviews. J Adv Nurs. 2019;75(7):1379.

    Article  PubMed  Google Scholar 

  32. Roman LC, Ancker JS, Johnson SB, Senathirajah Y. Navigation in the electronic health record: a review of the safety and usability literature. J Biomed Inform. 2017;67:69.

    Article  PubMed  Google Scholar 

  33. Varpio L, Rashotte J, Day K, King J, Kuziemsky C, Parush A. The EHR and building the patient’s story: a qualitative investigation of how EHR use obstructs a vital clinical activity. Int J Med Inform. 2015;84(12):1019.

    Article  PubMed  Google Scholar 

  34. Saranto K, Kinnunen UM, Kivekäs E, Lappalainen AM, Liljamo P, Rajalahti E, Hyppönen H. Impacts of structuring nursing records: a systematic review. Scand J Caring Sci. 2014;28(4):629.

    Article  PubMed  Google Scholar 

  35. Stanhope V, Matthews EB. Delivering person-centered care with an electronic health record. BMC Med Inform Decis Mak. 2019;19(1):168.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Keenan G, Yakel E, Dunn Lopez K, Tschannen D, Ford YB. Challenges to nurses’ efforts of retrieving, documenting, and communicating patient care information. J Am Med Inform Assoc. 2013;20(2):245.

    Article  PubMed  Google Scholar 

  37. Joukes E, Cornet R, de Bruijne MC, de Keizer NF. Eliciting end-user expectations to guide the implementation process of a new electronic health record: a case study using concept mapping. Int J Med Inform. 2016;87:111.

    Article  PubMed  Google Scholar 

  38. Rojas CL, Seckman CA. The informatics nurse specialist role in electronic health record usability evaluation. Comput Inform Nurs. 2014;32(5):214.

    Article  PubMed  Google Scholar 

  39. Cresswell K, Morrison Z, Crowe S, Robertson A, Sheikh A. Anything but engaged: user involvement in the context of a national electronic health record implementation. Inform Prim Care. 2011;19(4):191.

    PubMed  Google Scholar 

  40. Skyttberg N, Vicente J, Chen R, Blomqvist H, Koch S. How to improve vital sign data quality for use in clinical decision support systems? A qualitative study in nine Swedish emergency departments. BMC Med Inform Decis Mak. 2016;16:61.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Laukka E, Huhtakangas M, Heponiemi T, Kanste O. Identifying the roles of healthcare leaders in HIT implementation: a scoping review of the quantitative and qualitative evidence. Int J Environ Res Public Health. 2020;17(8):2865.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Ratwani R, Fairbanks T, Savage E, Adams K, Wittie M, Boone E, Hayden A, Barnes J, Hettinger Z, Gettinger A. Mind the gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform. 2016;7(4):1069.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Varsi C, Ekstedt M, Gammon D, Børøsund E, Ruland CM. Middle managers’ experiences and role in implementing an interactive tailored patient assessment ehealth intervention in clinical practice. Comput Inform Nurs. 2015;33(6):249.

    Article  PubMed  Google Scholar 

  44. Jedwab RM, Hutchinson AM, Manias E, Calvo RA, Dobroff N, Glozier N, Redley B. Nurse motivation, engagement and well-being before an electronic medical record system implementation: a mixed methods study. Int J Environ Res Public Health. 2021;18(5):2726.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Melin-Johansson C, Palmqvist R, Rönnberg L. Clinical intuition in the nursing process and decision-making-a mixed-studies review. J Clin Nurs. 2017;26(23–24):3936.

    Article  PubMed  Google Scholar 

  46. Holmström IK, Kaminsky E, Lindberg Y, Spangler D, Winblad U. Registered nurses’ experiences of using a clinical decision support system for triage of emergency calls: a qualitative interview study. J Adv Nurs. 2020;76(11):3104.

    Article  PubMed  Google Scholar 

  47. Barken TL, Thygesen E, Söderhamn U. Advancing beyond the system: telemedicine nurses’ clinical reasoning using a computerised decision support system for patients with COPD—an ethnographic study. BMC Med Inform Decis Mak. 2017;17(1):181.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Wouters LT, Zwart DL, Erkelens DC, Huijsmans M, Hoes AW, Damoiseaux RA, Rutten FH, de Groot E. Tinkering and overruling the computer decision support system: working strategies of telephone triage nurses who assess the urgency of callers suspected of having an acute cardiac event. J Clin Nurs. 2020;29(7–8):1175.

    Article  PubMed  Google Scholar 

  49. Walsh K, Wroe C. Mobilising computable biomedical knowledge: challenges for clinical decision support from a medical knowledge provider. BMJ Health Care Inform. 2020;27(2):e100121.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors deeply appreciate nurses for taking their time to participate in the online survey and personal in-depth interviews, which contributed greatly to this work.

Funding

This study was funded by the Nursing research subproject of Chinese Health Talent Training Project (2019-HLYJ-010).

Author information

Authors and Affiliations

  1. School of Nursing, Fudan University, Shanghai, China

    Zhai Yue & Yuxia Zhang

  2. Department of Nursing, Zhongshan Hospital, Fudan University, Shanghai, China

    Zhai Yue, Yu Zhenghong, Qi Zhang, Wei Qin & Yuxia Zhang

  3. Department of Information, Zhongshan Hospital, Fudan University, Shanghai, China

    Yang Chun

Authors
  1. Zhai Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Zhenghong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yang Chun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuxia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The study design and methodology were discussed and approved by all authors. Subject recruitment, data collection and interpretation of findings was carried out by ZY and ZQ. YZH also helped with subject recruitment. YZH and YC revised the manuscript for important intellectual content. The research process was under the supervision of ZYX. ZY drafted the article. QW and ZYX critically reviewed and the manuscript and approved the final version of the article. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yuxia Zhang.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the ethics committee of Zhongshan Hospital, Fudan University (2021-481R). Prior to the data collection, the primary investigator explained the purpose of the study and data confidentiality issues to the study subjects. Informed consent was obtained from all the nurses participating in the study. This study was performed in accordance with the Declaration of Helsinki and was reported in accordance with the Good Reporting of a Mixed Methods Study (GRAMMS) checklist.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Good Reporting of A Mixed Methods Study (GRAMMS) checklist.

Additional file 2: Fig. S1

. An illustration of the modules and operation procedures of Care Direct in client management.

Additional file 3

. The topic guide (final version).

Additional file 4

. Table S1. Responses to the NIS use behavior scale (n = 324). Table S2. Responses to the NIS effectiveness scale (n = 324).

Additional file 5

. Table S3. Upgrade frequency of each module of the system from January 2021 to April 2022.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhai, Y., Yu, Z., Zhang, Q. et al. Transition to a new nursing information system embedded with clinical decision support: a mixed-method study using the HOT-fit framework. BMC Med Inform Decis Mak 22, 310 (2022). https://doi.org/10.1186/s12911-022-02041-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-022-02041-y

Keywords

  • Hospital information system
  • Clinical decision support systems
  • Usability
  • User-centered design
  • Nursing informatics
  • Qualitative research

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us