Improving the nursing informatics competency of critical care nurses: results of an interventional study

Background Along with growth and development of health information technology (HIT), nursing informatics (NI) is becoming a fundamental part of all domains of nursing practice especially in critical care settings. Nurses at different levels of the nursing continuum are expected to equip with NI competency for providing patient-centered evidence-based care. Therefore, improvement of the nurses’ NI competency through educational programs is important and necessary for effective using of HIT. This study aimed to evaluate the impact of a training program on NI competency of critical care nurses. Methods In this interventional study, 60 nurses working in critical care units at hospitals aliated with a large University of Medical Sciences in the southeast of Iran were randomly and equally assigned to the control and intervention groups. NI competency was trained to the intervention group in a three-day workshop. Data were collected using demographic questionnaire and Nursing Informatics Competency Assessment Tool (NICAT) before and one month after the intervention. Results In the pretest stage, both intervention and control groups were at the “competent” level in terms of the NI competency, and no signicant difference was observed between them ( p =0.65). However, in the posttest, the NI competency and its dimensions signicantly increased in the intervention group with a large effect size compared with the control group ( p = 0.001). This difference showed that the intervention group achieved the “procient” level in posttest stage. Conclusions The improved scores of NI competency and its dimensions after using the training program implied the effectiveness of this method in enhancing the NI competency of nurses working in the critical care units. The higher eciency of the training program can be determined by its application in diverse domains of nursing practice. The project is a fundamental for improving nurses’ NI competency through continuous educational programs in Iran, other cultures and contexts. a total score 30 a score between 31-59 represents advanced beginner, a score between 60-89 indicates competent, a score from 90 to 119 offers procient, and a score between 120-150 is considered as an expert. Benner’s model outlines nursing skill acquisition from novice, advanced beginner, competent, procient, to expert. The novice learns the tools and thinks like a nurse. The advanced beginner nurse applies knowledge and clinical judgments in delivering care. A competent nurse can use the tools familiar the A procient can all system

with one's HIT, intent to reduce clinical working hours and quit the job or leave current practice [11]. Other studies suggest that the negative impacts are as a result of poorly designed user interface, disrupted work ow, communication breakdown and increased workload [9], low reliability and poor user-friendliness of HIT, time pressure and psychological distress [12].
However, healthcare has not been isolated from the digitalization. Digitalization re-engineers organizational processes and routines, encourages nurses especially those working in critical care settings to be equipped with NI competency and gets accustomed to change [12,13]. NI competency is de ned as the nurses' knowledge, skills, and attitudes to gather, store, retrieve, process and use information in nursing care [14]. The NI competency ranges from simple clinical skills to complex application-based knowledge [15]. According to American Nurses Association (ANA), various nursing informatics activities are prescribed within four levels of nursing practice, including beginner nurse, experienced nurse, informatics nurse specialist, and informatics innovator [16]. The Technology Informatics Guiding Education Reform (TIGER) Initiative in 2009 put recommendations for NI competency of the bedside nurses and integration of the NI into nursing education and development of all nurses in every role such as beside nurse, nurse leader and nurse educators. The recommendations are in three categories, including computer literacy, information literacy, and informatics management skills [5,14]. HIT would be ineffective in the healthcare settings without the existence of NI competency, imposing above mentioned negative impacts [15]. NI was considered as a requirement for EBP. When nurses have high levels of NI competency, they are better ready to EBP. Several HITs including programmed decision-support systems have been established from EBP [17].
Nurses working in critical care units need NI competency much higher than other nursing groups because of the sensitivity of working in critical care settings. Patients in critical settings need sophisticated hightech equipment, and specialized clinicians continuously monitor them. The patient safety and patient outcomes are more vital in such settings than other units [14,[18][19][20]. The literature review has recognized the gap in NI competency required for nurses in different work environments and critical care settings. The results suggested that more training courses are necessary to increase NI competency and to better meet the competency requirements of nursing profession [12,18,[21][22][23]. Furthermore, researchers emphasized that low NI competency of nurses were associated with negative impacts of HIT in patient care. Poor NI competency of both newly graduated and experienced nurses not only puts patients at risk for suboptimal care, but also places additional stresses on nurses who are expected to demonstrate improved patient outcomes by using HIT [24]. Therefore, nurses' NI competency in the delivery of patient care should be improved with continuous learning. Without educational interventions, nurses are not able to effectively use HIT in practice [12,13,25,26]. In addition, newly graduated nurses in clinical settings reported that the NI education they received was insu cient, their instructors demonstrated incomplete understanding of NI, and the skills taught in these classes were not transferred well to the workplace [24]. Researchers have reported that nursing schools lack standardization to teach NI competency required for nursing students. The integration of NI competency into nursing education will require nursing curricula reform and development of innovative educational programs [25].
Finally, HIT is now available, and strategies such as research, educational interventions, and modules should be added to professional development of nurses. Moreover, the strategies can support nurses to adapt with HIT in practice and reduce negative impacts of HIT. This process can be accomplished through developing and evaluating effective educational programs in different settings [7,11]. Few studies implemented a training program to improve NI competency among nurses and reported that nurses' NI competency increased after the educational intervention [4,27,28]. Furthermore, the research team, clinical nurses and educators in universities, believe that the NI competency depends on organizational context such as hospital status, size, and resource. Regarding the importance of the improving NI competency in critical care nurses and the scarcity of the related studies in the context of Iran, this study aimed to evaluate the impact of a training program on NI competency of nurses working in critical care units.

Study Design and Settings
The present study was an educational intervention to improve NI competency of nurses in critical care units. The study was conducted using a pretest-posttest design with the intervention and control groups.
The study settings were 18 critical care units, including10 intensive care units (ICUs), 6 coronary care units (CCUs), and 2 dialysis wards selected from three educational hospitals a liated with Kerman University of Medical Sciences in the southeast of Iran. In the organizational chart of the Iranian hospitals, ICUs, CCUs, and dialysis are classi ed into critical care units [29], and nurses working in these units have the same bene ts. According to the nursing curricula, nursing students spend critical courses in these settings. In addition, the distribution of nurses is similar in all Iranian hospitals. The majority of nurses have bachelor's degrees and are recruited in all hospitals with the same pattern. There is no registered nurse (RN) and informatics nurse specialist in the study settings. According to the nurse manager, nurses working in critical settings are not xed and have rotational shifts in many wards.
Experts in nursing education in Iran offered integration of NI competency and research in nursing in undergraduate nursing curricula in 2010. However, at the time of this study there were only 1.5 credits of research in nursing (34 hours, in third semester) and one credit of information technology in nursing (26 hours, in the rst semester) into the undergraduate nursing curricula. Although basic skills research and the use of computers may be taught in courses, perhaps the demands of the nursing curriculum do not allow for more education. Consequently, there are gaps in integration of the NI competency into nursing education. Furthermore, nurses graduated before 2010 have not received formal training on NI competency in nursing undergraduate curricula.
It is noteworthy that authorities of Kerman University of Medical Sciences select appropriate e-Health applications for the hospitals. For example, Hospital Information System (HIS) is currently implemented in nursing stations, pharmacy, laboratories, radiology, outpatient departments, and inpatient wards, etc. Hospitals have implemented HIS to automate hospital management tasks such as documentation of nursing care, cost management, resource management, medication management, o ce automation, accounting, nurses and physician performance assessment, and patient registration, etc. Following the implementation of HIS, nurses received regular training informally and formally. However, there is no formal education to improve nurses' NI competency. The rapidly growing areas of e-health have increased the demand for nurses with high NI competency in hospitals. This indicates that nurses require more continuing education to better meet requirements of their job.

Sampling
The study population included all nurses (N = 330) working in the critical care settings mentioned above at the time of data collection. The sample size was calculated using the sample size formula regarding α = 0.05, test power of 80%, and a large effect size (Cohen d = 0.7); the required sample size was 27 participants for each group (54 participants are needed for the two groups). The researchers added six more samples to avoid the effect of possible dropout in the study. Twenty nurses (10 nurses for the intervention and 10 nurses for control group) were selected from each of the three hospitals using the random number table. In total, 60 nurses were assigned into the intervention (n = 30) and control groups (n = 30). The inclusion criteria were nurses with a bachelor's degree, at least six months of work experience in the critical care units as well as not being scheduled for a clinical shift on the days of the workshop. The exclusion criteria included having one absence during the course and incomplete questionnaire. None of the nurses were excluded based on of inclusion criteria. Finally, 60 questionnaires were collected and analyzed.

Instruments
The instrument used in this study consisted of two questionnaires. The rst one was about the nurses' demographic and professional information including gender, marital status, age, work experience, work experience in critical care settings, work position, shift work, history of attendance at research, information literacy skills, information-seeking skills, and computer skills training ( Table 2).
The second was Nursing Informatics Competency Assessment Tool (NICAT) developed by Rahman [5] in the US in 2015 based on ANA standards (2008), TIGER recommendations (2009), and Benner's Dreyfus model of skill acquisition (1984). NICAT can be utilized for evaluation of educational and training programs. This tool has the potential to assess and evaluate all nurses during the new hire process, orientation, education, and clinical informatics system implementation. NICAT may not be applicable to bedside nurses with advanced degrees. The NICAT assesses IT competency with 30 items in three dimensions: 1. Computer literacy (10 items, items 1 to 10) shows the psychomotor skills to use computer tools, as well as knowledge of basic hardware and software functionality; these are all required for effective bedside nursing.
2. Informatics literacy (13 items, items 11 to 23) are the nurses' abilities to recognize the need for information, and to retrieve, evaluate, and use information for patient care appropriately.
shows the value of information system in improving patient safety, quality, and outcome.
The NICAT is scored on a ve-point Likert scale ranging from one to ve: not competent (1 score), somewhat competent (2 scores), competent (3 scores), very competent (4 scores), and expert (5 scores). The overall score range of the instrument is 30-150. The higher the scores the higher the IT competency; a total score of 30 shows novice, a score between 31-59 represents advanced beginner, a score between 60-89 indicates competent, a score from 90 to 119 offers pro cient, and a score between 120-150 is considered as an expert. Benner's model outlines nursing skill acquisition from novice, advanced beginner, competent, pro cient, to expert. The novice learns the tools and thinks like a nurse. The advanced beginner nurse applies knowledge and clinical judgments in delivering care. A competent nurse can use the tools and reason through problems familiar with the patient population. A pro cient nurse can use all tools, has experience to recognize a pattern of patient conditions, and anticipate patient needs. An expert nurse, in addition to all the above, has the vision to compose a team, look beyond the current moment, analyze system issues, and seek continuous improvements in practice.
Three groups including the nursing informatics team, the department of education, practice, and research, and the hospital clinical outcome department validated the NICAT. Clarity, readability, accuracy, question sequence, comprehensiveness of the questions, relevance, and appropriateness of the item were rated during the content validation and expert review. Abd El (2017) assessed the nursing informatics competencies of the critical care nurses. Researcher determined content validity with the help of information technology expert and con rmed the reliability using internal consistency. Cronbach's alpha coe cient was 0.824 for each dimension and 0.846 for all items indicating su cient internal consistency [14].
The authors of this study translated and validated NICAT in Iran. For cross-cultural comparison of the translation, the original NICAT was accurately translated into Persian (forward translation). A pro cient English translator did the backward translation of Persian version. Then, the translated version was matched with the original version. A number of nurses' perception of the items was investigated to check the face validity of NICAT. Two medical informatics specialists and eight nursing faculty members con rmed the content validity of the Persian version of NICAT. Content validity was enhanced by using experts' direct quotes, face-to-face discussions, and comments. Furthermore, 30 nurses participated in the pilot-test of the questionnaire and the Cronbach's alpha coe cient was used to assess its reliability (α= 0.95).

Data collection
The aim was to assess the effect of an educational program on the critical care nurses' NI competency.
Data were collected using an anonymous, self-reported, and structured questionnaire. To collect data, the rst researcher referred to the study settings in different shift works, distributed the questionnaires among nurses of the intervention and control groups in the pretest (before workshop) and posttest stages (one month after the workshop). She also trained participants how to ll out the questionnaires. To attain the highest response rate, the researcher spent appropriate time on data collection and coordination of workshop time with nurses in intervention group. She established a friendly relationship with the participants and determined a deadline to deliver completed questionnaires. Moreover, she contacted and reminded the intervention group to attend the workshop in the scheduled time. It should be noted that all of the participants completed questionnaires during paid work time simultaneously with routine or traditional training programs in hospitals, except that the intervention group was provided with additional materials derived from workshop and the control group received no this educational program. In other words, the two study groups had equivalent conditions for work duties and attendance in other educational programs in hospitals. However, to increase internal validity of the study, researchers coordinated with educational supervisors in the hospitals and monitored the study conditions thoroughly to ensure that the intervention and control groups were identical in all aspects, except attending our educational program.
Intervention procedure A three-day workshop was held in three weeks (an eight-hour session on a day) in the informatics center located in Deputy of research and technology of Kerman University of Medical Sciences, with an adequate number of personal computers and internet access. The intervention group was divided into two groups of 15 people to increase the opportunity of engagement in the workshop for the participants. Therefore, all the workshops were held with the same structure and topics in six days.
To prepare and develop the workshop content , the researchers reviewed the literature to nd necessary NI competencies for clinical nurses, and their education needs in NI competency [4,14,15,24,[30][31][32]. They discussed the extracted topics to achieve a consensus concerning goals and contents and teaching strategies. The literature revealed three key areas in NI competencies: computer literacy, informatics literacy, and information management skills applicable to the nursing care. These areas met and complied with ANA standards (2008) and TIGER recommendations (2009). The researchers employed the TIGER NI competencies as a guide and selected essential NI competencies. Nurses with a standard and consistent set of competencies can use HER better. The researchers also consulted with a multidisciplinary team, including three medical informatics specialists, two nursing faculty members, an expert in the eld of medical education, three educational supervisors, and two critical care nurses. The team members provided their experiences and perspectives on required training and competencies of clinical nurses concerning nursing informatics in current practice, as well as teaching-learning activities and available educational programs on health informatics in study settings. In this step, the topics were developed using focus group discussion and then the team reviewed them for suitability, feasibility, applicability and relevance to the nursing work ow. Each item of the developed content was frequently reviewed and revised. After coming to an agreement on the NI competencies, the researcher drafted the content of workshop.
Two informatics medical specialists and six nursing faculty members who were not in the research group

Statistical analysis
The data were analyzed in SPSS 21 using descriptive statistics (frequency, percentage, mean and standard deviation) and inferential statistics (independent samples t-test, paired t-test, chi square test, and the analysis of covariance). The Kolmogorov-Smirnov test showed that the data followed a normal distribution. The signi cance level was considered ≤0.05. signi cant difference was found between the intervention and control groups in demographic and professional information (Table 2). Moreover, no signi cant difference was observed in the mean scores of NI competency between the study groups at the pretest stage. Independent samples t-test indicated homogeneity of the participants in the two study groups at the baseline (Table 3).

Comparison of changes NI competency in the intervention group
The results of table 3 show that the total score of NI competency in the intervention group was at the "competent" level in pretest. The paired t-test revealed that the total score of NI competency in the intervention group increased statistically signi cant and achieved the "pro cient" . These signi cant differences show that the training program improved the scores of NI competency and its dimensions in the intervention group from the "competent" to "pro cient" levels with a very large effect size. The highest mean difference was associated with the informatics literacy dimension (mean difference = 15.26), while the lowest mean difference was associated with the informatics management skills dimension (mean difference = 8.00). The mean differences show that training program had the highest impact on the informatics literacy dimension and the lowest impact on the informatics management skills.

Comparison of changes of NI competency in the control group
A comparison of the pretest and posttest scores in the control group revealed no signi cant improvement in NI competency and its dimensions; the scores were at the "competent" level in both stages (t = -1.39, p = 0.25). In Table 4 we used covariance analysis test to control the impact of pretest on the NI competency of nurses. Concerning the pretest effect, the results showed a statistically signi cant difference between the control and intervention groups in the total posttest scores of NI competency and its dimensions. These results also are consistent with the results of Table 3.

Discussion Principal results
This study evaluated the effectiveness of a training program on NI competency of the critical care nurses in Iran. The results indicated that the program signi cantly improved NI competency and its dimensions in the intervention group compared with the control group, with a large effect size. The NI competency of the intervention group promoted from "competent" level in pretest to "pro cient" level after the training program. The results of several studies in different countries reported that nurses' NI competency was at the competent level before attending any educational intervention [33][34][35][36][37]. However, other studies indicated that nurses' NI competency was within the range of low level of competency [15,38,39]. Some other studies reported the nurses' NI competency at the expert and pro cient levels [2,40,41].
Several studies in different countries evaluated impact of the training programs on NI competency of the participants and reported its improvement [4,27,30,42]. Pereira et al. in Spain used three technological tools (power point, open meetings and babelium) to improve NI competency of the nursing students. Researchers also used peer-, self-and teacher assessment to assess the project effectiveness. They reported that NI competency of the students improved at the end of project, and more than 53% of students reported the development of their creativity. There was an acceptable correspondence between self-and peer-assessment [43]. In another study, researchers designed a blended course to develop nursing students' informatics competency. This project successfully improved nursing students' informatics competency. The majority of students perceived positively toward the teaching strategy of blendedlearning mode that was used throughout the course [31]. Similar Rajalahti et al. in Finland found that nurses and nurse educators who completed an educational project improved their NI competency. In postproject, nurses were better prepared and con dent to use the electronic health record (EHR), make better clinical decisions and provide better patient care. The nurse educators also better mastered EBP and use of nursing process models in their work [28] Iranian studies also reported a signi cant increase in NI competency of the faculty members [44] and nurses in ICUs after attending the educational program compared with the control group [18]. Similar results may be due to application of a similar training program and advancement of HIT in clinical setting and everyday life, which has prompted individuals and nurses to acquire and enhance their NI competency. Moreover, current nurses are novices in technology and are known as digital immigrants. Therefore, they have to learn and adopt many aspects of the new technology. If a digital immigrant really wants to reach digital natives, they will have to change. Therefore, nurses should be equipped with NI competencies in formal and informal training inside and outside the organization. On the other hand, managers and policymakers in the healthcare system have emphasized on this issue and strived to provide the necessary infrastructure to empower the staff nursing.
However, a study examined the effect of training on NI competency, which does not support our study. The researchers reported no signi cant difference between the intervention and control groups in nurses' NI competency after the intervention. As they reported, both groups improved their information seeking skills in the posttest [45].
The results indicated that the highest mean difference in the intervention group was associated with the informatics literacy dimension and the lowest mean difference was associated with the informatics management skills dimension. A study also showed that at the post-test stage, the level of nurses' NI competency was promoted in the intervention group, and educational intervention had the most impact on the dimension of informatics literacy [28]. Conversely, in another study which assessed effectiveness of a project on NI competency nurses, the lowest mean difference was related to the informatics literacy, while the highest mean difference was attributed to information management [4].
Our results showed that NI competency and all its dimensions remained at the "competent" level in the control group and no signi cant difference was observed between the pretest and posttest scores. It can be said that nurses' NI competency is a determinant factor in successful application of clinical information systems. The current conditions and the hospitals emphasis on application of HIT in different clinical settings required the nurses to seek minimum NI competency. Karimi et al. [46] as well as Raei and Haseli [44] showed that the scores of NI competency in the control group did not have a signi cant difference before and after the educational course. Esfandani et al. investigated critical care nurses and reported that the scores of information seeking skills were low in the control group with no signi cant difference in their scores before and after the intervention [18]. Conversely, a study also reported that the posttest score of information skills increased in the control group [45].
The discrepancy between results of the above-mentioned studies and the present study can be due the differences in community, sampling, randomization and matching of the groups, educational backgrounds, clinical and experiential learning environment, participants' previous skills, nursing education preparation levels, study design and educational content, data collection tools and conditions. For example, recommendations of the TIGER Initiative were fundamental for the development of data collection tool and educational content of our project.

Limitations
This study had some limitations that need to be addressed. First, this study included a limited training time, increasing the training time should be allowed for more topics and coverage of NI competency during the project. Multiple class offerings will allow more nurses to enroll in such project and thereby increase the generalizability and power of the results. Second, we used self-report tool to assess NI competency; therefore, the evaluation has been conducted at a low level of Kirkpatrick's model such as reaction, learning. Future research may look at the effectiveness of training program at higher levels such as in behavior and results or outcomes. We suggest the evaluation of educational interventions related to NI competency at all levels of Kirkpatrick's model to provide the evidence required to inform future initiatives.
Third, evaluation of NI competency might have been in uenced by the social desirability bias inherent in the self-assessment and self-report methodology applied in this study, because participants tend to overrate their levels of NI competency; therefore, the data might not re ect the actual level of nurses' NI competency. Future studies can be performed using blended methods of competency evaluation such as of Kirkpatrick's four-level model and 360 degree method to help determine the actual NI competency. Evaluation of NI competency is strengthened through a hybridization of different approaches, which is uniquely responsive and suitable for evaluating competency. The Kirkpatrick model has contributed to the model popularity and perseverance as the dominant model of competency evaluation. In addition, the comparative results of the 360-degree evaluation can help nurses better understand their competencies, and they are more motivated to advance professionally in the organization. Moreover, this evaluation will help the nurse manger better understand the training and development needs of nurses and plan for them. Finally, data collection was conducted one month after the intervention. Follow-ups with 3-6 month intervals are recommended to have results that are more accurate, compare the results to determine the long-term impact of training, and to assess the effect of educational courses on NI competency.

Conclusions
This project could signi cantly improve critical care nurses' NI competency and its dimensions. Recently, the need to continue NI competency development is recognized by national and international health care system and also the nursing community. It highlights the importance of initiative approaches to promote NI competency of the nurses in all elds. The results suggest that nurse managers, decision-makers, nurse educators, and authorities of clinical settings should organize appropriate interventions and training programs with the help of informatics specialists to improve nurses' NI competency particularly in the eld of information management skills. These strategies may include assessment of training needs, design and modi cation of curriculum for integrating topics like NI competency. The training process of development and the content used in this project can also be utilized as a sample for designing training programs for nurses and other health professionals. Persian version of the NICAT that was validated and translated in this study can be bene cial in NI competency assessment and examination of its status and gaps of nurses' competency in Iranian healthcare. In addition, nurse managers should highlight the importance of well-competent and e cient nurses in providing patient-centered evidence-based care. In this case, they should establish required resources to promote NI competency of nurses. It is essential to strengthen the collaboration and interaction between the universities and hospitals because nursing schools are in the best position to develop future nurses' NI competency required to use in the HIT. With integrating NI competency into nursing education and using of blended courses to develop nursing students' NI competency, future nurses will be able to effectively balance HIT with clinical demands. Further detailed research is also recommended to explore NI competency in healthcare and evaluate effectiveness of different approaches and models to enhance the NI competency of nurses in Iran and other cultures and contexts.