User acceptance of a picture archiving and communication system (PACS) in a Saudi Arabian hospital radiology department

The technology acceptance model (TAM)

Derived in its original form from the theory of reasoned action (TRA) [27], the TAM explains how users accept and use a technology (Figure 1). Developed by Davis [21], the model regards two acceptance measures as the primary determinants of behavioral intention to use technology: (1) Perceived usefulness (PU), defined as "the degree to which a person believes that using a particular system would enhance his or her job performance”; and (2) Perceived ease of use (PEU), defined as "the degree to which a person believes that using a particular system would be free from effort" ([21], p.985). The model proposes that PEU has a causal effect on PU, and that each of these has an influence on the user’s attitude towards use; both PU and attitude toward use influence behavioral intention (acceptance), which in turn influences usage [17].

The original TAM was extended (to create TAM2) [28] by adding some additional drivers of PU and PEU, including theoretical constructs of social influence processes and cognitive instrumental processes. There have also been attempts to enhance the explanatory power of the TAM [15, 19] by adding antecedent, mediating, and moderating variables to explain acceptance behavior in different settings. Lee et al. [16], in their review of the TAM, found that more than 20 different factors have been introduced into various versions of the model, including constructs representing education level, computer expertise, voluntariness, fit between user and system design features, management support, and socio-demographic variables. A later extension of the TAM was formulated by Venkatesh et al. [19] as the Unified Theory of Acceptance and Use of Technology (UTAUT). The UTAUT incorporates four direct determinants of behavioral intention (acceptance), which are performance expectancy (equivalent to PU), effort expectancy (equivalent to PEU), social influence, and facilitating conditions. The model also contains four moderating variables (experience, voluntariness, gender, and age) that have been found to be significant in some studies of technology acceptance.

Specific to the healthcare context, Holden and Karsh [22] found in their overview of healthcare-related TAM studies that almost all such studies had added variables to the basic TAM in an effort to better understand the antecedents of health IT acceptance or usage behavior. These variations have been added “to account for the complexity of healthcare’s socio-technical systems” ([22], p.166). Topics in healthcare have included the acceptance of telemedicine, electronic health records systems, computerized physician order entry systems, mobile healthcare systems, and PACSs [15, 22].

Study objectives

This study uses a variant of the TAM of Davis [21] to assess the acceptance of the host PACS by radiology staff at KAMC, Riyadh, Saudi Arabia. The specific objectives of the study are: 1) To assess radiology user acceptance (behavior) regarding the PACS in the radiology department at KAMC, in Riyadh, Saudi Arabia; 2) To determine the extent to which three constructs (perceived usefulness, perceived ease of use, and change) influence user acceptance; and 3) To determine whether user acceptance is explained by socio-demographic variables.

The instrument

The study instrument (questionnaire) consisted of two parts. Part 'A' included questions about respondent characteristics including age, gender, and job title, and about respondents’ length of experience with PACS and how frequently they used the system in their daily work, following Venkatesh et al. [19] and Duyck et al. [6]. The five variables measured and the response options were: (1) Age (open); (2) Gender (male, female); (3) Job title (consultant, radiologist, resident, technologist, other); (4) PACS experience (≤ 1 year, > 1 year); (5) PACS use frequency (in the past but not now, occasionally, frequently, always).

Part 'B' of the questionnaire included three sections. The first section was a use scale (measuring both PU and PEU) that contained ten statements, with statements 1–6 reflecting perceived usefulness and statements 7–10 reflecting ease of use. The participants graded their responses on each item using a five-point likert scale: strongly disagree = 1, disagree = 2, neither agree/disagree = 3, agree = 4 and strongly agree = 5. The items (statements) were: (1) Using the PACS enables me to accomplish tasks more quickly. (2) Using the PACS has improved the quality of my work in providing better patient care. (3) Using the PACS has increased my productivity. (4) Using the PACS has enhanced my effectiveness on the job. (5) Using the PACS has made my job easier to perform. (6) Using the PACS has given me greater control over my work schedule. (7) Learning to use the PACS has been easy for me. (8) My interaction with the PACS has been clear. (9) My interaction with the PACS has been understandable. (10) It is easy to become skillful at using the PACS.

The second section was a change construct which contained four statements in response to the question “How has the PACS made your job”? Customized five-point likert scale responses were used for the four statements, as follows: (1) More difficult = 1, difficult = 2, neutral = 3, easy = 4, easier = 5. (2) Not interesting = 1, less interesting = 2, neutral = 3, interesting = 4, more interesting = 5. (3) Extremely more stressful = 1, more stressful = 2, neutral = 3, stressful = 4, less stressful = 5. (4) Extremely less pleasant = 1, less pleasant = 2, neutral = 3, pleasant = 4, more pleasant = 5.

The third section was a behavioral (acceptance) construct which contained nine statements. The behavioral construct was used to represent the user’s behavior with respect to the PACS. The participants graded their responses on each item using a five-point likert scale: never = 1, occasionally = 2, fairly frequently = 3, very frequently = 4, always = 5. Five of the statements were “negative” (statements 2, 3, 5, 7, and 9) and were reversed in their scale values. The items (statements) were: (1) Admiring the PACS. (2) Difficulty learning to use the PACS. (3) Complaining about the PACS. (4) A high level of proficiency learning to use the PACS. (5) Lack of cooperation with PACS personnel. (6) Using the PACS appropriately. (7) The PACS slowing work performance. (8) Enjoying working on the PACS. (9) Bypassing the PACS, i.e., using film-based procedures to perform tasks.

After formulating the items of the constructs in accordance with the literature, two steps were followed to increase the content validity of the questionnaire. First, the comments and suggestions of PACS administrators were taken into account. Second, a pilot study was performed on a small sample of consultants, radiologists, residents, and technologists who were using PACS in King Khaled University Hospital in Riyadh, after which minor adjustments were made to some aspects of the questionnaire.

Data analysis

Of the 120 questionnaires distributed, 89 (74 %) were returned completed. Questionnaire data were analyzed with SPSS (Statistical Package for Social Science). Descriptive analysis used in the study included frequencies, percentages, means, and standard deviations. Inferential analysis included independent and paired sample t-tests, ANOVA, and stepwise multiple regression. The significance level used for the inferential statistics was 0.05. A sample mean approach was used to replace missing values.

Questionnaire reliability

The reliability of the questionnaire was measured using Cronbach’s alpha. Values of this measure were 0.88 for the use (PU and PEU) scale, 0.72 for the change construct, and 0.74 for the acceptance construct. Cronbach's alpha should not be less than 0.70 for newly developed instruments [29].

Respondent characteristics

The age of participants in the study ranged from 22 to 69 years old with an average of 34.5 years. Fewer than half the participants were aged 30 or younger (Table 1). Seventeen (19.1 %) of the participants each had one year or less experience with PACS and 72 participants (80.9 %) each had more than one year experience.

Use, change, and behavior constructs

Concerning the change construct, the respondents reported that the PACS has made their job easy (a mean rating of 4.47) (Table 2). They also found that the PACS has made their job interesting (mean rating of 4.25), pleasant (4.16), and less stressful (4.13).

The behavioral construct was the dependent variable used to measure the users’ acceptance levels (Table 2). The overall behavior rating of the PACS in the radiology department in KAMC was 3.86, which suggests that the PACS users showed a fairly high level of acceptance of the system. There were more item scores <4 in the behavior construct (5/9) than in the other constructs (all items in which scored >4). The item “a high level of proficiency for learning to use the PACS” yielded by some margin the lowest individual rating.

Factors influencing the overall level of acceptance

SMR was also used to determine which particular items in each scale had a significant relationship to the overall acceptance of the PACS. The result of the SMR revealed that three variables (PACS enhancing effectiveness on the job (for PU), learning to use PACS has been easy (for PEU), and PACS has made the job more pleasant (for the change construct)) were significantly related to overall acceptance.

Acceptance level and socio-demographic variables

Factors influencing user acceptance

All three antecedent constructs—PU, PEU, and change—were found to be statistically significant predictors of IT user acceptance. PU was the most important individual predictor (38 % of variation explained), followed by the change scale (4.5 %) and PEU (3.5 %). In their overview of TAM studies in healthcare, Holden and Karsh [22] discovered 16 studies that had tested the relationship between PU and the outcome variable, all of which were significant. In contrast, 7 out of 13 studies that tested the influence of PEU found it to be significantly related to the outcome variable. The apparent lesser influence of PEU on user IT behavior found for PACS users at KAMC is therefore consistent with existing studies of acceptance of various technologies in healthcare settings. In this context, this finding has been explained in previous studies by the above-average intelligence of physicians and technologists [15].

Specifically regarding PACS technology acceptance, Duyck et al. [6] using the UTAUT model found that PU was a strong determinant of physicians’ and radiologists’ behavioral intention to use PACS (overall model R² adj. of 33 %, 21 months post-implementation), but PEU was insignificant. Duyck et al. [23] also using UTAUT discovered that PU was the best predictor of acceptance, with PEU also being a significant predictor, and the overall model R² adj. was 37 % (all PACS-using physicians but not radiologists, 21 months post-implementation). In a related study, Duyck et al. [26] achieved an R² adj. of 48 % for the UTAUT studying radiologists and technologists. Pare et al. [24] used the Information Systems Success model, which differs from the TAM and its variants in both the suite of independent variables used and in the ultimate dependent variable. However, in their study, 24 months after PACS implementation, PU was a significant influence on “user satisfaction” for clinicians but not for radiologists or technologists, and PEU was significant for radiologists but not for technologists or clinicians. Pynoo et al. [25] used the UTAUT and path analysis (including indirect effects on acceptance via PU and PEU), and discovered that PU influenced physician acceptance at both 4 and 16 months post-implementation, whereas PEU was insignificant at both times (although PEU was found to be important at the actual time of implementation). The multiple correlation coefficient for acceptance was 0.60 at 16 months, meaning about 36 % of the variation in acceptance was explained by the overall model.

When the present findings regarding antecedents of acceptance are also considered, it would appear that PU is much more important than PEU for PACS acceptance. The percentage of variance in user acceptance explained by the various models used in PACS studies (including this investigation) lies in a fairly tight range of 33-48 %. Although this level of explanation is quite reasonable, it seems that there is a ceiling of explanation for PACS acceptance using TAMs (including variants/extensions). Some predictors evidently remain unknown or incompletely quantified given that more than half of the variance in acceptance remains unexplained. The change construct employed in this study showed some promise as a predictor of IT acceptance, and its use could be considered in future studies.

Influence of professional user groups, gender, age, and experience

Duyck et al. [6], using the UTAUT, surveyed the acceptance of a PACS by radiologists and hospital physicians in a Belgian hospital. User acceptance ratings were higher for radiologists than for physicians, apparently because the former group experienced more benefits of the PACS and also had to use PACS more often through the working day. Duyck et al. [26] found that radiologists had higher ratings on PU, PEU, and acceptance than did technologists. However, Pare et al. [24] found similar acceptance ratings between radiologists, clinicians, and technologists. The present results show that there was no overall difference between professional user groups in PACS acceptance levels, although technologists had the lowest level of acceptance out of the five user groups surveyed. Technologists may have lower acceptance ratings as they are involved in PACS in a more limited way than are clinicians and radiologists [24], who additionally use the system for diagnostic and interventional purposes; alternatively, the training/familiarization program used (although extensive) may not have catered to their particular needs as successfully as it did for clinicians and radiologists.

Although various papers have found that age and gender are significant factors in attitudes toward IT in general, the review of Ward et al. [11] specifically on healthcare-related studies found no strong evidence for either an age or gender influence. Duyck et al. [6] found that male users rated PU more highly than did females, but it should be noted that in the UTAUT, gender is treated as a moderating influence and has no direct effect on acceptance. This study found there was no significant difference between men and women in PACS acceptance levels, or any influence of age, supporting the finding of Pare et al. [24].

Some healthcare studies have revealed positive relationships between IT acceptance and the years of experience in computer use [11]. Duyck at al. [23], using the UTAUT, discovered that the length of experience with PACS led to an overall improved attitude toward PACS. At KAMC, the length of experience using the technology had no direct influence on staff acceptance. This may reflect the particular professional groups surveyed, being radiologists and referring physicians in Duyck et al. [23] and additionally including technologists in this study, or it may be due to differences in the training/familiarization programs run at each hospital. The contrasting findings are unlikely to be due to PACS system architecture and features, as the hospitals in the respective studies both use General Electric PACSs. The very recent study of Pynoo et al. [25] found that although there was no main effect of experience on acceptance, it did show interaction effects with PU at 16 months post-implementation.

Overall, the findings in the literature (e.g. [11]) regarding professional grouping, gender, age, and experience with respect to acceptance of both PACSs and other health-related information technologies are mixed, and none of the characteristics appears to be a consistent indicator of users’ acceptance. This may partly reflect the differences between studies in the formulations of models used [15].

Successful PACS adoption

There are many factors that influence whether an IT system, including a PACS, is successfully adopted into a healthcare organization, including organizational (managerial and structural) factors, technological factors, and behavioral factors [3, 30]. User acceptance is a major factor influencing successful PACS adoption, as indicated by previous investigations (e.g., [19, 21, 27, 30]). If the technology is not used, or under-used, or mis-used, then many of the tangible and intangible benefits of PACSs [1, 2, 4, 5] relating to organizational efficiency, financial considerations, and improved patient care may not be fully realized. Resistance to IT among clinicians is well known ([3, 10, 30, 31]), and such resistance has led in too many instances to failed, prolonged, or suboptimal implementation of new systems [10]. Resistance can be traced to various problems, including poor input into managing organizational change, insufficient pre-implementation training, installation issues, and personal-level factors including physicians’ time for adopting new technology, skepticism of the reliability and benefits of new technology, and users’ IT familiarity and ability [3, 10, 15]. PACS users at KAMC benefitted from a long (14-month) period of familiarization and training prior to the “go live” date. It is likely that this was an important facet of enhancing the subsequent levels of staff satisfaction and acceptance of the PACS, and hence was an important part of successful system adoption (e.g., [7]). Ayal and Seidmann [5], in their study of PACS implementation at a hospital in Rochester, New York, found that that physician satisfaction with the PACS service post-implementation was highly correlated with satisfaction with the implementation process, suggesting that sufficient and proper training is critical in increasing the level of users’ acceptance.

The two contrasting case studies of Pare and Trudel [3] exemplify the importance of the role of staff behavior and acceptance in an institution’s successful (or otherwise) adoption of a PACS. In respect of such resistance/acceptance, the findings of this study should be regarded as very encouraging. The overall level of acceptance for PACS users at KAMC is high, as indicated by the high mean values for the model constructs (PU 4.33/5, PEU 4.15/5, and acceptance 3.86/5). Previous studies of PACS acceptance also report positive attitudes from surveyed users. Duyck et al. [6] reported 86 % of responding physicians as giving positive scores (i.e. 4/7 or better) for PU, and 98 % for PACS acceptance. Pynoo et al. [25] recorded mean scores for PU, PEU, and acceptance of 4.70/7, 5.06/7, and 6.29/7, respectively, for physicians. The present study’s mean PU and PEU scores are higher (proportionally) than reported by Pynoo et al., but the acceptance score is lower. The lower acceptance score is probably due to the technologists surveyed here, who reported lower acceptance ratings than other user groups including radiologists and clinicians.

Cultural beliefs and values are considered by some researchers to play a part in the adoption, acceptance, and use of technology [32–34]. Straube et al. [34] refer to technical, organizational, and human constraints in Arab countries concerning IT transfer, which lead to resistance to technology. Based on arguments concerning culture-specific beliefs and values in Arab countries, it might be expected that the surveyed users would exhibit lower levels of PU and acceptance than their western counterparts, and that women would show lower levels than men. Although this study did not specifically test for a cultural influence, the high levels of PU and of user acceptance and the lack of consistent differences across different types of user (e.g., male v female) suggest that acceptance of PACS does not contain a cultural dimension in this case, supporting the result of Baker et al. [35] for knowledge workers in Saudi Arabia. The current finding may reflect the fact that the radiology department staff surveyed comprised a mix of cultures and nationalities, including Americans, Australians, and Europeans, as well as Saudi nationals: the increasing global mobility of healthcare professionals may reduce/remove the influence of host cultural elements in user acceptance of PACSs, as well as that of other health IT systems. The finding may also be due to the PACS being supplied by a multinational vendor (GE), rather than by a local vendor.