A survey of factors affecting clinician acceptance of clinical decision support

In response in part to the Institute of Medicine's reports "Crossing the Quality Chasm" [1] and "To Err is Human" [2], and the American Medical Informatics Association's position paper on the use of clinical decision support in electronic prescribing [3] there is increased pressure to implement state of the art clinical information systems (CISs) with real-time clinical decision support capabilities. Unfortunately, several recent reports have documented that a disturbingly high percentage (i.e., 54 – 91%) of real-time clinical decision support suggestions are being over-ridden, or ignored, by clinicians [4–6].

Granted, there are certainly cases in which "overriding" the computer-generated alert is the correct action on the part of the clinician including: the benefits of the action outweigh the risks, there is no good alternative, this is an "expected" side-effect of a particular therapy or procedure, or that the medication was previously or currently tolerated, to name just a few. On the other hand in all of the studies cited above, in which the clinicians overrode a very high percentage of all alerts, the authors found that in almost all cases the computer-generated alerts were "true-positives" meaning that most observers would consider the clinicians actions to ignore the alert to be contrary to "best clinical practices".

We are in the process of designing, implementing, and evaluating many new clinical decision support features and interventions [7, 8]. Based on our knowledge of the literature and extensive clinical informatics experience, we recognize that there are a myriad of factors associated with clinicians' refusal to accept, or follow, computer-generated, care suggestions based on clinical guidelines including lack of: awareness that the guideline even exists, familiarity with the recommendation, agreement with the suggestion, belief that they could even perform the expected behavior (often referred to as: self-efficacy), belief that the expected improvement in outcome will occur, ability to overcome the inertia of previous practice, and the existence of external barriers to the performance of the recommendations (e.g., no time or no reminder system) [9]. In addition to these mostly internal, provider-related factors, there are also many computer-related hypotheses for why clinicians refuse to follow these suggestions including: failure to provide patient-specific information (which was not shown to be a factor in this study) [10], specific aspects of the human-computer interaction surrounding the presentation of the reminders, for example, presenting fully-completed orders that follow the guideline on the same screen as the reminder, rather than placing them "one click away", using a distinctive color scheme to "highlight" the recommendation, disabling the escape key which made it more difficult to override the suggestion, setting the default value of the suggestion to "order" rather than "not to order", and presenting the same reminder over and over to all clinicians who viewed a particular patient's data (i.e., until the suggestion was accepted) [11]. While we were not able to follow all of these "best practices" for the design of interactive clinical decision support features due to inherent limitations of our commercially available EMR and some institutional resistance on the part of clinical and information system administrators, we are doing our best to remove as many potential barriers as possible.

We hypothesized that there are other factors that may account for clinicians' refusal to follow computer-generated clinical suggestions with the intended action of removing, or at least reducing, as many of the identified barriers as we can. Therefore, we undertook this study to begin exploring these other potential factors affecting clinician acceptance of clinical decision support at the point of care.

Following Institutional Review Board (IRB) approval, we sent an anonymous, 2-page survey (see Additional file 1) along with a cover letter that introduced the project, to all 225 adult primary care physicians (i.e., all physician members of the Internal Medicine and Family Medicine departments) via interdepartmental mail. Completed surveys were returned over a 5-week period. No follow-up attempts were made to increase the survey return rate out of respect for clinicians' valuable time and attention and fear of compromising our ability to use these same clinicians as subjects in future research studies.

The questions were designed to identify potential factors affecting CDS system acceptance as well as specific system usage patterns on the part of clinicians. In addition, we asked a few questions to help us identify potential differences in CDS system acceptance that might be explained by basic demographic characteristics. Most of the questions could be answered using a 5-point Likert scale where 1 represented "Never"; 2 = "Rarely"; 3 = "Some of the time"; 4 = "Most of the time"; 5 = "Always". A few questions asked clinicians to answer questions based on whether they were "More", "Less", or "Equally" likely to behave in a certain way. Finally, we asked a few "open-ended" questions to allow respondents to answer in their own words. These responses were then coded for further analysis.

Following manual entry of the survey results, we calculated the mean, standard deviation, and range of responses for each numeric answer. For each open-ended, text-based answer field, we coded and tabulated the results according to frequency of occurrence. For several key variables we calculated cross tabulations in an attempt to identify interesting correlations between variables.

Perhaps the most interesting finding was that specific patient characteristics were associated with the decision to accept or ignore various clinical decision support features. Specifically, clinicians indicated they were more willing to accept clinical decision support when the patient was elderly, had multiple medications or chronic conditions, and much less likely to accept the computer suggestions when the patient was presenting for an acute condition. Since all of these patient characteristics are known prior to the visit, it is possible that the organization could eliminate the presentation of the alerts in the urgent care setting. However while possible, there is uncertainty as to whether such an action would be appropriate. For example, many clinician leaders feel that every opportunity should be taken to provide preventive procedures or health maintenance screening tests to our membership, including when patients present with acute conditions. On the other hand, some front line clinicians feel that it is not their responsibility to address these deferrable events during visits in the urgent care clinic.

The other interesting finding was how often clinicians reported that they were behind schedule and that these clinicians were much less likely to accept alert suggestions when they were behind schedule. This raises the difficult question of whether we should "turn off" some or all of the alert types when the clinicians are significantly behind schedule. While one could argue that this would save the clinicians a little time, one could also argue that during these times of extraordinary stress is exactly when the alerts are the most needed.

It was difficult, if not impossible, for us to separate the differences in responses due to gender from that of the number of years with Kaiser Permanente since those clinicians who have been with Kaiser Permanente for 1–3 years are predominately female (73%) while those with more than 15 years of service are 90% male. Therefore, we caution readers not to make too much of the fact that either gender or years of service with Kaiser Permanente has significant affects on clinicians acceptance of various clinical decision support features since both of these factors clearly have significant affects on many aspects of one's life.

The decision of whether to include estimates of the amount of time a clinician is behind schedule with the provider or environmental factors may be controversial since clinicians within our organization do not have any management authority over their office or medical assistant staffing. Therefore, clinicians have little control over how their day is structured and may get behind schedule due to many different reasons including a) his/her particular practice style that does not mesh well with the 20-minute visit schedule and the unpredictable nature of clinical encounters coupled with the increasing demands and expectations for care and service during these encounters, b) inefficiencies on the part of the clinic staff in preparing the examination rooms and bringing the patients in from the waiting room and taking their vital signs, etc., c) patients often arrive late for their scheduled visit and are "worked" into the remaining schedule for the day, or d) and perhaps more likely, a combination of one or more of these and other less well identified factors.

Regardless of the appropriate categorization of this factor, or why the clinicians are behind schedule, there was a strong correlation between those physicians who reported being behind schedule and several key practices that our organization is trying to promote, including accepting safety alerts, looking up information on specific medications, and using on-line reference information. Of specific interest to our organization's leadership was the fact that the clinicians who reported being behind schedule more often also reported that they had less access to computers in their examination rooms. This information coupled with another recent report on computers in the exam room that associated improvements in patient satisfaction in three distinct areas a) satisfaction with visit components; (b) comprehension of the visit; and (c) perceptions of the physician's use of the computer [21] has helped justify a recent decision to equip all of our examination rooms with computers.

As a result of this study and several other co-occurring efforts, the clinical and administrative leadership of Kaiser Permanente, Northwest, temporarily turned off all health maintenance reminders and re-examined all of their logic and trigger points. After several weeks the new, and improved, alerts were re-initiated region-wide.

Based on the results of this survey we believe that patient and environmental characteristics are among the most important factors affecting the stated acceptance of various clinical decision support suggestions. There was no indication that any of the commonly thought of clinician characteristics such as age, years with the organization, or gender have much to do with the decision to accept or ignore various clinical decision support features. The fact that clinicians reported being behind schedule so often coupled with the finding that they stated they were much less likely to accept clinical decision support alerts when behind must be examined more closely. Finally, even though clinicians do not accept all of the clinical decision support that the system presents, overall clinicians indicate the benefits of these alerts still outweigh their costs.