Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Impact of computerized physician order entry (CPOE) system on the outcome of critically ill adult patients: a before-after study

  • Hasan M Al-Dorzi1, 2,
  • Hani M Tamim1, 2,
  • Antoine Cherfan1,
  • Mohamad A Hassan1,
  • Saadi Taher1 and
  • Yaseen M Arabi1, 2Email author
BMC Medical Informatics and Decision Making201111:71

DOI: 10.1186/1472-6947-11-71

Received: 26 March 2011

Accepted: 19 November 2011

Published: 19 November 2011

Abstract

Background

Computerized physician order entry (CPOE) systems are recommended to improve patient safety and outcomes. However, their effectiveness has been questioned. Our objective was to evaluate the impact of CPOE implementation on the outcome of critically ill patients.

Methods

This was an observational before-after study carried out in a 21-bed medical and surgical intensive care unit (ICU) of a tertiary care center. It included all patients admitted to the ICU in the 24 months pre- and 12 months post-CPOE (Misys®) implementation. Data were extracted from a prospectively collected ICU database and included: demographics, Acute Physiology and Chronic Health Evaluation (APACHE) II score, admission diagnosis and comorbid conditions. Outcomes compared in different pre- and post-CPOE periods included: ICU and hospital mortality, duration of mechanical ventilation, and ICU and hospital length of stay. These outcomes were also compared in selected high risk subgroups of patients (age 12-17 years, traumatic brain injury, admission diagnosis of sepsis and admission APACHE II > 23). Multivariate analysis was used to adjust for imbalances in baseline characteristics and selected clinically relevant variables.

Results

There were 1638 and 898 patients admitted to the ICU in the specified pre- and post-CPOE periods, respectively (age = 52 ± 22 vs. 52 ± 21 years, p = 0.74; APACHE II = 24 ± 9 vs. 24 ± 10, p = 0.83). During these periods, there were no differences in ICU (adjusted odds ratio (aOR) 0.98, 95% confidence interval [CI] 0.7-1.3) and in hospital mortality (aOR 1.00, 95% CI 0.8-1.3). CPOE implementation was associated with similar duration of mechanical ventilation and of stay in the ICU and hospital. There was no increased mortality or stay in the high risk subgroups after CPOE implementation.

Conclusions

The implementation of CPOE in an adult medical surgical ICU resulted in no improvement in patient outcomes in the immediate phase and up to 12 months after implementation.

Keywords

Intensive care unit critical illness CPOE safety management mortality morbidity

Background

Computerized Physician Order Entry (CPOE) is the process of entering medication orders and other physician's instructions electronically using a computer-based system to ensure standardized, legible and complete orders [1]. Its implementation has been recommended to improve patient safety and outcomes [2, 3] primarily by reducing medication errors [4] that usually arise from faulty prescriptions [5]. However, CPOE itself can lead to new types of errors [68], which may be detrimental to patients. This may be more pronounced in critically ill patients, a more vulnerable population due to the severity of their illness, the complexity of interventions done and the diversity of equipments used in the intensive care unit (ICU). The evidence for the CPOE effectiveness in improving the outcomes of critically ill patients is scarce and remains controversial. Two before-after studies in pediatric ICUs showed no significant change in mortality after CPOE implementation [9, 10]. Del Beccaro et al observed a statistically insignificant decrease in mortality from 4.2% to 3.46% in the 13-month pre- and post-CPOE implementation [9]. Keene et al found no change in mortality post-CPOE after adjusting for all covariates (OR, 0.71; 95% CI, 0.32-1.57)[10]. On the other hand, Han and colleagues observed an increase in mortality from 2.8% to 6.6% during the 5-month period after the initiation of a CPOE system at a pediatric hospital with a tertiary care ICU [11]. Literature lacks information on the impact of CPOE implementation on the outcomes of adult critically ill patients. The objective of our study, conducted as a quality improvement project, was to evaluate the outcomes of critically ill patients after CPOE implementation in an adult ICU.

Methods

Setting

The study was conducted at a 21-bed medical surgical closed ICU in a 900-bed teaching tertiary care hospital, King Abdulaziz Medical City-Riyadh, in Saudi Arabia. It was staffed by board certified intensivists 24 hours per days, 7 days a week as described in a study from our ICU elsewhere [12] with rotating residents from different specialties. Nursing staff were multinational with patient-to-nurse ratio of 1:1. The hospital was accredited by Joint Commission International.

Study Design

The study was conducted as an observational before-after study. Data were extracted from a prospectively collected electronic ICU database. We included all consecutive patients admitted to the adult ICU (age ≥ 12 years) in the 24 months pre- and 12 months post-CPOE implementation. We excluded patients with brain death admitted as potential organ donors. For patients who had multiple admissions to the ICU within the same hospitalization, only the first admission was considered in the analysis. The institutional review board of King Abdulaziz Medical City approved the study and due to its observational nature, informed consent was waived.

CPOE Implementation

Before CPOE implementation, physicians wrote medication orders on paper; nurses then faxed the orders to the ICU satellite pharmacy; the pharmacist would screen the order, make necessary adjustments after consulting with the prescribing physician if needed, transcribe the order and dispense the medication. The adult ICU was the first area in the hospital to implement CPOE. Other hospital areas continued to use the traditional paper-based order writing during the study period. The used system was a product from Misys®, which was later acquired by QuadraMed, and allowed medication and nonmedication orders as well as access to the patient's chart, laboratory and radiological data. It included a basic real-time clinical decision support system so that important interactions of prescribed medications, dose checking and allergy notification appeared at the time of prescription as pop-ups. In the year preceding CPOE implementation, a multidisciplinary team prepared electronic order sets and protocols. The team used the existing paper-based order sets including ICU admission orders, traumatic brain injury protocol and antimicrobial therapy order set and converted them with some modifications into CPOE order sets. Some of these order sets, such as the traumatic brain injury protocol, implemented since 2001, had been studied in our ICU and been shown to improve outcomes [13]. This was followed by training of ICU physicians, pharmacists and nurses on the use of CPOE functions. Following the "Go Live" on December 4, 2006, physicians directly entered all medications and most nonmedication orders, such as specific nursing instructions, requisition of laboratory tests and radiological examinations and consultations of physiotherapy and nutrition services, from any computer terminal inside or outside the unit. Pop-up alerts appeared on the screen and required verification before order acceptance. Pharmacists screened electronic prescriptions and advised physicians in case of errors. Nurses checked their bedside computer terminals for new and scheduled orders. Some orders, such as diet and blood transfusions, continued on papers. Technical support and guidance were provided 24 hours a day, 7 days a week for 2 weeks from the hospital's information technology specialists, then as needed. To address the observed difficulties and limitations and enhance the CPOE system, orders and order sets were revised upon agreement of physicians, nurses, clinical pharmacists and medical informatics specialists. Such revisions included transforming complex intravenous orders that required more than 3 steps into single-step orders, optimizing doses and frequency of intravenous antibiotics and simplifying radiological and laboratory orders.

Measurements and outcomes

The following data were extracted from the ICU database: age, gender, Acute Physiology and Chronic Health Evaluation (APACHE) II score [14], source of admission (ward, emergency department, operating room and other hospital), main reason for ICU admission (as per APACHE II definitions), cardiac arrest and sepsis as a reason for admission, the presence of chronic health illness (as defined by APACHE II system), admission Glasgow Coma Scale, admission creatinine and International Normalized Ratio (INR), requirement for vasopressors within the first 24 hours after admission and requirement for mechanical ventilation during ICU stay. The primary outcome was ICU mortality. Secondary outcomes were: duration of mechanical ventilation, ICU and hospital length of stay and hospital mortality. Comparisons were made for different time periods: one, three and 24 months before implementation compared with one, three and 12 months post-implementation. The reason for selecting these periods was to evaluate CPOE effect in the immediate and more extended period of implementation and also to control possible seasonal variation of different outcomes. In addition, we evaluated the impact of CPOE in selected groups of patients, whose outcome was thought to be particularly affected by CPOE. These included: age 12-17 years since Han et al reported an increased mortality in the pediatric patients post-CPOE implementation [11], diagnosis of sepsis because outcomes of these patients can be affected by timely interventions such as timely fluid and antimicrobial therapy [15] and traumatic brain injury as such patients may benefit from protocolized management. Additionally, because patients with higher severity of illness were thought to be more vulnerable, we evaluated the effect of CPOE in patients with high APACHE II score,≥ the median score which was 23.

Statistical analysis

SAS software (version 9.0; SAS Institute, Cary, NC) was used to analyze data. Continuous variables were reported as means with standard deviation and categorical variables as absolute and relative frequencies. The Chi-squared test was used to assess the differences between categorical variables and the student t-test was used for the analysis of differences in the means of continuous variables. To examine the association between CPOE implementation and various outcomes, we used stepwise multivariate logistic regression analysis. Independent variables entered in the model were important factors that clinically affect outcomes and baselines characteristics that were significantly different in the 24 month pre- and 12 months post-CPOE in addition to CPOE implementation status. The following variables were entered in the model: age, gender, APACHE II, source of admission, non-operative trauma and post-operative trauma as main reason for admission, cardiac arrest as a reason for admission, chronic health illness (liver, cardiovascular and renal), creatinine, INR, vasopressors use, and mechanical ventilation. Results were reported as adjusted odds ratios (aOR) for categorical outcomes (ICU and hospital mortality) and beta (β) coefficient for continuous outcomes (duration of mechanical ventilation, length of stay in the ICU and the hospital) with 95% confidence intervals (CI). This analysis was performed for the whole cohort and for the selected subgroups of patients. At α = 0.05, our study had an 80% power to detect a 4.8% decrease or a 5.2% increase in mortality after CPOE implementation using the observed mortality (23.4%) in pre-CPOE implementation period and the number of patients in the pre-CPOE (n = 1638) and post-CPOE (n = 898) periods. A P-value < 0.05 was considered statistically significant.

Results

General characteristics of all patients

In the 24 months pre-CPOE implementation (December 4, 2004 to December 3, 2006), 1638 patients were admitted to the ICU (13104 patient-days) compared to 898 patients (7274 patient-days) in the 12 months post-CPOE implementation (December 4, 2006 to December 3, 2007). Table 1 describes the characteristics of patients admitted to the ICU in the following periods: 24 vs. 12 months, 12 vs. 12 months, 3 vs. 3 months and 1 vs. 1 month pre- and post-CPOE implementation. There were no significant differences in age, admission APACHE II score, admission Glasgow Coma Scale, creatinine and INR in these pre- and post-implementation periods. Compared to the 24 months pre-CPOE implementation, the percentage of female patients and postoperative trauma admissions were higher in the following 12 months (39% vs. 35%, p = 0.04). Also there were significant imbalances between the two groups in these variables: source of admission before ICU, chronic health illnesses and cardiac arrest as a reason for ICU admission. In addition, vasopressors (50% vs. 36%, p < 0.0001) and mechanical ventilation (77% vs. 73%, p = 0.004) were used more commonly in the 24 months pre-CPOE compared to 12 month post-CPOE implementation.
Table 1

Comparison of baseline characteristics of admissions to the intensive care unit pre- and post-computerized physician order entry (CPOE) implementation in four different periods: 24 months pre- vs. 12 months post-CPOE, 12 months pre- vs. 12 months post-CPOE, three months pre- vs. three months post-CPOE vs. one month pre- and one month post-CPOE.

Variable

24 months pre N = 1638

12 months post N = 898

P-value

12 months pre N = 813

12 months post N = 898

P-value

3-months pre N = 192

3-months post N = 239

P-value

1-month pre N = 69

1-month post N = 75

P-value

Age in years, mean ± SD

52.1 ± 21.5

52.4 ± 21.1

0.74

51.7 ± 21.7

52.4 ± 21.1

0.51

50.1 ± 21

53.6 ± 21.1

0.09

48.0 ± 22.2

50.3 ± 21.0

0.52

Female gender, N (%)

567 (34.7)

349 (38.9)

0.04

284 (34.9)

349 (38.9)

0.09

64 (33.3)

97 (40.6)

0.12

28 (40.6)

32 (42.7)

0.80

APACHE II, mean ± SD

24.1 ± 9.4

24.2 ± 9.7

0.83

24.7 ± 9.6

24.2 ± 9.7

0.36

25.1 ± 10.2

25.1 ± 10.4

1.0

25.3 ± 9.3

24.7 ± 10.9

0.75

Source of admission, N (%)

   Emergency department

450 (27.5)

179 (19.9)

 

223 (27.4)

179 (19.9)

 

64 (33.3)

50 (20.9)

 

25 (36.2)

22 (29.3)

 

   Ward

668 40.8)

331 (36.9)

0.0001

326 (40.1)

331 (36.9)

< 0.0001

75 (39.1)

101(42.3)

0.02

28 (40.6)

22 (29.3)

0.14

   Operating room

418 (25.5)

335 (39.8)

 

214 (26.3)

335 (39.8)

 

43 (22.4)

73 (30.5)

 

14 (20.3)

27 (36.0)

 

   Other hospital

100 (6.1)

53 (5.9)

 

43 (5.3)

53 (5.9)

 

10 (5.2)

15 (6.3)

 

2 (2.9)

4 (5.3)

 

Main reason for admission, N (%)

   Respiratory

304 (18.6)

150 (16.7)

0.24

136 (16.7)

150 (16.7)

0.99

44 (22.9)

46 (19.3)

0.35

22 (31.9)

16 (21.3)

0.15

   Cardiovascular

514 (31.4)

258 (28.7)

0.16

269 (33.1)

258 (28.7)

0.05

47 (24.5)

70 (29.3)

0.26

11 (15.9)

21 (28)

0.08

   Neurologic

119 (7.3)

66 (7.4)

0.94

60 (7.4)

66 (7.4)

0.98

22 (11.5)

23 (9.6)

0.54

10 (14.5)

2 (2.7)

0.01

   Other medical

83 (5.1)

35 (3.9)

0.18

35 (4.3)

35 (3.9)

0.67

7 (3.7)

8 (3.4)

0.87

3 (4.4)

1 (1.3)

0.27

   Non-operative trauma

229 (14)

83 (9.2)

0.0005

108 (13.3)

83 (9.2)

0.008

30 (15.6)

23 (9.6)

0.06

9 (13)

9 (12)

0.85

   Post-operative trauma

387 (23.7)

306 (34.1)

< 0.0001

205(25.2)

306 (34.1)

< 0.0001

42 (21.9)

69 (28.9)

0.10

14 (20.3)

26 (34.7)

0.05

   Cardiac arrest as admission diagnosis

135 (8.3)

45 (5.0)

0.002

76 (9.4)

45 (5.0)

0.0005

17 (8.9)

13 (5.4)

0.17

6(8.7)

2 (2.7)

0.11

   Sepsis on admission

420 (25.7)

251 (28.0)

0.21

241 (29.6)

251 (28.0)

0.21

51 (26.6)

67 (28.0)

0.73

16 (23.2)

19 (25.3)

0.76

Chronic health illnesses, N (%)

   Chronic liver disease

259 (15.8)

97 (10.8)

0.0005

117 (14.4)

97 (10.8)

0.03

26 (13.5)

32 (13.4)

0.96

10 (14.5)

7 (9.3)

0.34

   Chronic cardiovascular

451 (27.6)

289 (32.2)

0.01

250 (30.8)

289 (32.2)

0.52

59 (30.7)

114 (47.7)

0.0004

24 (34.8)

31 (41.3)

0.42

   Chronic respiratory

413 (25.2)

224 (24.9)

0.87

259 (31.9)

224 (24.9)

0.0015

57 (29.7)

94 (39.3)

0.04

27 (39.1)

26 (34.7)

0.58

   Chronic renal disease

349 (21.3)

157 (17.5)

0.02

210 (25.8)

157 (17.5)

< .0001

54 (28.1)

64 926.8)

0.76

19 (27.5)

23 (30.7)

0.68

   Immunocompromised

228 (13.9)

124 (13.8)

0.93

117 (14.4)

124 (13.8)

0.93

19 (9.9)

42 (17.6)

0.02

7 (10.1)

11 (14.7)

0.41

Glasgow coma scale

9.2 ± 4.3

8.9 ± 4.5

0.17

8.7 ± 4.5

9.2 ± 4.3

0.01

8.4 ± 4.4

9.0 ± 4.2

0.15

8.4 ± 4.0

9.3 ± 4.0

0.22

Admission creatinine in μmol/L

163 ± 161

158 ± 146

0.44

166 ± 164

158 ± 146

0.37

195 ± 214

162 ± 142

0.08

162 ± 170

167 ± 165

0.87

Admission INR

1.6 ± 1.1

1.5 ± 0.8

0.10

1.6 ± 1.0

1.5 ± 0.8

0.24

1.5 ± 0.8

1.4 ± 1.0

0.57

1.4 ± 0.8

1.8 ± 1.5

0.09

Vasopressor use*

822 (50.2)

327 (36.4)

< 0.0001

373 (45.9)

327 (36.4)

< 0.0001

92 (47.9)

97 (40.6)

0.13

28 (40.6)

33 (44.0)

0.68

Mechanically ventilated, No.%

1265 (77.3)

662 (73.7)

0.04

636 (78.2)

662 (73.7)

0.03

152 (79.2)

179 (74.9)

0.29

55 (79.7)

62 (82.7)

0.65

ICU: intensive care unit; CPOE: computerized physician order entry; APACHE: Acute Physiology and Chronic Health Evaluation; INR: International Normalized Ratio.

* Vasopressors use was defined as use of dopamine > 5 mcg/Kg/min or any dose of norepinephrine, epinephrine, phenylephrine or vasopressin.

Outcomes of patients pre- and post-CPOE implementation

Table 2 compares the outcomes of all patients in the cohort in relationship to CPOE implementation. The implementation of CPOE was not associated with a statistically significant change in ICU mortality when comparing the 24 months pre-CPOE to the 12 months post-CPOE (aOR 0.98, 95% CI 0.7-1.3), the 12 months pre-CPOE to the 12 months post-CPOE (aOR 0.96, 95% CI 0.7-1.3), the 3 months pre-CPOE to the 3 months post-CPOE (aOR 0.83, 95% CI 0.5-1.5), the 1 month pre-CPOE to the 1 month post-CPOE implementation (aOR 1.17, 95% CI 0.4-3.2). Similarly, CPOE implementation was not associated with significant differences in hospital mortality, duration of mechanical ventilation and of length of stay in the ICU and hospital in any of the four periods.
Table 2

Outcomes of patients in different pre- and post-computerized physician order entry (CPOE) periods with adjusted risk calculated using multivariate logistic regression analysis.

A. Comparisons in the 24 months pre- vs. 12 months post-CPOE implementation

Variable

Pre-CPOE

Post-CPOE

Adjusted OR a or β b coefficient

95% CI

P-value

ICU mortality, N (%)

382 (23.4)

187 (20.8)

0.98

0.7 - 1.3

0.87

Hospital mortality, N (%)

633 (38.7)

319 (35.5)

1.00

0.8 - 1.3

0.80

Mechanical ventilation duration in days, mean ± SD

7.2 ± 9.8

6.3 ± 10.8

-0.30

-1.3 - 0.7

0.54

ICU LOS in days, mean ± SD

8.0 ± 10.5

8.1 ± 16.9

1.13

-0.2 - 2.5

0.10

Hospital LOS in days, mean ± SD

46.8 ± 77.5

46.6 ± 84.6

3.60

-4.5 - 11.8

0.39

B. Comparisons in the 12 months pre- vs. 12 months post-CPOE implementation

Variable

Pre-CPOE

Post-CPOE

Adjusted OR a or β b coefficient

95% CI

P-value

ICU mortality, No.%

190 (23.4)

187 (20.8)

0.96

0.7 - 1.3

0.78

Hospital mortality, No.%

296 (36.4)

319 (35.5)

1.24

0.9 - 1.6

0.13

MV duration, mean ± SD

7.6 ± 10.3

6.3 ± 10.8

-0.56

-1.7 - 0.6

0.35

ICU LOS, mean ± SD

8.4 ± 10.5

8.1 ± 16.9

0.78

-0.9 - 2.5

0.37

Hospital LOS, mean ± SD

44.2 ± 64.5

46.6 ± 84.6

5.04

-4.1 - 14.2

0.28

C. Comparisons in the 3 months pre- vs. 3 months post-CPOE implementation

Variable

Pre-CPOE

Post-CPOE

Adjusted OR a or β b coefficient

95% CI

P-value

ICU mortality, N (%)

52 (27.1)

55 (23.0)

0.83

0.5 - 1.5

0.55

Hospital mortality, N (%)

74 (38.5)

90 (37.7)

1.04

0.6 - 1.8

0.90

Mechanical ventilation duration in days, mean ± SD

8.0 ± 9.9

6.1 ± 8.0

-1.49

-3.2 - 0.2

0.09

ICU LOS in days, mean ± SD

9.2 ± 11.2

6.9 ± 8.4

-1.77

-3.6 - 0.1

0.06

Hospital LOS in days, mean ± SD

41.2 ± 53.9

41.1 ± 53.1

1.21

-10.4 - 12.8

0.84

D. Comparisons in the 1 month pre- vs. 1 month post-CPOE implementation

Variable

Pre-CPOE

Post-CPOE

Adjusted OR a or β b coefficient

95% CI

P-value

ICU mortality, N (%)

16 (23.2)

16 (21.3)

1.17

0.4 - 3.2

0.77

Hospital mortality, N (%)

24 (34.8)

28 (37.3)

2.10

0.8 - 5.5

0.14

Mechanical ventilation duration in days, mean ± SD

8.3 ± 9.8

5.8 ± 9.8

-2.74

-5.4 - (-0.1)

0.04

ICU LOS in days, mean ± SD

9.1 ± 10.0

6.5 ± 6.7

-2.52

-5.3 - 0.3

0.08

Hospital LOS in days, mean ± SD

39.0 ± 45.9

41.2 ± 55.1

-1.28

-19.2 - 16.6

0.89

These outcomes include ICU and hospital mortality, duration of mechanical ventilation and length of stay in the hospital and the intensive care unit. Adjustment was done for age, gender, Acute Physiology and Chronic Health Evaluation II score, source of admission, reason for admission (non-operative trauma and post-operative trauma), cardiac arrest as a reason for admission, chronic health illnesses (liver, cardiovascular and renal), creatinine, INR, use of vasopressors, and mechanical ventilation.

ICU: intensive care unit; CPOE: computerized physician order entry; CCRT: continuous renal replacement therapy; OR: odds ratio; CI: confidence interval CPOE: computerized physician order entry, ICU: intensive care unit, LOS: length of stay, SD: standard deviation, APACHE: Acute Physiology and Chronic Health Evaluation

a: OR is reported for categorical outcomes.

b: β coefficient is reported for continuous variables.

*The other variables that remained significant in the stepwise logistic multivariate regression analysis were: male gender (OR 1.39, 95% CI 1.07-1.81), APACHE II (OR 1.09, 95% CI 1.07-1.11), requirement for mechanical ventilation (OR 21.9, 95% CI 7.62-62.94), liver cirrhosis (OR 2.97, 95% CI 2.16-4.10), postoperative trauma (OR 0.38, 95% CI 0.25-0.58) and INR (OR 1.53, 95% CI 1.34-1.76).

**The other variables that remained significant in the stepwise multivariate logistic regression analysis were: age (OR 1.01, 95% CI 1.01-1.02), APACHE II (OR 1.09, 95% CI 1.07-1.11), requirement for mechanical ventilation (OR 2.451, 95% CI 1.67-3.59), liver cirrhosis (OR 2.39, 95% CI 1.72-3.31) chronic renal failure (OR 1.61, 95% CI 1.16-2.22), postoperative trauma (OR 0.43, 95% CI 0.31-0.59), non-operative trauma (OR 0.66, 95% CI 0.44-1.01), INR (OR 1.56, 95% CI 1.34-1.82) and creatinine (OR 1.00, 95% CI 0.99-1.00).

Outcomes of patient subgroups according to CPOE status

Table 3 describes the outcomes of four subgroups of patients in the 24 months pre- and 12 months post-CPOE implementation. There were no significant differences in ICU mortality of patients 12 to 17 years old, those with sepsis as an admission diagnosis, those with traumatic brain injury and those with APACHE II score ≥ 23.0. Also there were no significant differences in hospital mortality, the duration of mechanical ventilation and of stay in the ICU and hospital within the same periods pre- and post-CPOE in the 4 subgroups.
Table 3

Outcomes of four patient subgroups in the 24 months pre- and 12 months post-CPOE implementation with adjusted risk calculated using multivariate logistic regression analysis.

Variable

Pre-CPOE No. (%)

Post-CPOE No. (%)

Adjusted ORa or βb coefficient

95% CI

P value

A. patients in the age group 12-17 years, N = 129

ICU mortality, No.%

9 (11)

5 (10.6)

0.78

0.1 - 4.1

0.77

Hospital mortality, No.%

14 (17.1)

7 (14.9)

0.93

0.3 - 3.2

0.90

Mechanical ventilation duration in days, mean ± SD

7.7 ± 7.2

6.3 ± 9.0

-0.19

-3.2 - 2.8

0.90

ICU LOS in days, mean ± SD

8.2 ± 7.2

8.6 ± 17.0

2.41

-2.6 - 7.4

0.34

Hospital LOS in days, mean ± SD

42.8 ± 59.4

31.6 ± 44.1

-6.70

-30.8 - 17.4

0.58

B. Admission diagnosis of sepsis, N = 638

ICU mortality, No.%

152 (39)

96 (40.3)

1.09

0.7 - 1.7

0.70

Hospital mortality, No.%

230 (59)

141 (59.2)

1.01

0.7 - 1.6

0.96

Mechanical ventilation duration in days, mean ± SD

8.4 ± 10.6

7.3 ± 8.1

-1.62

-3.3 - 0.1

0.06

ICU LOS in days, mean ± SD

9.4 ± 10.7

8.4 ± 8.7

-1.41

-3.2 - 0.4

0.13

Hospital LOS in days, mean ± SD

40.7 ± 75.6

50.1 ± 125

13.07

-5.2 - 31.3

0.16

C. Traumatic brain injury, N = 322

ICU mortality, No.%

34 (14.5)

6 (6.7)

0.53

0.1 - 1.1

0.06

Hospital mortality, No.%

44 (19)

10 (11.1)

0.62

0.3 - 1.5

0.28

Mechanical ventilation duration in days, mean ± SD

10.7 ± 7.9

9.8 ± 9.6

-0.69

-2.7 - 1.3

0.50

ICU LOS in days, mean ± SD

11.2 ± 8.2

12.8 ± 15.1

2.24

-0.3 - 4.8

0.09

Hospital LOS in days, mean ± SD

72.2 ± 118.7

72.2 ± 82.2

13.7

-14.2 - 41.6

0.33

D. APACHE II > 23, N = 1065

ICU mortality, No.%

256 (35.6)

121 (35)

1.03

0.8 - 1.4

0.87

Hospital mortality, No.%

419 (58.3)

197 (56.9)

1.00

0.7 - 1.3

1.00

Mechanical ventilation duration in days, mean ± SD

10.1 ± 10.9

10.4 ± 14.1

0.02

-1.5 - 1.5

0.98

ICU LOS in days, mean ± SD

10.9 ± 11.3

11.7 ± 14.8

0.38

-1.2 - 2.0

0.64

Hospital LOS in days, mean ± SD

49.6 ± 88.2

52.2 ± 110.8

5.02

-7.2 - 17.3

0.42

These outcomes include ICU and hospital mortality, duration of mechanical ventilation and duration of stay in the hospital and the intensive care unit. Adjustment was done for age, gender, APACHE II, source of admission, reason for admission (non-operative trauma and post-operative trauma), cardiac arrest as a reason for admission, chronic health illnesses (liver, cardiovascular and renal), creatinine, INR, use of vasopressors, and mechanical ventilation.

ICU: intensive care unit; CPOE: computerized physician order entry; OR: odds ratio; CI: confidence interval; APACHE: Acute Physiology and Chronic Health Evaluation CPOE: computerized physician order entry, LOS: length of stay, SD: standard deviation.

a: OR is reported for categorical outcomes.

b: β coefficient is reported for continuous variables.

Discussion

Our study demonstrates that CPOE implementation was not associated with any significant improvement in mortality or any change in other outcomes within 12 months of implementation in critically ill patients older than 12 years.

CPOE has been advocated as a tool to enhance patient care and safety and improve resource utilization by ensuring standardized, legible, and complete orders thus eliminating prescription errors, providing advisories orders for drugs that can potentially harm patients and offering physicians evidence-based clinical decision support. Several hospital wide studies showed benefits attributed to CPOE [1618]. A systematic review that evaluated the effect of CPOE on patient safety found that 23 of 25 selected studies showed 13-99% relative risk reduction in medication errors, 6 of 9 selected studies showed 35-98% relative risk reduction in potential adverse drug events and 4 of 7 selected studies showed 30-84% relative risk reduction in adverse drug events [4]. Other studies showed that the impact of CPOE implementation might extend to mortality benefit. A cross-sectional study of urban hospitals in Texas found that automated order entry was associated with significant reduction in the adjusted odds of death for myocardial infarction and coronary artery bypass graft procedures [19]. Longhurst and colleagues found a decrease in the mean monthly adjusted mortality rate by 20% (95% CI 0.8%-40%, p = 0.03) at a tertiary care children hospital post-CPOE implementation [20]. Moreover, other studies showed a CPOE-associated improvement in hospital resource utilization, resulting in decreased hospital lengths of stay and costs [21, 22].

On the other hand, other hospital-wide reports raised concerns about the effectiveness of CPOE in averting medication errors and promoting patient safety. In fact, some studies demonstrated that CPOE can induce errors. Koppel et al identified 22 situations in which the CPOE system facilitated medication errors [7]. These situations fell into two categories: information errors generated by fragmentation of data and flaws in human-machine interface [7]. The causes of CPOE pitfalls include poor quality of CPOE system, inadequate technical support, healthcare provider resistance to change, insufficient preparation and insufficient user training [23].

Data on CPOE implementation in ICU setting are scarce. On one hand, ICUs environment is well controlled and monitored and has higher staffing level than other areas of the hospital. On the other hand, critically ill patients are a vulnerable population and the healthcare providers working in the ICU have many responsibilities and frequently perform multiple and complex tasks and so are prone to errors. One study looked at the frequency of distraction or interruptions that occur during computerized order entry in one medical ICU and found that one distraction or interruption occurred approximately every 5 minutes and preceded 2 order entry errors [24]. Reports on the impact of CPOE on the outcome of ICU patients have revealed conflicting results. Some studies showed benefit or no harm from CPOE use in the ICU. A prospective trial that compared paper-based order entry in 14 ICU beds (80 patient-days) to CPOE implemented in 8 ICU beds (80 patient-days) found that CPOE was associated with significant lower incidence of medication prescription errors and adverse drug events [25]. Another study showed that CPOE in the ICU was associated with significant reduction in the time between ordering and reporting stat tests [26]. Development of evidence-based decision algorithm for red cell transfusion facilitated by CPOE led to significant reduction in inappropriate transfusions and decrease in blood transfusion from 1.08 ± 2.3 units 0.86 ± 2.3 units (p < 0.001)[27]. In a before-after study at a 20-bed tertiary pediatric ICU, Del Beccaro, et al observed a statistically insignificant decrease in mortality from 4.2% to 3.46% in the 13-month pre- and post-CPOE implementation respectively [9]. Keene et al also found no increased mortality post-CPOE initiation even post-adjusting for all covariates (OR, 0.71; 95% CI, 0.32-1.57) [9, 10]. On the other hand, the report of Han et al [11] in 2005 was alarming. The authors reviewed the mortality data at a pediatric hospital in the 13 months before (n = 1394, 56.7% were admitted to ICU) and 5 months after (n = 548, 56.9% were admitted to ICU) implementation of a different CPOE system (Millenium Powerchart software system, Cerner Corporation, Kansas City, MO) and found a significant increase in mortality post-CPOE implementation from 2.8% to 6.6% (OR: 3.71; 95% CI: 2.13-6.46)[11]. Our study demonstrated no change in the mortality of critically ill patients > 12 years of age post-CPOE implementation suggesting safety of its implementation in our setting.

The observed differences in the outcomes among studies are probably related to inherent methodological differences, patient population as well as differences in the CPOE systems and their implementation process. Additionally, the observed impact of CPOE is influenced by the comparison group or period; as such no benefit will be observed if the CPOE is compared to a very well organized and robust paper-based order system, while the same CPOE will reveal major improvement if implemented where the paper-based system was not well established. This may explain the lack of benefit in our ICU, since the implementation converted pre-existing well established paper-based order sets into CPOE order sets. In particular, our evidence-based traumatic brain injury order set has been shown to improve outcome; hence the lack of benefit of CPOE in this population. Factors related to CPOE system and implementation also greatly influence its impact. CPOE systems vary considerably in their ease to use, ability to customize and the presence and level of clinical decision support. The success of implementation depends on multiple factors including organizational readiness for change, engagement of end users, the level of technical support and the project management. Hence, commitment of hospital management and healthcare providers to CPOE is crucial [28, 29]. Sittig et al's commentary [30] on the study by Han et al [11] illustrates many of the above points. They thought that the increased mortality was related to multiple factors that accompanied CPOE implementation and significantly affected hospital workflow. These factors included quick hospital-wide implementation of the system over a 6-day period, pharmacy centralization for all medications including vasoactive agents which may have affected the care of ICU patients, delay in patient registration causing delays in management of seriously ill patients, increased physician workload leading to less focus on patient stabilization and management at presentation, wireless networking which led to delays in order entry during peak operational periods, order entering taking long time (1-2 minutes for a single order), decreased collaborative patient care due to the diminution of face-to face communication and interaction between nurses and physicians, ICU nurses spending significant amount of time at a computer terminal away from the bedside, and not using order sets [30]. Harrison and colleagues evaluated the unintended consequences of health information technology in general including CPOE and demonstrated other important factors such as the elimination of informal interactions and redundant checks that help detect errors, spending more time on order entry rather patient care, overdependence on decision support system for appropriate prescribing leading to management difficulties when CPOE was unavailable or down [31]. Additionally, Karsh et al described eleven fallacies of health information technology which undermine its efficacy and suggested that collaboration between human factors engineers, applied psychologists, medical sociologists, communication scientists, cognitive scientists and interaction designers were needed to overcome these challenges [32]. In our ICU, CPOE implementation was well planned and closely monitored. Nevertheless, we faced multiple challenges which included occasional server slowness, sporadic resistance to CPOE use, alert fatigue largely due to redundant low risk pop-ups and complex orders. That ICU was the only hospital area that implemented CPOE in the study period was another challenge that necessitated additional training efforts for residents who periodically rotated in the ICU. All of these factors may have contributed to our findings of no difference in outcomes, but subsequently led to the formation of the ICU CPOE committee to enhance CPOE and optimize orders and ordering process. From our experience, these CPOE challenges decreased with time, which suggests that CPOE effectiveness in improving outcomes may require time longer than one year to be observed. It is also important to note that some of the challenges observed in other studies were institution-specific. For example, the observed delay in the registration of patients transferred from other hospitals in Han's et al study [11] was not an issue in our hospital as the registration process did not change post-CPOE implementation. Moreover, a more refined clinical decision support system that would advise on medication dose and duration or on optimal laboratory testing rates, that would utilize patient data and provide guideline-based care recommendations, or that would aid in weaning mechanical ventilation [33] might result in different outcomes.

This study must be interpreted in light of its strengths and limitations. The study used a prospectively collected data and was the first to address the relationship between CPOE and patient outcomes in adult ICU setting. It evaluated a commercial CPOE system which may strengthen the generalizability of our findings. Moreover, we described the CPOE experience at a Saudi tertiary care ICU staffed by multi-national healthcare workers with different backgrounds in information technology. Additionally, the study was done in a period during which there was no change in the ICU structure and function. Among the limitations was the potential time bias considering the pre-post design. However, we adjusted for all potential confounders in our multivariate analysis. Because of the study design, we were not able to examine intermediate endpoints such as technical performance, adverse drug events, adherence to guidelines and staff satisfaction. We believe that these endpoints should be targets for future research.

Conclusions

In conclusion, our study found that CPOE implementation in an adult medical surgical ICU was associated with no improvement in ICU and hospital mortality in the immediate period and up to 12 months after implementation. CPOE can be an important tool to improve health care, but its effectiveness in improving the outcomes of critically ill patients is unproven and may require ICU-specific reengineering and customization to achieve the goal of a demonstrable mortality reduction.

Key messages

  • Implementation of Computerized physician order entry (CPOE) was found to be associated with no significant changes in morbidity and mortality of critically ill patients in the immediate period and up to 12 months afterwards.

  • These findings were also seen in four high risk groups of patients: age 12-17 years, traumatic brain injury, diagnosis of sepsis on admission and admission APACHE II ≥ 23.

  • CPOE effectiveness in the ICU may require ICU-specific reengineering and customization to achieve the goal of mortality reduction.

Declarations

Acknowledgements

The authors thank Dr. Asgar H. Rishu for his contribution in data acquisition.

Funding

None

Authors’ Affiliations

(1)
King Abdulaziz Medical City
(2)
College of Medicine, King Saud bin Abdulaziz University for Health Sciences

References

  1. Eslami S, de Keizer NF, Abu-Hanna A: The impact of computerized physician medication order entry in hospitalized patients--a systematic review. Int J Med Inform. 2008, 77: 365-376. 10.1016/j.ijmedinf.2007.10.001.View ArticlePubMedGoogle Scholar
  2. [http://www.leapfroggroup.org/for_hospitals/leapfrog_safety_practices/cpoe]
  3. Institute of Medicine: "Crossing the quality chasm: A new health system for the 21st century". 2001, National Academies Press. Washington DCGoogle Scholar
  4. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U: The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc. 2008, 15: 585-600. 10.1197/jamia.M2667.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, Vander Vliet M, Seger DL: Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998, 280: 1311-1316. 10.1001/jama.280.15.1311.View ArticlePubMedGoogle Scholar
  6. Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF: High rates of adverse drug events in a highly computerized hospital. Arch Intern Med. 2005, 165: 1111-1116. 10.1001/archinte.165.10.1111.View ArticlePubMedGoogle Scholar
  7. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, Strom BL: Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005, 293: 1197-1203. 10.1001/jama.293.10.1197.View ArticlePubMedGoogle Scholar
  8. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH: Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006, 13: 547-556. 10.1197/jamia.M2042.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Del Beccaro MA, Jeffries HE, Eisenberg MA, Harry ED: Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics. 2006, 118: 290-295. 10.1542/peds.2006-0367.View ArticlePubMedGoogle Scholar
  10. Keene A, Ashton L, Shure D, Napoleone D, Katyal C, Bellin E: Mortality before and after initiation of a computerized physician order entry system in a critically ill pediatric population. Pediatr Crit Care Med. 2007, 8: 268-271. 10.1097/01.PCC.0000260781.78277.D9.View ArticlePubMedGoogle Scholar
  11. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, Bayir H, Orr RA: Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005, 116: 1506-1512. 10.1542/peds.2005-1287.View ArticlePubMedGoogle Scholar
  12. Arabi Y, Alshimemeri A, Taher S: Weekend and weeknight admissions have the same outcome of weekday admissions to an intensive care unit with onsite intensivist coverage. Crit Care Med. 2006, 34: 605-611.View ArticlePubMedGoogle Scholar
  13. Arabi YM, Haddad S, Tamim HM, Al-Dawood A, Al-Qahtani S, Ferayan A, Al-Abdulmughni I, Al-Oweis J, Rugaan A: Mortality reduction after implementing a clinical practice guidelines-based management protocol for severe traumatic brain injury. J Crit Care. 2010, 25: 190-195. 10.1016/j.jcrc.2009.05.004.View ArticlePubMedGoogle Scholar
  14. Knaus WA, Draper EA, Wagner DP, Zimmerman JE: APACHE II: a severity of disease classification system. Crit Care Med. 1985, 13: 818-829. 10.1097/00003246-198510000-00009.View ArticlePubMedGoogle Scholar
  15. Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, Suppes R, Feinstein D, Zanotti S, Taiberg L, Gurka D, Cheang M: Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med. 2006, 34: 1589-1596. 10.1097/01.CCM.0000217961.75225.E9.View ArticlePubMedGoogle Scholar
  16. Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, Laffel G, Sweitzer BJ, Shea BF, Hallisey R: Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 1995, 274: 29-34. 10.1001/jama.274.1.29.View ArticlePubMedGoogle Scholar
  17. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, Boyle D, Leape L: The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999, 6: 313-321. 10.1136/jamia.1999.00660313.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Upperman JS, Staley P, Friend K, Neches W, Kazimer D, Benes J, Wiener ES: The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital. J Pediatr Surg. 2005, 40: 57-59. 10.1016/j.jpedsurg.2004.09.024.View ArticlePubMedGoogle Scholar
  19. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR: Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med. 2009, 169: 108-114. 10.1001/archinternmed.2008.520.View ArticlePubMedGoogle Scholar
  20. Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS, Dawes CG, Sharek PJ: Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics. 126: 14-21.
  21. Mekhjian HS, Kumar RR, Kuehn L, Bentley TD, Teater P, Thomas A, Payne B, Ahmad A: Immediate benefits realized following implementation of physician order entry at an academic medical center. J Am Med Inform Assoc. 2002, 9: 529-539. 10.1197/jamia.M1038.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Tierney WM, Miller ME, Overhage JM, McDonald CJ: Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA. 1993, 269: 379-383. 10.1001/jama.269.3.379.View ArticlePubMedGoogle Scholar
  23. Papshev D, Peterson AM: Electronic prescribing in ambulatory practice: promises, pitfalls, and potential solutions. Am J Manag Care. 2001, 7: 725-736.PubMedGoogle Scholar
  24. Collins S, Currie L, Bakken S, Cimino JJ: Interruptions during the use of a CPOE system for MICU rounds. AMIA Annu Symp Proc. 2006, 895-Google Scholar
  25. Colpaert K, Claus B, Somers A, Vandewoude K, Robays H, Decruyenaere J: Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care. 2006, 10: R21-10.1186/cc3983.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Thompson W, Dodek PM, Norena M, Dodek J: Computerized physician order entry of diagnostic tests in an intensive care unit is associated with improved timeliness of service. Crit Care Med. 2004, 32: 1306-1309. 10.1097/01.CCM.0000127783.47103.8D.View ArticlePubMedGoogle Scholar
  27. Rana R, Afessa B, Keegan MT, Whalen FX, Nuttall GA, Evenson LK, Peters SG, Winters JL, Hubmayr RD, Moore SB, Gajic O: Evidence-based red cell transfusion in the critically ill: quality improvement using computerized physician order entry. Crit Care Med. 2006, 34: 1892-1897. 10.1097/01.CCM.0000220766.13623.FE.View ArticlePubMedGoogle Scholar
  28. Ash JS, Stavri PZ, Kuperman GJ: A consensus statement on considerations for a successful CPOE implementation. J Am Med Inform Assoc. 2003, 10: 229-234. 10.1197/jamia.M1204.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Southon FC, Sauer C, Grant CN: Information technology in complex health services: organizational impediments to successful technology transfer and diffusion. J Am Med Inform Assoc. 1997, 4: 112-124. 10.1136/jamia.1997.0040112.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM: Lessons from "Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system". Pediatrics. 2006, 118: 797-801. 10.1542/peds.2005-3132.View ArticlePubMedGoogle Scholar
  31. Harrison MI, Koppel R, Bar-Lev S: Unintended consequences of information technologies in health care--an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007, 14: 542-549. 10.1197/jamia.M2384.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Karsh BT, Weinger MB, Abbott PA, Wears RL: Health information technology: fallacies and sober realities. J Am Med Inform Assoc. 2010, 17: 617-623. 10.1136/jamia.2010.005637.View ArticlePubMedPubMed CentralGoogle Scholar
  33. Bouadma L, Lellouche F, Cabello B, Taille S, Mancebo J, Dojat M, Brochard L: Computer-driven management of prolonged mechanical ventilation and weaning: a pilot study. Intensive Care Med. 2005, 31: 1446-1450. 10.1007/s00134-005-2766-2.View ArticlePubMedGoogle Scholar
  34. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/11/71/prepub

Copyright

© Al-Dorzi et al; licensee BioMed Central Ltd. 2011

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.