- Research article
- Open Access
- Open Peer Review
Systematic review of clinical decision support interventions with potential for inpatient cost reduction
BMC Medical Informatics and Decision Makingvolume 13, Article number: 135 (2013)
Healthcare costs are increasing rapidly and at an unsustainable rate in many countries, and inpatient hospitalizations are a significant driver of these costs. Clinical decision support (CDS) represents a promising approach to not only improve care but to reduce costs in the inpatient setting. The purpose of this study was to systematically review trials of CDS interventions with the potential to reduce inpatient costs, so as to identify promising interventions for more widespread implementation and to inform future research in this area.
To identify relevant studies, MEDLINE was searched up to July 2013. CDS intervention studies with the potential to reduce inpatient healthcare costs were identified through titles and abstracts, and full text articles were reviewed to make a final determination on inclusion. Relevant characteristics of the studies were extracted and summarized.
Following a screening of 7,663 articles, 78 manuscripts were included. 78.2% of studies were controlled before-after studies, and 15.4% were randomized controlled trials. 53.8% of the studies were focused on pharmacotherapy. The majority of manuscripts were published during or after 2008. 70.5% of the studies resulted in statistically and clinically significant improvements in an explicit financial measure or a proxy financial measure. Only 12.8% of the studies directly measured the financial impact of an intervention, whereas the financial impact was inferred in the remainder of studies. Data on cost effectiveness was available for only one study.
Significantly more research is required on the impact of clinical decision support on inpatient costs. In particular, there is a remarkable gap in the availability of cost effectiveness studies required by policy makers and decision makers in healthcare systems.
Healthcare costs are increasing rapidly and at an unsustainable rate in many countries. In the United States, inpatient care is the single largest contributor to national health expenditures, accounting for 31.5% of $2.7 trillion dollars of health expenditures in 2011 . As such, inpatient care is a significant driver of increased health spending. In 2011, the annual spending on hospital care in the U.S. grew 4.3% as compared to 3.9% growth in overall health expenditures . Contributing to the importance of addressing inpatient costs is the fact that reducing these costs has the potential to financially benefit inpatient healthcare organizations regardless of reimbursement models. Traditional episode-of-care payment systems (for example, Medicare's inpatient prospective payment system), bundled payments systems, and comprehensive payment systems (embodied in accountable care organizations) are all examples of reimbursement models under which healthcare organizations stand to benefit from reducing inpatient costs .
Clinical decision support (CDS) represents a promising approach to both improving outcomes and decreasing costs . Several past systematic reviews have examined outcomes related to clinical decision support systems in the inpatient setting, but few have focused on the impact of CDS on inpatient costs specifically [4–9]. One review published in 2006 evaluated cost as an outcome . However, that review was focused on health information technology (IT) in general rather than CDS specifically. Moreover, it only included studies published through January 2004. A second, more recent review on CDS included cost outcomes but was limited to studies with a randomized trial design . The design and timing of these two reviews potentially excluded relevant CDS intervention trials. In particular, non-randomized research designs are commonly used to evaluate CDS interventions.
Given the importance of limiting the growth of inpatient costs and the potential benefit of CDS, we sought to (i) inclusively identify promising interventions that could serve as models for more widespread implementation and to (ii) identify gaps in the literature warranting further research. As such, we systematically reviewed both randomized and non-randomized trials of CDS systems with the potential to reduce inpatient or emergency department (ED) costs.
Using a search strategy adapted from a previous systematic review , we searched MEDLINE through July 18, 2013. The latest search was performed on that date. We used a combination of the following search terms: decision support systems, clinical; decision-making, computer-assisted; computerized decision support; reminder systems; guideline adherence; and medical informatics. Details of the search strategy are available in Appendix I. Search results were limited to human subjects and the English language.
Inclusion and exclusion criteria
We defined a CDS system as a system designed to directly aid in clinical decision making, in which characteristics of individual patients are matched to a knowledge base for the purpose of presenting patient-specific assessments or recommendations to clinicians . Inclusion criteria were as follows: peer-reviewed primary manuscript; clinical trial of a CDS system in an inpatient or emergency department (ED) setting; and use of either cost or a proxy measure for cost (e.g., length of stay; see Data Extraction section for full list) as an outcome metric. Of note, we opted to include studies in an ED setting, as these costs often become part of inpatient costs because many ED patients are admitted to the hospital. Exclusion criteria were as follows: non-English manuscript; or use of CDS in the control group.
Titles and abstracts from retrieved references were evaluated by a single reviewer to determine potential inclusion eligibility. The full texts of studies that appeared to be potentially eligible were then evaluated by the same reviewer. Final inclusion determinations were made using the full texts. In cases where a study’s inclusion status was unclear upon review by the primary reviewer, the authors jointly reviewed the study and made a consensus decision.
Data extraction was performed by a single reviewer using a standardized form. Any issues involving uncertainty were resolved through author consensus. For each article that met inclusion criteria, data were extracted on setting, trial design, intervention, and trial results. Setting was assigned as one of three categories: ED, ICU, or hospital. A hospital setting typically included inpatient wards, but could also include a combination of wards, ED, ICU, or surgical settings. Trial design was assigned based on categories defined by the Cochrane Effective Practice and Organization of Care Group . Design categories included randomized controlled trials (RCTs), non-randomized controlled trials (NRCTs), controlled before-after studies (CBA), and interrupted-time-series studies (ITSs).
Abstracted trial results included outcomes with potential cost saving implications, whether costs were directly measured, whether there was a statistically and clinically significant improvement in cost or in a proxy measure, and whether the study could be considered a cost effectiveness study. Specifically, measures with potential cost saving implications consisted of direct cost measures or other proxy measures with cost ramifications. Proxy cost measures included length of stay, readmission rates, resource utilization metrics (e.g., imaging studies), adverse events, and process measures correlated with adverse events. A change in a cost or proxy measure was decided a priori to be the primary outcome measure. Clinical significance of results was determined by author consensus. To be considered a cost effectiveness study, the study must have accounted at least for the personnel costs included in developing and deploying the intervention. For commercial CDS systems, at least the cost of licensing the software must also have been considered.
Extracted data were analyzed and presented in table form and narrative summary. Additionally, significant themes, trends, and patterns were noted and discussed. To better understand the potential relationship between study outcomes and potential explanatory factors, Fisher’s exact test of independence was conducted with the independent variable being a statistically and clinically significant improvement in an actual or proxy cost measure. One of the potential explanatory variables examined was study quality, with CBA trials and NRCTs conducted at single sites distinguished from studies that utilized more rigorous study designs. Other potential explanatory variables examined included study setting and clinical domain. For the purposes of this analysis, clinical domains with two or fewer studies were combined into a single category. A p-value of < 0.05 was considered statistically significant.
Results and discussion
The literature search returned a total of 7,663 unique references. 7,500 references were excluded after screening of titles and abstracts. We reviewed 163 full-text articles, of which 78 [13–90] met criteria for inclusion in the review (see Figure 1). Characteristics of these studies are summarized in Additional file 1: Table S1.
A majority (52.6%) of studies were published during or after 2008 [14, 15, 18–20, 25, 26, 29, 32–34, 38–40],[43–47, 49, 50, 55–61, 65–67, 70, 74–76, 78],[81, 82, 85, 86, 90]. The earliest included study was published in 1989 . This high concentration of studies published between 2008 and 2013 represents large recent growth in the evaluation of inpatient CDS systems and is likely consistent with increasing adoption of health IT generally.
A majority of the studies (55.1%) took place in a general hospital setting ([14–17, 19, 24–26, 28, 30, 32, 33, 37],[41–43, 46, 48–50, 52–56, 60–62, 64, 66, 67],[69–72, 74, 76–78, 80, 85, 87, 90]. 29.5% of the studies occurred in an ICU setting [18, 23, 31, 34–36, 39, 40, 44, 47],[51, 57–59, 63, 65, 68, 73, 79, 82],[86, 88, 89], and 15.4% took place in an ED setting [13, 20–22, 27, 29, 38, 45, 75, 81],[83, 84].
A large proportion (84.6%) [13–18, 20–27, 29, 30, 32–37, 39–52, 54–58, 60],[61, 64–70, 72–78, 82–90] of the studies were quasi-experimental trials, which can be defined as studies that aim to evaluate interventions without the use of randomization . Overall, the most common design used in the studies was the controlled before-after design, wherein researchers used historical controls prior to the implementation of an intervention. Of the 78 studies, 61 (78.2%) were some form of controlled before-after study [13–18, 20–27, 29, 30, 32–37, 39–51, 54, 55],[57, 60, 61, 64, 65, 68–70, 72–78, 82–90]. Only 12 (15.4%) of the studies were randomized controlled trials (RCTs) [19, 28, 31, 38, 53, 59, 62, 63],[71, 79–81]. The remaining five studies were interrupted-time-series studies [56, 58] and non-randomized controlled trials [52, 66, 67].
The frequent use of quasi-experimental designs in medical informatics evaluations has been noted previously [91, 92], and the results of this review are consistent. Given the overwhelming prevalence of quasi-experimental designs, reviews of CDS systems that only include RCTs are bound to exclude a large portion of the published literature. With respect to this systematic review, a deliberate decision was made to include quasi-experimental studies, as one of the primary goals of this study was to inclusively identify CDS interventions that have the potential for reducing inpatient costs. At the same time, the inclusion of quasi-experimental study designs certainly resulted in the inclusion of studies that are more subject to bias than RCTs.
The most common clinical focus targeted by CDS systems in the review was pharmacotherapy, with 53.8% of studies focused on this area [16–18, 20, 23, 24, 26, 35, 37, 39],[40, 42, 43, 45, 47, 48, 50, 51],[54, 57–62, 64, 65, 67–69, 71–74, 76–79, 85],[88–90]. Of the studies focused on pharmacotherapy, the most common areas of specific focus were nephrotoxic drugs (23.8%) [24, 41, 60, 69, 76–78, 85, 90], antibiotics (21.4%) [20, 23, 26, 35, 54, 62, 68, 71],, and insulin management (14.3%) [39, 40, 57, 59, 65, 79].
The second most common area of clinical focus was venous thromboembolism prophylaxis, which accounted for 9% of studies in the review [28, 30, 43, 46, 53, 55, 87]. Examples of other clinical areas addressed included blood transfusion management [14, 36, 80], ventilation management [31, 34, 63], and radiology utilization [22, 29, 75].
The significant focus on pharmacotherapy within the included studies may reflect the importance of drug selection within computerized provider order entry (CPOE) systems, which are foundational to CDS in many inpatient settings. 50% [14, 16, 17, 19, 22–26, 28–30, 35–37, 41–43],[45, 46, 48, 50, 56, 58, 60, 61],[64, 66–68, 72, 73, 75, 76, 80, 85],[87, 89, 90] of the studies overall involved CDS in the context of CPOE, of which 61.5% [16, 17, 23, 24, 26, 35, 37, 41],[42, 45, 48, 50, 58, 60, 61, 64],[67, 68, 72, 73, 76, 85, 89, 90] were focused on pharmacotherapy.
Only one of the 78 studies (1.3%) was considered to be a cost effectiveness study . This study evaluated the use of a well-known diagnostic decision support system, DXplain, with residents in a teaching hospital. The authors reported that access to DXplain had been provided at no charge for the purposes of the study, but that an annual license would have cost their organization $4,000-$6,000 per year . It is telling that the only study to address cost effectiveness in this review concerned a simple license to a stand-alone, diagnostic CDS system. The majority of the studies in this review dealt with more comprehensive, integrated systems either purchased through vendors or developed locally. Under those circumstances, providing information about cost of development, implementation, or licensing fees is presumably more difficult. However, the near complete lack of this type of information is concerning given the need for such cost-effectiveness information by public policy developers and decision makers within healthcare organizations.
Direct measurement of cost
10 (12.8%) studies in the review directly measured costs [20, 24, 33, 35–37, 62, 64, 68, 71]. Eight of these studies focused on pharmacotherapy [20, 24, 35, 37, 62, 64, 68, 71], while the other two addressed management of blood transfusion  and general medical diagnosis . These studies included two RCTs [62, 71], with the remainder of studies having a quasi-experimental design. Of these studies, five reported a statistically and clinically significant improvement in a cost measure [33, 35, 36, 62, 71]. Except in one case as outlined, the cost involved in implementing these interventions was not studied. Because investments in CDS are like any other business investment, having only one side of the financial equation (cost impact) is insufficient for making public policy and business decisions.
Use of proxy cost measures
87.1% of the studies in the review solely reported proxy measures as indicators of impact on cost [13–19, 21–23, 25–32, 34, 38–61, 63, 65–67, 69],[70, 72–90]. The most commonly used type of proxy measure in this group was process measures that were associated with adverse events. Of the studies that solely used proxy measures, 52.9% reported this type of measure [15–17, 22, 25, 26, 28, 30, 32, 34],[41–43, 46, 50–54, 60, 61, 66, 67, 70],[73, 74, 76–78, 80, 85–87, 89, 90]. Examples of other proxy measures reported by these studies included rates of adverse events (reported by 39.7% of studies [16–19, 23, 25, 26, 39, 40, 43, 45–47],[53–55, 57–59, 63, 65, 67, 69, 72, 73],[79, 90]), length of stay (reported by 22.1% of studies [15, 19, 31, 38, 44, 45, 47, 49],[57, 63, 66, 67, 72, 81, 82]), resource utilization metrics (reported by 16.2% of studies [13, 14, 29, 38, 48, 56, 75, 81],[82, 86, 88]) and patient charges (reported by 5.9% of studies [21, 27, 83, 84]).
As noted, a strikingly small percentage of the studies directly measured an intervention’s impact on cost. Therefore, in the majority of cases we were left to infer a possible cost savings from non-financial proxy measures. Doing so has some inherent limitations. For example, five studies reported patient charges as an outcome [21, 27, 33, 83, 84]. This is not a direct measure of cost, and it can be unclear as to how charges actually relate to cost . We assumed that an institution’s costs were at least proportional to what it charged a patient. However, given that we did not know the actual relationship between costs and charges at any given institution, this assumption suffered from an element of uncertainty.
A second limitation of using proxy measures is illustrated by the conflict of explicit cost measures and proxy measures within the same study. For example, two studies reported no differences in actual measured costs but reported decreases in length of stay [24, 64]. For this review, we considered length of stay a reasonable proxy measure for cost. But in these two instances, shorter lengths of stay did not coincide with actual decreased costs. The reverse of this situation was present in two studies, where directly measured costs decreased, but no difference in length of stay was detected [33, 62]. It is notable that of the ten studies in the review that directly measured costs, four demonstrated discrepancies between explicit cost measures and available proxy cost measures.
A third limitation of using proxy measures is related to the inconsistent relationship between process measures correlated with adverse events and the actual rates of those adverse events. We considered measures of adverse events an appropriate proxy measure for cost given the potential for these events to necessitate the utilization of additional resources. We went a step further and included process measures correlated with adverse events as proxy measures as well. For example, one study in the review reported the rate of compliance with venous thromboembolism prophylaxis guidelines (a process measure correlated with an adverse event) . Alternatively, another study reported the actual incidence of venous thromboembolism (a measure of an adverse event) . For this review, we made the assumption that an improvement in a process measure associated with an adverse event would be associated with an improvement in the incidence of that adverse event. Decreased incidence of an adverse event, in turn, would be associated with cost savings. However, in one study, process measures correlated with an adverse event were significantly improved, but there was no improvement in the incidence of the actual adverse event . More perplexingly, another study reported improvements in a process measure, no improvement in the correlated adverse event, and a significant improvement in length of stay .
Impact on cost/proxy measures
55 (70.5%) of the studies reported a statistically and clinically significant improvement in a cost or proxy measure [13–18, 21–23, 25, 26, 28, 30, 32, 33],[35, 36, 38–43, 45–50, 53–57, 59–62, 65–67, 69],[71, 73–75, 77, 78, 80, 82, 84, 86–88],. However, when considered in the context of the lack of direct cost measurements, the limitations of proxy cost measures, and the prevalence of quasi-experimental designs, it is difficult to know what level of confidence to place in that finding. On the face of it, CDS does appear to be a promising intervention for reducing inpatient costs. However, further research is clearly needed in order to more concretely characterize the benefits that have been achieved and that might be achieved in the future.
Relationship between primary study outcome and potential explanatory factors
Table 1 provides the results of Fisher’s exact tests of independence between the primary study outcome and potential explanatory factors. There was a trend towards single-site CBA and NRCT studies having more positive outcomes than more rigorous studies (75% vs. 56%, p = 0.14), which was an expected finding given the bias known in the literature for poorer-quality studies more frequently showing positive outcomes. Study setting did not appear to be significantly associated with study outcomes, whereas clinical domain did appear to have a significant relationship with study outcomes (p = 0.04). In particular, all of the seven studies on VTE prophylaxis and all of the three studies on transfusion had successful outcomes. However, we caution against over-interpreting the implications of these statistical analyses, as the interventions were heterogeneous in nature, and as the proxy outcome measures utilized in these analyses had several important limitations as discussed earlier. The primary finding from our study remains that additional research is required on the true cost implications of CDS in the inpatient setting.
Models for more widespread implementation
One of the aims of this study was to identify interventions that were promising for more widespread implementation. As noted, a large proportion of the CDS interventions found in this review are related to pharmacotherapy within the context of CPOE. As organizations continue to adopt or expand CPOE systems with integrated CDS, a potentially promising area to focus on is the management of antibiotics, as three of the five studies that directly measured costs and demonstrated improvement in cost metrics [35, 62, 71] were focused specifically on the management of antibiotics. Another promising area for implementation is venous thromboembolism prophylaxis, as all seven of the studies with this focus demonstrated improvements in the outcomes of interest [28, 30, 43, 46, 53, 55, 87].
Our study is potentially limited by the use of only one database, MEDLINE, for our search. As a result, there was a risk to exclude relevant articles. However, in a previous systematic review of CDS interventions , which searched MEDLINE, CINAHL, and the Cochrane Central Register of Controlled Trials, all 88 of the studies included in the final study sample were indexed and available in MEDLINE. Therefore, we believe that this risk is limited. A further potential limitation of our study is the use of a single reviewer to perform study selection and data extraction. However, any issues involving uncertainty were resolved through author consensus to manage this risk.
Health IT, and CDS in particular, has been touted for many years as a highly promising strategy for improving clinical care and “bending the cost curve” [94, 95]. However, more recent analyses have found that health IT systems such as EHR systems are not having the anticipated benefits in cost reduction [96, 97]. This study adds to these concerns that the potential benefits of health IT and CDS are not well grounded in empirical evidence, with only ten studies directly measuring costs and only one actually measuring cost-effectiveness of CDS for inpatient cost reduction.
As healthcare organizations continue to rapidly adopt health IT, leadership within those organizations must decide how to best use limited resources. Presumably, the potential cost savings associated with intervention candidates is a major factor in making those decisions. However, as a discipline, informatics does not appear to be meeting the needs of these healthcare decision makers with regard to CDS, as we have not been providing sufficient, rigorous data related to the cost benefits of CDS interventions in the inpatient setting. Further research with specific attention to cost implications of CDS systems in the inpatient setting is clearly needed.
Search strategy details
exp Decision Support Systems, Clinical/
Decision Making, Computer-Assisted/
"computerized decision support".mp.
exp Reminder Systems/
exp Guideline Adherence/
exp Medical Informatics/
5 and 6
1 or 2 or 3 or 4 or 7
limit 8 to (english language and humans)
CLF was an Air Force flight surgeon and is currently a post-doctoral fellow in biomedical informatics. All authors are physicians, and BEB and KK are faculty members in biomedical informatics.
Keehan SP, Cuckler GA, Sisko AM, Madison AJ, Smith SD, Lizonitz JM, Poisal JA, Wolfe CJ: National health expenditure projections: modest annual growth until coverage expands and economic growth accelerates. Health Aff (Millwood). 2012, 31: 1600-1612. 10.1377/hlthaff.2012.0404.
Miller HD: From volume to value: better ways to pay for health care. Health Aff (Millwood). 2009, 28: 1418-1428. 10.1377/hlthaff.28.5.1418.
Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE: A roadmap for national action on clinical decision support. J Am Med Inform Assoc. 2007, 14: 141-145. 10.1197/jamia.M2334.
Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998, 280: 1339-1346. 10.1001/jama.280.15.1339.
Kaushal R, Shojania KG, Bates DW: Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003, 163: 1409-1416. 10.1001/archinte.163.12.1409.
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005, 293: 1223-1238. 10.1001/jama.293.10.1223.
Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330: 765-10.1136/bmj.38398.500764.8F.
Sintchenko V, Magrabi F, Tipper S: Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review. Med Inform Internet Med. 2007, 32: 225-240.
Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J: Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010, 182: E216-225. 10.1503/cmaj.090578.
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG: Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006, 144: 742-752. 10.7326/0003-4819-144-10-200605160-00125.
Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, et al: Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012, 157: 29-43. 10.7326/0003-4819-157-1-201207030-00450.
EPOC resources for review authors.http://epocoslo.cochrane.org/epoc-specific-resources-review-authors,
Aase O: Clinical experience with a decision support computer program using Bayes' theorem to diagnose chest pain patients. Cardiology. 1999, 92: 128-134. 10.1159/000006960.
Adams ES, Longhurst CA, Pageler N, Widen E, Franzon D, Cornfield DN: Computerized physician order entry with decision support decreases blood transfusions in children. Pediatrics. 2011, 127: e1112-1119. 10.1542/peds.2010-3252.
Austrian JS, Adelman JS, Reissman SH, Cohen HW, Billett HH: The impact of the heparin-induced thrombocytopenia (HIT) computerized alert on provider behaviors and patient outcomes. J Am Med Inform Assoc. 2011, 18: 783-788. 10.1136/amiajnl-2011-000138.
Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, et al: Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998, 280: 1311-1316. 10.1001/jama.280.15.1311.
Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, Boyle D, Leape L: The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999, 6: 313-321. 10.1136/jamia.1999.00660313.
Bertsche T, Pfaff J, Schiller P, Kaltschmidt J, Pruszydlo MG, Stremmel W, Walter-Sack I, Haefeli WE, Encke J: Prevention of adverse drug reactions in intensive care patients by personal intervention based on an electronic clinical decision support system. Intensive Care Med. 2010, 36: 665-672. 10.1007/s00134-010-1778-8.
Boustani MA, Campbell NL, Khan BA, Abernathy G, Zawahiri M, Campbell T, Tricker J, Hui SL, Buckley JD, Perkins AJ, et al: Enhancing care for hospitalized older adults with cognitive impairment: a randomized controlled trial. J Gen Intern Med. 2012, 27: 561-567. 10.1007/s11606-012-1994-8.
Buising KL, Thursky KA, Black JF, MacGregor L, Street AC, Kennedy MP, Brown GV: Improving antibiotic prescribing for adults with community acquired pneumonia: Does a computerised decision support system achieve more than academic detailing alone?–A time series analysis. BMC Med Inf Decis Mak. 2008, 8: 35-10.1186/1472-6947-8-35.
Buller-Close K, Schriger DL, Baraff LJ: Heterogeneous effect of an Emergency Department Expert Charting System. Ann Emerg Med. 2003, 41: 644-652. 10.1067/mem.2003.182.
Carton M, Auvert B, Guerini H, Boulard J-C, Heautot J-F, Landre M-F, Beauchet A, Sznajderi M, Brun-Ney D, Chagnon S: Assessment of radiological referral practice and effect of computer-based guidelines on radiological requests in two emergency departments. Clin Radiol. 2002, 57: 123-128. 10.1053/crad.2001.0827.
Chan ALF, Wang H-Y, Leung HWC: Incorporation of a gentamicin dosage calculator into a computerized prescriber-order-entry system. Am J Health-Syst Pharm. 2006, 63: 1344-1345. 10.2146/ajhp050474.
Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, Lee R, Mekala A, Song J, Komaroff AL, Bates DW: Guided medication dosing for inpatients with renal insufficiency. JAMA. 2001, 286: 2839-2844. 10.1001/jama.286.22.2839.
Cho A, Lee JE, Yoon JY, Jang HR, Huh W, Kim Y-G, Kim DJ, Oh HY: Effect of an electronic alert on risk of contrast-induced acute kidney injury in hospitalized patients undergoing computed tomography. Am J Kidney Dis. 2012, 60: 74-81. 10.1053/j.ajkd.2012.02.331.
Cox ZL, Nelsen CL, Waitman LR, McCoy JA, Peterson JF: Effects of clinical decision support on initial dosing and monitoring of tobramycin and amikacin. Am J Health-Syst Pharm. 2011, 68: 624-632. 10.2146/ajhp100155.
Day F, Hoang LP, Ouk S, Nagda S, Schriger DL: The impact of a guideline-driven computer charting system on the emergency care of patients with acute low back pain. Proc Annu Symp Comput Appl Med Care. 1995, 576-580.
Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ: A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001, 345: 965-970. 10.1056/NEJMsa010181.
Drescher FS, Chandrika S, Weir ID, Weintraub JT, Berman L, Lee R, Van Buskirk PD, Wang Y, Adewunmi A, Fine JM: Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med. 2011, 57: 613-621. 10.1016/j.annemergmed.2010.09.018.
Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E: A clinical decision support system for prevention of venous thromboembolism: effect on physician behavior. JAMA. 2000, 283: 2816-2821. 10.1001/jama.283.21.2816.
East TD, Heermann LK, Bradshaw RL, Lugo A, Sailors RM, Ershler L, Wallace CJ, Morris AH, McKinley B, Marquez A, et al: Efficacy of computerized decision support for mechanical ventilation: results of a prospective multi-center randomized trial. Annual Symposium. Proc AMIA Symp. 1999, 251-255.
Eden A, Pizov R, Toderis L, Kantor G, Perel A: The impact of an electronic reminder on the use of alarms after separation from cardiopulmonary bypass. Anesth Analg. 2009, 108: 1203-1208. 10.1213/ane.0b013e3181984ef9.
Elkin PL, Liebow M, Bauer BA, Chaliki S, Wahner-Roedler D, Bundrick J, Lee M, Brown SH, Froehling D, Bailey K, et al: The introduction of a diagnostic decision support system (DXplainTM) into the workflow of a teaching hospital service can decrease the cost of service for diagnostically challenging Diagnostic Related Groups (DRGs). Int J Med Inf. 2010, 79: 772-777. 10.1016/j.ijmedinf.2010.09.004.
Eslami S, de Keizer NF, Abu-Hanna A, de Jonge E, Schultz MJ: Effect of a clinical decision support system on adherence to a lower tidal volume mechanical ventilation strategy. J Crit Care. 2009, 24: 523-529. 10.1016/j.jcrc.2008.11.006.
Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Lloyd JF, Burke JP: A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med. 1998, 338: 232-238. 10.1056/NEJM199801223380406.
Fernandez Perez ER, Winters JL, Gajic O: The addition of decision support into computerized physician order entry reduces red blood cell transfusion resource utilization in the intensive care unit. Am J Hematol. 2007, 82: 631-633. 10.1002/ajh.20888.
Fischer MA, Solomon DH, Teich JM, Avorn J: Conversion from intravenous to oral medications: assessment of a computerized intervention for hospitalized patients. Arch Intern Med. 2003, 163: 2585-2589. 10.1001/archinte.163.21.2585.
Fitzgerald M, Cameron P, Mackenzie C, Farrow N, Scicluna P, Gocentas R, Bystrzycki A, Lee G, O'Reilly G, Andrianopoulos N, et al: Trauma resuscitation errors and computer-assisted decision support. Arch Surg. 2011, 146: 218-225. 10.1001/archsurg.2010.333.
Flanders SJ, Juneja R, Roudebush CP, Carroll J, Golas A, Elias BL: Glycemic control and insulin safety: the impact of computerized intravenous insulin dosing. Am J Med Qual. 2009, 24: 489-497. 10.1177/1062860609338406.
Fogel SL, Baker CC: Effects of computerized decision support systems on blood glucose regulation in critically ill surgical patients. J Am Coll Surg. 2013, 216: 828-833. 10.1016/j.jamcollsurg.2012.12.015. discussion 833–825
Galanter WL, Didomenico RJ, Polikaitis A: A trial of automated decision support alerts for contraindicated medications using computerized physician order entry. J Am Med Inform Assoc. 2005, 12: 269-274. 10.1197/jamia.M1727.
Galanter WL, Polikaitis A, DiDomenico RJ: A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. J Am Med Inform Assoc. 2004, 11: 270-277. 10.1197/jamia.M1500.
Galanter WL, Thambi M, Rosencranz H, Shah B, Falck S, Lin F-J, Nutescu E, Lambert B: Effects of clinical decision support on venous thromboembolism risk assessment, prophylaxis, and prevention at a university teaching hospital. Am J Health-Syst Pharm. 2010, 67: 1265-1273. 10.2146/ajhp090575.
Giuliano KK, Lecardo M, Staul L: Impact of protocol watch on compliance with the surviving sepsis campaign. Am J Crit Care. 2011, 20: 313-321. 10.4037/ajcc2011421.
Griffey RT, Lo HG, Burdick E, Keohane C, Bates DW: Guided medication dosing for elderly emergency patients using real-time, computerized decision support. J Am Med Inform Assoc. 2012, 19: 86-93. 10.1136/amiajnl-2011-000124.
Haut ER, Lau BD, Kraenzlin FS, Hobson DB, Kraus PS, Carolan HT, Haider AH, Holzmueller CG, Efron DT, Pronovost PJ, Streiff MB: Improved prophylaxis and decreased rates of preventable harm with the use of a mandatory computerized clinical decision support tool for prophylaxis for venous thromboembolism in trauma. Arch Surg. 2012, 147: 901-907. 10.1001/archsurg.2012.2024.
Hoekstra M, Vogelzang M, Drost JT, Janse M, Loef BG, van der Horst ICC, Zijlstra F, Nijsten MWN: Implementation and evaluation of a nurse-centered computerized potassium regulation protocol in the intensive care unit–a before and after analysis. BMC Med Inf Decis Mak. 2010, 10: 5-10.1186/1472-6947-10-5.
Hulgan T, Rosenbloom ST, Hargrove F, Talbert DA, Arbogast PG, Bansal P, Miller RA, Kernodle DS: Oral quinolones in hospitalized patients: an evaluation of a computerized decision support intervention. J Intern Med. 2004, 256: 349-357. 10.1111/j.1365-2796.2004.01375.x.
Jones S, Mullally M, Ingleby S, Buist M, Bailey M, Eddleston JM: Bedside electronic capture of clinical observations and automated clinical alerts to improve compliance with an early warning score protocol. Crit Care Resusc. 2011, 13: 83-88.
Kazemi A, Ellenius J, Pourasghar F, Tofighi S, Salehi A, Amanati A, Fors UGH: The effect of computerized physician order entry and decision support system on medication errors in the neonatal ward: experiences from an Iranian teaching hospital. J Med Syst. 2011, 35: 25-37. 10.1007/s10916-009-9338-x.
Kellett J: Decision support and the appropriate use of fibrinolysis in myocardial infarction. Eff Clin Pract. 2001, 4: 1-9.
Knirsch CA, Jain NL, Pablos-Mendez A, Friedman C, Hripcsak G: Respiratory isolation of tuberculosis patients using clinical guidelines and an automated clinical decision support system. Infect Control Hosp Epidemiol. 1998, 19: 94-100. 10.1086/647773.
Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, Goldhaber SZ: Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005, 352: 969-977. 10.1056/NEJMoa041533.
Larsen RA, Evans RS, Burke JP, Pestotnik SL, Gardner RM, Classen DC: Improved perioperative antibiotic use and reduced surgical wound infections through use of computer decision analysis. Infect Control Hosp Epidemiol. 1989, 10: 316-320. 10.1086/646035.
Lecumberri R, Marques M, Diaz-Navarlaz MT, Panizo E, Toledo J, Garcia-Mouriz A, Paramo JA: Maintained effectiveness of an electronic alert system to prevent venous thromboembolism among hospitalized patients. Thromb Haemost. 2008, 100: 699-704.
Levick DL, Stern G, Meyerhoefer CD, Levick A, Pucklavage D: Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inf Decis Mak. 2013, 13: 43-10.1186/1472-6947-13-43.
Lipton JA, Barendse RJ, Schinkel AFL, Akkerhuis KM, Simoons ML, Sijbrands EJG: Impact of an alerting clinical decision support system for glucose control on protocol compliance and glycemic control in the intensive cardiac care unit. Diabetes Technol Ther. 2011, 13: 343-349. 10.1089/dia.2010.0100.
Maat B, Rademaker CM, Oostveen MI, Krediet TG, Egberts TC, Bollen CW: The effect of a computerized prescribing and calculating system on hypo- and hyperglycemias and on prescribing time efficiency in neonatal intensive care patients. JPEN J Parenter Enteral Nutr. 2013, 37: 85-91. 10.1177/0148607112444608.
Mann EA, Jones JA, Wolf SE, Wade CE: Computer decision support software safely improves glycemic control in the burn intensive care unit: a randomized controlled clinical study. J Burn Care Res. 2011, 32: 246-255. 10.1097/BCR.0b013e31820aaebf.
Matsumura Y, Yamaguchi T, Hasegawa H, Yoshihara K, Zhang Q, Mineno T, Takeda H: Alert system for inappropriate prescriptions relating to patients' clinical condition. Methods Inf Med. 2009, 48: 566-573. 10.3414/ME9244.
McCoy AB, Waitman LR, Gadd CS, Danciu I, Smith JP, Lewis JB, Schildcrout JS, Peterson JF: A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis. 2010, 56: 832-841. 10.1053/j.ajkd.2010.05.024.
McGregor JC, Weekes E, Forrest GN, Standiford HC, Perencevich EN, Furuno JP, Harris AD: Impact of a computerized clinical decision support system on reducing inappropriate antimicrobial use: a randomized controlled trial. J Am Med Inform Assoc. 2006, 13: 378-384. 10.1197/jamia.M2049.
McKinley BA, Moore FA, Sailors RM, Cocanour CS, Marquez A, Wright RK, Tonnesen AS, Wallace CJ, Morris AH, East TD: Computerized decision support for mechanical ventilation of trauma induced ARDS: results of a randomized clinical trial. J Trauma. 2001, 50: 415-424. 10.1097/00005373-200103000-00004. discussion 425
Mekhjian HS, Kumar RR, Kuehn L, Bentley TD, Teater P, Thomas A, Payne B, Ahmad A: Immediate benefits realized following implementation of physician order entry at an academic medical center. J Am Med Inform Assoc. 2002, 9: 529-539. 10.1197/jamia.M1038.
Meyfroidt G, Wouters P, De Becker W, Cottem D, Van den Berghe G: Impact of a computer-generated alert system on the quality of tight glycemic control. Intensive Care Med. 2011, 37: 1151-1157. 10.1007/s00134-011-2159-7.
Milani RV, Lavie CJ, Dornelles AC: The impact of achieving perfect care in acute coronary syndrome: the role of computer assisted decision support. Am Heart J. 2012, 164: 29-34. 10.1016/j.ahj.2012.04.004.
Milani RV, Oleck SA, Lavie CJ: Medication errors in patients with severe chronic kidney disease and acute coronary syndrome: the impact of computer-assisted decision support. Mayo Clin Proc. 2011, 86: 1161-1164. 10.4065/mcp.2011.0290.
Mullett CJ, Evans RS, Christenson JC, Dean JM: Development and impact of a computerized pediatric antiinfective decision support program. Pediatrics. 2001, 108: E75-10.1542/peds.108.4.e75.
Nash IS, Rojas M, Hebert P, Marrone SR, Colgan C, Fisher LA, Caliendo G, Chassin MR: Reducing excessive medication administration in hospitalized adults with renal dysfunction. Am J Med Qual. 2005, 20: 64-69. 10.1177/1062860604273752.
Niemi K, Geary S, Quinn B, Larrabee M, Brown K: Implementation and evaluation of electronic clinical decision support for compliance with pneumonia and heart failure quality indicators. Am J Health-Syst Pharm. 2009, 66: 389-397. 10.2146/ajhp080143.
Paul M, Andreassen S, Tacconelli E, Nielsen AD, Almanasreh N, Frank U, Cauda R, Leibovici L, Group TS: Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial. J Antimicrob Chemother. 2006, 58: 1238-1245. 10.1093/jac/dkl372.
Peterson JF, Kuperman GJ, Shek C, Patel M, Avorn J, Bates DW: Guided prescription of psychotropic medications for geriatric inpatients. Arch Intern Med. 2005, 165: 802-807. 10.1001/archinte.165.7.802.
Potts AL, Barr FE, Gregory DF, Wright L, Patel NR: Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004, 113: 59-63. 10.1542/peds.113.1.59.
Qian Q, Manning DM, Ou N, Klarich MJ, Leutink DJ, Loth AR, Lopez-Jimenez F: ACEi/ARB for systolic heart failure: closing the quality gap with a sustainable intervention at an academic medical center. J Hosp Med. 2011, 6: 156-160. 10.1002/jhm.803.
Raja AS, Ip IK, Prevedello LM, Sodickson AD, Farkas C, Zane RD, Hanson R, Goldhaber SZ, Gill RR, Khorasani R: Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012, 262: 468-474. 10.1148/radiol.11110951.
Riggio JM, Cooper MK, Leiby BE, Walenga JM, Merli GJ, Gottlieb JE: Effectiveness of a clinical decision support system to identify heparin induced thrombocytopenia. J Thromb Thrombolysis. 2009, 28: 124-131. 10.1007/s11239-008-0279-x.
Rind DM, Safran C, Phillips RS, Slack WV, Calkins DR, Delbanco TL, Bleich HL: The effect of computer-based reminders on the management of hospitalized patients with worsening renal function. Proc Annu Symp Comput Appl Med Care. 1991, 28-32.
Roberts GW, Farmer CJ, Cheney PC, Govis SM, Belcher TW, Walsh SA, Adams RJ: Clinical decision support implemented with academic detailing improves prescribing of key renally cleared drugs in the hospital setting. J Am Med Inform Assoc. 2010, 17: 308-312.
Rood E, Bosman RJ, van der Spoel JI, Taylor P, Zandstra DF: Use of a computerized guideline for glucose regulation in the intensive care unit improved both guideline adherence and glucose regulation. J Am Med Inform Assoc. 2005, 12: 172-180.
Rothschild JM, McGurk S, Honour M, Lu L, McClendon AA, Srivastava P, Churchill WH, Kaufman RM, Avorn J, Cook EF, Bates DW: Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion. 2007, 47: 228-239. 10.1111/j.1537-2995.2007.01093.x.
Roukema J, Steyerberg EW, van der Lei J, Moll HA: Randomized trial of a clinical decision support system: impact on the management of children with fever without apparent source. J Am Med Inform Assoc. 2008, 15: 107-113.
Salinas J, Chung KK, Mann EA, Cancio LC, Kramer GC, Serio-Melvin ML, Renz EM, Wade CE, Wolf SE: Computerized decision support system improves fluid resuscitation following severe burns: an original study. Crit Care Med. 2011, 39: 2031-2038. 10.1097/CCM.0b013e31821cb790.
Schriger DL, Baraff LJ, Buller K, Shendrikar MA, Nagda S, Lin EJ, Mikulich VJ, Cretin S: Implementation of clinical guidelines via a computer charting system: effect on the care of febrile children less than three years of age. J Am Med Inform Assoc. 2000, 7: 186-195. 10.1136/jamia.2000.0070186.
Schriger DL, Baraff LJ, Rogers WH, Cretin S: Implementation of clinical guidelines using a computer charting system. Effect on the initial care of health care workers exposed to body fluids. JAMA. 1997, 278: 1585-1590.
Sellier E, Colombet I, Sabatier B, Breton G, Nies J, Zapletal E, Arlet J-B, Somme D, Durieux P: Effect of alerts for drug dosage adjustment in inpatients with renal insufficiency. J Am Med Inform Assoc. 2009, 16: 203-210. 10.1197/jamia.M2805.
Tafelski S, Nachtigall I, Deja M, Tamarkin A, Trefzer T, Halle E, Wernecke KD, Spies C: Computer-assisted decision support for changing practice in severe sepsis and septic shock. J Int Med Res. 2010, 38: 1605-1616. 10.1177/147323001003800505.
Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW: Effects of computerized physician order entry on prescribing practices. Arch Intern Med. 2000, 160: 2741-2747. 10.1001/archinte.160.18.2741.
Thursky KA, Buising KL, Bak N, Macgregor L, Street AC, Macintyre CR, Presneill JJ, Cade JF, Brown GV: Reduction of broad-spectrum antibiotic use with computerized decision support in an intensive care unit. Int J Qual Health Care. 2006, 18: 224-231. 10.1093/intqhc/mzi095.
Vardi A, Efrati O, Levin I, Matok I, Rubinstein M, Paret G, Barzilay Z: Prevention of potential errors in resuscitation medications orders by means of a computerised physician order entry in paediatric critical care. Resuscitation. 2007, 73: 400-406. 10.1016/j.resuscitation.2006.10.016.
Wang H-Y, Lu C-L, Wu M-P, Huang M-H, Huang Y-B: Effectiveness of an integrated CPOE decision-supporting system with clinical pharmacist monitoring practice in preventing antibiotic dosing errors. Int J Clin Pharmacol Ther. 2012, 50: 375-382. 10.5414/CP201678.
Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, Finkelstein J: The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006, 13: 16-23.
Liu JLY, Wyatt JC: The case for randomized controlled trials to assess the impact of clinical information systems. J Am Med Inform Assoc. 2011, 18: 173-180. 10.1136/jamia.2010.010306.
Finkler SA: The distinction between cost and charges. Ann Intern Med. 1982, 96: 102-109. 10.7326/0003-4819-96-1-102.
Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, Scoville R, Taylor R: Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff (Millwood). 2005, 24: 1103-1117. 10.1377/hlthaff.24.5.1103.
Mongan JJ, Ferris TG, Lee TH: Options for slowing the growth of health care costs. N Engl J Med. 2008, 358: 1509-1514. 10.1056/NEJMsb0707912.
Kellermann AL, Jones SS: What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood). 2013, 32: 63-68. 10.1377/hlthaff.2012.0693.
Himmelstein DU, Wright A, Woolhandler S: Hospital computing and the costs and quality of care: a national study. Am J Med. 2010, 123: 40-46. 10.1016/j.amjmed.2009.09.004.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/13/135/prepub
The authors would like to thank Polina V. Kukhareva, MPH for assisting with the statistical analyses.
This study was funded in part by National Library of Medicine training grant T15-LM07124 (CLF) and the University of Utah Department of Biomedical Informatics (KK and BEB).
KK has or is currently serving as a consultant on CDS to the following organizations: the Office of the National Coordinator for Health IT, Partners HealthCare, RAND Corporation, ARUP Laboratories, Inflexxion, Inc., Intelligent Automation, Inc., McKesson InterQual, and ESAC, Inc. KK receives royalties for a Duke University-owned CDS technology for infectious disease management known as CustomID that he helped develop. KK was formerly a consultant for Religent, Inc. and a co-owner and consultant for Clinica Software, Inc., both of which provide commercial CDS services, including through use of a CDS technology known as SEBASTIAN that KK developed. KK no longer has a financial relationship with either Religent or Clinica Software. The other authors have no competing interests to declare.
All authors contributed to the design of the study. CLF served as the primary literature reviewer. All authors contributed to the data analysis and manuscript preparation. All authors read and approved the final manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.