Skip to content


BMC Medical Informatics and Decision Making

Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Effect of clinical decision rules, patient cost and malpractice information on clinician brain CT image ordering: a randomized controlled trial

BMC Medical Informatics and Decision MakingBMC series – open, inclusive and trusted201818:20

Received: 17 September 2017

Accepted: 5 March 2018

Published: 12 March 2018



The frequency of head computed tomography (CT) imaging for mild head trauma patients has raised safety and cost concerns. Validated clinical decision rules exist in the published literature and on-line sources to guide medical image ordering but are often not used by emergency department (ED) clinicians. Using simulation, we explored whether the presentation of a clinical decision rule (i.e. Canadian CT Head Rule - CCHR), findings from malpractice cases related to clinicians not ordering CT imaging in mild head trauma cases, and estimated patient out-of-pocket cost might influence clinician brain CT ordering. Understanding what type and how information may influence clinical decision making in the ordering advanced medical imaging is important in shaping the optimal design and implementation of related clinical decision support systems.


Multi-center, double-blinded simulation-based randomized controlled trial. Following standardized clinical vignette presentation, clinicians made an initial imaging decision for the patient. This was followed by additional information on decision support rules, malpractice outcome review, and patient cost; each with opportunity to modify their initial order. The malpractice and cost information differed by assigned group to test the any temporal relationship. The simulation closed with a second vignette and an imaging decision.


One hundred sixteen of the 167 participants (66.9%) initially ordered a brain CT scan. After CCHR presentation, the number of clinicians ordering a CT dropped to 76 (45.8%), representing a 21.1% reduction in CT ordering (P = 0.002). This reduction in CT ordering was maintained, in comparison to initial imaging orders, when presented with malpractice review information (p = 0.002) and patient cost information (p = 0.002). About 57% of clinicians changed their order during study, while 43% never modified their imaging order.


This study suggests that ED clinician brain CT imaging decisions may be influenced by clinical decision support rules, patient out-of-pocket cost information and findings from malpractice case review.

Trial registration

NCT03449862, February 27, 2018, Retrospectively registered.


Clinical decision makingCT brainMild head traumaPatient cost informationSimulation researchMalpractice informationCanadian CT head ruleEmergency department cliniciansEvidence-based medicine


Minor head trauma is a common condition treated by emergency department (ED) clinicians [1, 2]. Over one million computed tomography (CT) scans are conducted annually in the United States for these patients, with less than 10% demonstrating findings that change medical management [35]. The need for brain CT in patients with minor head trauma has been thrust into the national spot light because of concerns about the long-term danger of low-dose radiation exposure and the desire to decrease unnecessary health care costs [4, 6].

There are numerous reasons why clinicians may order advanced medical imaging for patients despite clinical evidence suggesting otherwise. These issues include, but are not limited to, unfamiliarity or mistrust of clinical decision rules [5, 7] and estimation of radiation burden [811], fear of malpractice lawsuits [1214], and lack of knowledge of the cost of medical imaging.

With respect to minor head trauma, several validated evidence-based clinical decision rules have been published to help guide clinicians in ordering brain CTs [3, 5, 15]. While these rules differ with respect to sensitivity and specificity, they provide a medically and legally justified pathway to support decision making. Despite this, evidence suggests that many clinicians do not follow clinical decision rules unless reinforced by practice policy and/or integrated into clinical work flow [12, 16].

It is well documented in the literature that “clinician fear of a malpractice suit” exists, and influences clinical decision making [3, 12, 14, 17]. Fear of a lawsuit has led to increased CT ordering, despite the existence and validation of clinical decision rules [16, 18].

Finally, substantial literature reveals that clinicians are unfamiliar with cost implications of testing to patients, payers, and health systems [1921]. However, evidence suggests that clinician awareness of testing cost may influence their decision making, especially toward less-costly testing options [2123]. One abstract even shows that insured medical patients are more likely to receive a brain CT in cases of minor head injuries than patients without health insurance [16].

This study expands our previous research on how information influences clinician decision making in medical image ordering for adult and pediatric patients in the primary care environment [23, 24]. In our earlier work, we incorporated simulation-based methodology to explore how clinician medical image ordering behavior might be influenced by the introduction of clinical decision rules, estimated radiation exposure information, and estimated cost. We analyzed the temporal effect of information presentation to clinicians and the relationship between clinician demographics and medical image ordering behavior.

Goals of the investigation

In our current study, we seek to explore whether presentation of the Canadian CT Head Rule (CCHR), findings from a medical-legal review of malpractice cases related to lack of CT ordering in head trauma victims, and estimated patient out-of-pocket cost for CT imaging might influence clinician ordering in response to clinical vignettes in the simulated emergency department environment. Our goal was to further inform practicing emergency department clinicians and the medical informatics community as they collaborate to build clinical decision support systems that aid clinical decision making.


Study design and setting

This was a multi-center, double-blinded, with balanced ([1:1]) randomization, parallel-group study conducted in the United States with the Departments of Emergency Medicine, Greenville Health System (Greenville, SC) and Emory Healthcare (Atlanta, GA). The study was approved by the Institutional Review Boards of both health systems. There were no changes to the methods after the simulation trial had commenced. As this was a web-accessible simulation study, participants could participate in the study anywhere connectivity allowed. Participants could access the simulation study via a computer or portable device (e.g. tablet) that was internet-accessible.

Selection of participants

Study participation was limited to clinicians, with image ordering capability for patients, and employed in the emergency department of one of the two health systems. Clinicians included attending physicians, resident physicians, physician assistants and nurse practitioners Randomization occurred when the clinicians indicated their occupational profession (see above). Specifically, an electronic balancer allocated each participant (by occupational profession) to one of two groups. The two groups were the LEGAL-COST group or the COST-LEGAL group, both described below, which differed in the temporal order in which they were presented information in the study. The study was double-blinded in that neither the investigators nor the participants were aware of the order in which legal and cost information would be presented. For clarity, both groups received all information which differs from a traditional interventional trial with intervention and control groups.


An electronic email invitation from each department’s academic leadership was sent to all ED clinicians and EM residents. Reminder emails were sent to all at approximately 14 days and 21 days after the initial invitation. Leadership was blinded to participation, assuring voluntary participation without fear of repercussion.


Our intervention consisted of two clinical vignettes, follow-up decision screens and collection of demographic data (Additional file 1). Both clinical vignettes were jointly developed by the emergency medicine and radiology clinician study authors. The cases were designed to provide sufficient information for the clinician to make a determination regarding need for CT imaging. Both cases were designed to fall below the threshold, as outlined in the CCHR, for requiring CT imaging. To further unify the cases both patients had known normal renal function and no allergy to contrast.

Following an electronic-based informed consent process, participants were presented with clinical vignette #1 which described a 58-year-old female (simulation patient) who presents to the Emergency Department after falling on ice hitting her head on the sidewalk (Additional file 1). Following the case presentation, the clinicians were prompted to make a medical image ordering decision for this patient among three options: CT brain (without and with contrast), CT (without contrast), or no imaging.

After their initial imaging decision, the clinicians were presented with the brain CT ordering criteria based on the CCHR [1, 15]. We included a hyperlink to three manuscripts (2 abstracts, 1 full-text) supporting the criteria for clinicians who desired to review further material [15, 25, 26]. The clinicians were provided a first opportunity to modify their initial imaging order.

After their opportunity to modify based on the CCHR, the next topic presented was estimated out of pocket costs regarding the expense of the ED visit with and without imaging. The costs were based on actual local ED charges with the average patient out-of-pocket expense after insurance for a brain CT identified as $843. This was derived from actual data (year 2015) calculated from a Level 1 Trauma Center in the Southeast United States. Following presentation of this information, clinicians were provided a second opportunity to modify their initial imaging order.

The third topic presented was a collection of findings from a malpractice case law review (years 1972–2014) covering situations where the clinician did not order a brain CT for a minor head trauma. Participants were provided with an additional hyperlink to the original published article for review [3]. The malpractice law review was included as an evidence source that addresses “clinician fear of malpractice lawsuit if not ordering a brain CT for minor head trauma”. Following presentation of this information, clinicians were provided a third and final the opportunity to modify their initial imaging order.

After all of the information was presented, constituting the breadth of the intervention, vignette #2 was presented to assess how clinicians might apply their new knowledge for similar scenarios moving forward. Vignette #2 describes a 62- year old female (simulation patient) who had a witnessed slip and fall at home (Additional file 1). Following this case presentation, the clinicians were given a single opportunity to make a medical image ordering decision among three options: CT brain (without and with contrast), CT (without contrast), or no imaging. Thus, a total of 5 clinical decisions were recorded for each participant based on 2 cases.

Demographic and general survey data were gathered from participants. Demographic data included age, gender, role (i.e. practicing clinician, trainee), and years of clinical practice. Two exploratory questions on the participant’s economic attitudes required responses presented on a Likert-like 1–7 scale. The first was “Making better use of my resources makes me feel good”; participants were asked to agree or disagree with the statement. The second was “I believe in being careful in how I spend my money”; participants were asked to agree or disagree with the statement. Both of these questions were derived from the consumer-oriented literature where the focus was on measurement of consumer frugality [27].

Our study concluded with an option for participants to earn a no-cost one Category 1 AMA physician continuing medical education (CME) credit. Participants were redirected from our study website to the Continuing Medical Education Office of the Greenville Health System. There participants reviewed summary material from our study, were provided the opportunity to review full-text reference papers, then complete a post-education survey assessing comprehension to receive CME credit.

Methods and measurements

As presented in Fig. 1, and described above, participants made image ordering decisions at five points in the study. The decisions were made by the participants within the simulation study and recorded in our server-based analysis database in descriptive form (e.g. no imaging, brain CT without contrast) in Microsoft Excel® format. A copy of the database was distributed to researchers at Clemson University where the descriptive data were properly coded for analysis by two researchers (MW, RG). The coded file was then imported into SAS, v.9.4 (Cary, NC).
Fig. 1

Study Flow. Note: Diamonds denote clinician medical imaging decision point

Following the second image ordering decision point the participants were asked how they signed orders and prescriptions (i.e. as a physician, nurse practitioner, physician assistant, or other). Based on the response, participants were stratified by clinician type and balanced randomized [1:1] to one of two parallel arms (i.e. LEGAL-COST group or the COST-LEGAL group). Stratification was used to ensure the clinician types were equally distributed and thereby ensuring the two arms were homogeneous. The difference between the two arms was the temporal presentation of supplemental decision information. The LEGAL-COST group was presented information on malpractice case law then patient out-of-pocket cost information. The COST-LEGAL group was presented information on patient out-of-pocket cost information then malpractice case law information. The participant study flow is shown in Fig. 1.


The primary outcome measure for the study was the physicians’ selection of imaging tests after receiving CCHR information, malpractice case review information, and patient out-of-pocket expense information. We also measure the physicians’ selection of imaging tests immediately following clinical vignette #1 and after presentation of clinical vignette #2.


Data were recorded in our intervention server, housed at the National Institutes of Health, in a Microsoft Excel spreadsheet. The data were downloaded and analyzed using SAS, ver. 9.4 (Cary, NC). In our study, because clinicians’ ordering image vs. not ordering image was the main comparison of interest, CT brain modalities (with or without contrast) were grouped together and compared with the no imaging order group. To compare the clinicians’ change in image ordering, McNemar test was employed and a multiple comparison adjustment was performed using Bonferroni correction. For the comparison of the demographic or other professional characteristics among different clinician groups, Chi-square test was employed to compare the proportions of categorical variables; if more than 20% of the cells had sample size less than 5, Fisher Exact test was applied instead. Analysis of Variance (ANOVA) test was used to compare the mean (standard deviation) of continuous variables.

A sample size of 155 will achieve 80% power to detect a difference in the proportion of selecting CT imaging if 30% of clinicians choose CT order in the absence of evidence and change to no imaging order when the evidence is presented and 15% of clinicians choose no imaging order in the absence of evidence and changed to CT order when the evidence is presented.


Characteristics of study subjects

Participants in the study included 150 emergency medicine physicians, 12 nurse practitioners, and 5 physician assistants allocated to one of 2 groups; the LEGAL-COST group (n = 82) and the COST-LEGAL group (n = 85). There was one clinician missing the initial imaging order and another five clinicians missing the 3rd order. These participants were included in the study sample and analyzed. All other data items were complete.

Approximately 90% of the participants were practicing clinicians with 10% being trainees; balanced among the 2 groups. Gender was balanced with slightly more males than females participating in both groups. Approximately two thirds of participants had > 5 years of clinical practice experience; included in the group were the > 40% with > 10 years of clinical practice. Just over half of the participants were ≤40 years of age (Table 1). There were no significant differences between the 2 groups with respect to demographics (Table 1) or initial image ordering decisions to clinical vignette #1 (Table 2).
Table 1

Participant demographics


LEGAL-COST group (n = 82)

COST-LEGAL group (n = 85)


Both groups (n = 167)

Clinician type





73 (89.0%)

77 (90.6%)


150 (89.8%)

 Nurse Pract.

6 (7.3%)

6 (7.1%)


12 (7.2%)

 Physician Assist.

3 (3.7%)

2 (2.4%)


5 (3.0%)





 Practicing clinician

73 (89.0%)

77 (90.6%)


150 (89.8%)


9 (11.0%)

8 (9.4%)


17 (10.2%)

Years of Practice




 Less than 5 years

32 (39.0%)

24 (28.2%)


56 (33.5%)

 5–10 years

17 (20.7%)

23 (27.1%)


40 (24.0%)

 More than 10 years

33 (40.3%)

38 (44.7%)


71 (42.5%)






49 (59.8%)

43 (50.6%)


92 (55.1%)


33 (40.2%)

42 (49.4%)


75 (44.9%)

Age (year)




  < 30

6 (7.3%)

10 (11.8%)


16 (9.6%)


40 (48.8%)

34 (40.0%)


74 (44.3%)


14 (17.1%)

21 (24.7%)


35 (20.9%)


22 (26.8%)

20 (23.5%)


42 (25.2%)

*P-value was calculated using Chi-square test except for the comparison of clinician type, P-value was calculated using Fisher exact test (more than 20% of cells with sample size less than 5)

Notes: LEGAL-COST indicates exposure to legal then cost information; COST-LEGAL indicates exposure to cost then legal information

Table 2

Proportion of computed tomography ordering in clinician medical image order decisions


Case #1

Case #2


Initial choice (Initial order)

CCHR (2nd order)

Legal (3rd order for Legal-Cost group; or 4th order for Cost-Legal group)

Cost (4thorder for Legal-Cost group; or 3rd order for Cost-Legal group)

Final choice (5th order)

Both groups

n (%)

116 (66.9%)

76 (45.8%)

85 (52.5%)

80 (47.9%)

109 (65.7%)

(n = 167)



Initial vs. CCHR: 0.002

Initial vs. Legal: 0.002

Initial vs. Cost: 0.002

Initial vs. Final: 1.00


n (%)

56 (68.3%)

36 (43.9%)

38 (49.4%)

39 (47.6%)

50 (61.0%)

(n = 82)



Initial vs. CCHR: 0.003

Initial vs. Legal: 0.05

CCHR vs. Legal: 0.8

Initial vs. Cost: 0.01

Legal vs. Cost: 1.00

Initial vs. Final: 1.00


n (%)

60 (71.4%)

40 (47.6%)

47 (55.3%)

41 (48.2%)

59 (70.2%)

(n = 85)



Initial vs. CCHR: 0.002

Initial vs. Legal: 0.05

Cost vs. Legal: 1.00

Initial vs. Cost: 0.002

CCHR vs. Cost: 1.00

Initial vs. Final: 1.00

Abbreviations: CCHR Canadian CT Head Rule

*P-values were calculated using McNemar test and a multiple comparison adjustment was performed using Bonferroni correction

aLEGAL-COST indicates exposure to legal then cost information

bCOST-LEGAL indicates exposure to cost then legal information

Main results

Clinician image ordering decisions are presented in Table 2. For clinical vignette #1, 116 of the 167 participants (66.9%) initially order a CT image. After presentation of the CCHR, with option to access abstracts or full-text manuscript supporting the rule, the number of clinicians ordering a CT image dropped to 76 (45.8%), which represents a 21.1% statistically significant (P = 0.002) reduction in CT ordering in favor of no medical imaging. It is noteworthy that only 7.8% (n = 13) of the 167 participants accessed either ≥1 abstract (n = 6) or full-text manuscript (n = 7) indexed to the decision screen prior to making their imaging decision.

Following the CCHR-related imaging decisions, clinicians were presented with either LEGAL (LEGAL-COST group) or COST (COST-LEGAL group) information. In the LEGAL-COST group, after receiving information about malpractice judgements against clinicians who did not order a CT of the brain in mild trauma cases, the number of clinicians ordering a CT image was 38 (49.4%) and the difference was significant (P = 0.05) when compared to their initial order, and but not significant when compared to their previous image order that followed presentation of the CCHR. After presentation of patient out-of-pocket expense information, the number of clinicians ordering a CT was 39 (47.6%) where the difference was significant (P = 0.01) when compared with their initial order but not significant when compared to their previous image order that followed presentation of malpractice judgement information.

In the COST-LEGAL group, after presentation of patient out-of-pocket expense information, the number of clinicians ordering a CT image was 41 (48.2%) and the difference was significant (P = 0.002) when compared to their initial order, but not significant when compared to their previous image order that followed presentation of the CCHR. After receiving information about malpractice judgements, the number of clinicians ordering a CT was 47 (55.3%) and the difference was significant (P = 0.05) when compared with their initial order but not significant when compared to their previous image order that followed presentation of patient out-of-pocket expense information.

When comparing the clinician’s initial decision regarding medical imaging ordering in response to clinical vignette #1 to their decision in response to clinical vignette #2, the differences were not significant. It is noteworthy that the clinical scenarios in both vignette’s, albeit not precisely the same case, were within the criteria for no medical imaging necessary when applied to the CCHR. In clinical vignette #2, approximately two thirds of clinicians ordered a CT for their patients which is consistent with their response to clinical vignette #1 (Table 2).

Comparing the initial medical imaging decision and the three subsequent imaging decision options for clinical vignette #1, 49 (30.4%) of clinicians always ordered a CT image, 20 (12.4%) of clinicians never ordered a CT image, and 98 (57.2%) changed their CT image order at least once (Table 3). Of 36 participants who changed their imaging order more than once, 27 (16.2%) changed their order at least twice, 6 (3.6%) changed their order at least three times, and 3 (1.8%) modified their imaging order four times (data not shown in the table).
Table 3

Clinician demographics and imaging decisions (by CT ordering behavior group)


Clinicians who always ordered CT (n = 49)

Clinicians who never ordered CT (n = 20)

Clinicians who changed CT order (n = 98)


Total (n = 167)

Provider type, %
















Gender, %
















Years of clinical practice, %




  < 5












  > 10






Resource question, mean (SD)

6.00 (1.13)

6.45 (0.69)

6.17 (0.95)


6.16 (0.98)

Money question, mean (SD)

5.73 (1.18)

6.30 (0.73)

6.00 (1.06)


5.96 (1.08)

aChi-square test was employed to compare the proportion of categorical variable; Analysis of Variance (ANOVA) test was employed to compare the means of continuous variable

Notes: CT, computed tomography; NP, nurse practitioner; PA, physician assistant; SD, standard deviation; mean response to resource question and money question based on 1–7 Likert-like scaled response

The majority of clinicians who always ordered CT imaging had accumulated > 10 years in clinical practice and scored lower on both the use of resource and care with money questions as compared with clinicians never ordering CT or those that changed their imaging order, but the difference did not achieve statistical significance (Table 3). In contrast, the majority who never ordered a CT image had accumulated < 10 years in clinical practice and scored higher on both the use of resource and care with money questions as compared to clinicians who always ordered a CT image or those that changed their imaging order (Table 3).

When aggregating data from both groups comparing CT image ordering and response to the “use of resource question”, those participants who indicated a 7 score (strongly agree) on the “use of resource question” did not have a significantly different CT ordering behavior from those with a question score of 1–6 for clinical vignette #1. However, this shifted in clinical vignette #2 where the two groups were significantly different (p = 0.02); those scoring a 7 were more likely not to order a CT image while those scoring 1–6 were more likely to order CT image (Table 4). When aggregating data from both groups comparing CT image ordering and response to the “being careful in spending money” question, there were no statistical differences recognized (Table 4).
Table 4

Comparison of clinician response to attitudinal survey questions on resource utilization and spending money to CT ordering behavior


Case #1 (initial order)


Case #2 (5thorder)


Survey question


No image



No image


Better use of my resources makes me feel good

 strongly agree (7)

51 (44.4%)

23 (46.0%)


42 (38.5%)

33 (57.9%)


 < strongly agree (1–6)

64 (55.7%)

27 (54.0%)


67 (61.5%)

24 (42.1%)

I believe in being careful in how I spend my money

 strongly agree (7)

40 (34.78%)

24 (48.0%) 26


38 (34.9%)

26 (45.6%)


 < strongly agree (1–6)

75 (65.22%)



71 (65.1%)

31 (54.4%)

*P-value was calculated from Chi-square test


Our study was based in a simulation environment. The sounds, interruptions, clinical pressures, and triage triggers of emergency department workflow were absent. Furthermore, participants were not responsible for following policies or other considerations in the clinical scenarios. It is possible that the clinicians may have manipulated their medical imaging decisions in search of the “right answer” and did not truly consider the implication of new information in their clinical care. The clinicians may not have conceptualized that the decision screens and variables (i.e. evidence, cost, legal) were fully applicable in clinical scenario #2 in the same manner as scenario #1. This study includes participants from 2 southern US states that have different state tort and liability malpractice reforms and environments that may not be applicable to other US practice locations.


Evidence from our simulation-based study suggests that ED clinician decision making may be influenced by clinical decision rules, patient out-of-pocket cost information, and findings from malpractice case review.

Some (49 of 167; 29.3%) selected CT imaging for their simulated patient and were impervious to all information presented. It is possible that these clinicians believed the CT scan was the best test for the patient and were not influenced to change their ordering behavior. The majority of these clinicians were male, recorded > 10 years of clinical practice, and scored the lowest response on our use of resource and care in use of money questions.

The largest group of clinicians (98 of 167; 58.7%) modified their medical imaging order at least once for clinical vignette #1 when presented clinical decision rule, cost, and malpractice care review information. Of these clinicians about one third (36 of 98; 36.7%) modified their imaging order more than once.

An unexpected finding in our research was clinician medical image ordering in response to clinical vignette #2. The second case was similar to the first clinical vignette in that in neither case, when applied to the clinical decision rule, would a CT image be indicated for the patient. It appears that when presented with a new case the clinicians reverted to their original medical image ordering behavior. This may be due to the limitations of a simulation-based study or possibly due to their interpretation of vignette #2 differently than anticipated. Our study was not designed or powered to address why clinicians did not change their ordering behavior in response to clinical vignette #2.


Our research contributes to the body of evidence on clinician decision making and key information that may influence medical image ordering behavior. The findings suggest that clinicians may respond to key information if presented within the context of their clinical workflow. In our study, we provided links to source documents and evidence supporting the information. Clinicians are typically decisive in their decision making with their patients. Medical informaticians and health information technologists should be extremely thoughtful in their information presentation when designing clinical decision support systems and other tools.



American Medical Association


Analysis of variance


Canadian CT Head Rule


Continuing medical education


Computed tomography


Emergency department


Emergency medicine



The authors acknowledge the professional and much appreciated support of Clemson University Research Associate Karen Edwards, MS who coordinated the bulk of logistics and communication related to Institutional Review Board approval at each site.


Grant support (2015001097) for the research was provided by the Health Sciences Center, Greenville Health System, Greenville, SC, USA. The funder had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript. This research was supported by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM) and Lister Hill National Center for Biomedical Communications (LHNCBC).

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. The intervention slide set used in this simulation research is attached as Additional file 1.

Authors’ contributions

RG, RP, SL, DW and ZC conceived the study and designed the trial. RP, SL, DW, and ZC developed the clinical vignettes and supporting data, then verified the clinical vignettes fit within the decision rule parameters. LZ and MW provided statistical powering and an analysis strategy for the study, led data collection from the portal, then conducted data analysis. Both LZ and MW also assisted RG on interpretation of findings. RG, RP, SL, PF, DW, and FL supervised the conduct of the trial and data collection. PF and FL designed the intervention interface, information security, and data retrieval strategy. RP and DW undertook recruitment of clinicians and data collection. RG drafted the article, and all authors contributed substantially to its revision and approved final version. RG take responsibility for the paper as a whole.

Ethics approval and consent to participate

This simulation study was approved by the Institutional Review Boards of Clemson University (Clemson, SC), the Greenville Health System (Greenville, SC), and Emory Healthcare (Atlanta, GA)). The National Institutes of Health Office of Human Subjects Research designated this research as exempt from Institutional Review Board review.

This was a web-based simulation study and the informed consent was built into the intervention. It was included in the intervention slides (Additional file 1) just after the welcome. It was an electronic consent document where the participant was required to accept to proceed with the study. If he/she did not consent the system thanked them for their interest but did not let them proceed.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

Department of Public Health Sciences, Clemson University, Clemson, USA
Department of Emergency Medicine, Greenville Health System, Greenville, USA
Department of Radiology, Greenville Health System, Greenville, USA
Department of Emergency Medicine, Emory University, Atlanta, USA
Lister Hill National Center for Biomedical Communication, National Library of Medicine, Bethesda, USA


  1. Bouida W, Marghli S, Souissi S, Ksibi H, Methammem M, Haguiga H, Khedher S, Boubaker H, Beltaief K, Grissa MH, et al. Prediction value of the Canadian CT head rule and the New Orleans criteria for positive head CT scan and acute neurosurgical procedures in minor head trauma: a multicenter external validation study. Ann Emerg Med. 2013;61(5):521–7.View ArticlePubMedGoogle Scholar
  2. Stiell IG, Lesiuk H, Wells GA, McKnight RD, Brison R, Clement C, Eisenhauer MA, Greenberg GH, MacPhail I, Reardon M, et al. The Canadian CT head rule study for patients with minor head injury: rationale, objectives, and methodology for phase I (derivation). Ann Emerg Med. 2001;38(2):160–9.View ArticlePubMedGoogle Scholar
  3. Lindor RA, Boie ET, Campbell RL, Hess EP, Sadosty AT. Failure to obtain computed tomography imaging in head trauma: a review of relevant case law. Acad Emerg Med. 2015;22:1493–8.View ArticlePubMedGoogle Scholar
  4. Melnick ER, Szlezak CM, Bentley SK, Dziura JD, Kotlyar S, Post LA. CT overuse for mild traumatic brain injury. Jt Comm J Qual Patient Saf. 2012;38:483–9.View ArticlePubMedGoogle Scholar
  5. Morton MJ, Korley FK. Head computed tomography use in the emergency department for mild traumatic brain injury: integrating evidence into practice for the resident physician. Ann Emerg Med. 2012;60(3):361–7.View ArticlePubMedGoogle Scholar
  6. Sharp AL, Nagaraj G, Rippberger EJ, Shen E, Swap CJ, Silver MA, McCormick T, Vinson DR, Hoffman JR. Computed tomography use for adults with head injury: describing likely avoidable emergency department imaging based on the Canadian head CT rule. Acad Emerg Med. 2017;24(1):22–30.View ArticlePubMedGoogle Scholar
  7. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PAC, Rubin HR. Why don't physicians follow clinical practice guidelines?: a framework for improvement. J Am Med Assoc. 1999;282(15):1458–65.View ArticleGoogle Scholar
  8. Borgen L, Stranden E, Espeland A. Clinicians' justification of imaging: do radiation issues play a role? Insights Imaging. 2010;1:193–200.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Arslanoglu A, Bilgin S, Kubali Z, Ceyhan MN, Ilhan MN, Maral I. Doctors' and intern doctors' knowledge about patients' ionizing radiation exposure doses during common radiological examinations. Diag Interv Radiol. 2007;13:53–5.Google Scholar
  10. Lee RK, Chu WCW, Graham CA, Rainer TH, Ahuja AT. Knowledge of radiation exposure in common radiological investigations: a comparison between radiologists and non-radiologists. Emerg Med J. 2012;29:306–8.View ArticlePubMedGoogle Scholar
  11. Puri S, Hu R, Quazi RR, Voci S, Veazie P, Block R. Physicians' and midlevel providers' awareness of lifetime radiation-attributable cancer risk associated with commonly performed CT studies: relationship to practice behavior. AJR Am J Roentgenol. 2012;199(6):1328–36.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Derrick BJ, Quaas JW, Wiener DE, Polan RM, Chawla M, Mazori D, Newman DH. Head CT utilization for minor head injury: what motivates patients to present to the emergency department for evaluation, and why to emergency physicians choose to evaluate them with CT? Ann Emerg Med. 2014;58(4S):S248.Google Scholar
  13. Lehnert BE, Bree RL. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support. J Am Coll Radiol. 2010;7:192–7.View ArticlePubMedGoogle Scholar
  14. McBride JF, Wardrop RMI, Paxton BE, Mandrekar J, Fletcher JG. Effect on examination ordering by physician attitude, common knowledge, and practice behavior regarding CT radiation exposure. Clin Imaging. 2012;36:455–61.View ArticlePubMedGoogle Scholar
  15. Harnan SE, Pickering A, Pandor A, Goodacre SW. Clinical decision rules for adults with minor head injury: a systematic review. J Trauma. 2011;71(1):245–51.View ArticlePubMedGoogle Scholar
  16. Dickinson J, Fortin E, Fisher J, Qiu S, Irvin CB. Privately insured medical patients are more likely to have a head CT. Ann Emerg Med. 2014;64(4S):S88.View ArticleGoogle Scholar
  17. Rosenthal DI, Weilburg JB, Schultz T, Miller JC, Nixon V, Dreyer KJ, Thrall JH. Radiology order entry with decision support: initial clinical experiences. J Am Coll Radiol. 2006;3(10):799–806.View ArticlePubMedGoogle Scholar
  18. Rohacek M, Albrrecht M, Kleim B, Zimmerman H, Exadaktylos A. Reasoons for ordering computed tomorgraphy scans of the head in patients with minor brain injury. Inquiry. 2012;43:1415–8.Google Scholar
  19. Schilling UM. Cost awareness among Swedish physicians working at the emergency department. Eur J Emerg Med. 2009;16(3):131–4.View ArticlePubMedGoogle Scholar
  20. Allan GM, Lexchin J. Physician awareness of diagnostic and nondrug theraputic costs: a systematic review. Int J Technol Assess Health Care. 2008;24(2):158–65.View ArticlePubMedGoogle Scholar
  21. Sehgal RT, Gorman P. Internal medicine physicians' knowledge of health care charges. J Grad Med Educ. 2011;3(2):182–7.View ArticleGoogle Scholar
  22. Covington MF, Agan DL, Liu Y, Johnson JO, Shaw DJ. Teaching cost-conscious medicine: impact of a simple educational intervention on appropriate abdominal imaging at a community-based teaching hospital. J Grad Med Educ. 2013;5(2):284–8.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Gimbel RW, Fontelo P, Stephens MB, Olsen CH, Bunt C, Ledford CJW, Cook CAL, Liu F, Burke HB. Radiation exposure and cost influence physician medical image decision making: a randomized controlled trial. Med Care. 2013;51(7):628–32.View ArticlePubMedGoogle Scholar
  24. Bunt CW, Burke HB, Towbin AJ, Hoang A, Stephens MB, Fontelo P, Liu F, Gimbel RW. Point-of-care estimated radiation exposure and imaging guidelines can reduce pediatric radiation burden. J Am Board Fam Med. 2015;28(3):343–50.View ArticlePubMedGoogle Scholar
  25. Stiell IG, Wells GA, Vandemheen K, Clement C, Lesiuk H, Laupacis A, McKnight RD, Verbeek R, Brison R, Cass D, et al. The Canadiant CT head rule for patients with minor head injury. Lancet. 2001;357(9266):1391–6.View ArticlePubMedGoogle Scholar
  26. Stiell IG, Clement C, Rowe BH, Schull MJ, Brison R, Cass D, Eisenhauer MA, McKnight RD, Bandiera G, Holroyd B, et al. Comparison of the Canadian CT head rule and the New Orleans criteria in patients with minor head injury. JAMA. 2005;294(12):1511–8.View ArticlePubMedGoogle Scholar
  27. Lastovicka JL, Bettencourt LA, Hughner RS, Kuntze RJ. Lifestyles of the tight and frugal: theory and measurement. J Consum Res. 1999;26(1):85–98.View ArticleGoogle Scholar


© The Author(s). 2018