- Open Access
Improved preventive care clinical decision-making efficiency: leveraging a point-of-care clinical decision support system
BMC Medical Informatics and Decision Making volume 21, Article number: 315 (2021)
Electronic medical records are widely used in family practices across Canada and can improve health outcomes. However, recent reports indicate that physicians using electronic medical records work longer and have less direct patient contact which may contribute to burnout. Therefore, new and innovative digital tools are essential to reduce physician workloads and improve patient-physician interaction to address physician burnout. The objective of this study was to assess the efficiency and accuracy of clinical decision-making when using a new preventive care point-of-care clinical decision support system (CDSS). An estimate of the potential annual time savings was also determined. This study also assessed physician reported perceived usefulness and ease of use of the CDSS.
Quantitative and qualitative data were collected during this study. Each participant evaluated two simulated patient charts and identified which preventive care metrics were due. The participants recorded their decisions and the time required to assess each chart. Participants then completed a Technology Acceptance Model survey regarding the perceived usefulness and ease of use of the CDSS, which included qualitative feedback. The amount of time saved was determined and participants’ clinical decision-making accuracy was scored against current Canadian preventive care guidelines. The number of preventive care specific visits completed per year was determined using clinic billing data.
The preventive care CDSS saved an average of 195.7 s of chart review time (249.5 s vs 445.2 s; P < 0.001). A total of 1520 preventive visits were performed at Primrose and Bruyère Family Medicine Centres. Extrapolated across the organization, implementation of the new tool could save 82.6 h per year. Decision-making accuracy was not affected by the new tool (78.4% vs 80.9%, P > 0.05). Participants rated the perceived ease of use and usefulness to be very high.
New digital tools may reduce providers’ workload without impacting clinical decision-making accuracy. Participants indicated that the preventive care CDSS was useful and easy to use. Further software development and clinical studies are required to further improve and characterize the effect this new CDSS has when implemented in clinical practice.
Preventive care services effectively reduce morbidity, mortality, and overall health care expenditures. In Canada, preventive care services are largely delivered by primary care providers, focusing on screening and counselling on lifestyle risk factors, infectious diseases, metabolic disorders, immunizations, and cancer [1, 2]. Many Canadian primary care physicians perform preventive care services through opportunistic or organized approaches. Opportunistic preventive care occurs as an add-on when patients present for non-preventive care services. Organized preventive care occurs locally within primary care practices via dedicated preventive care visits  or regionally/nationally through outreach to individuals , like Cancer Care Ontario’s mailed cancer screening letters .
Canada Health Infoway—a federally funded non-profit aiming to digitally transform Canada’s health care system—has demonstrated that EMR use improves workflow efficiencies, cost efficiencies, health outcomes, patient safety, and interprofessional communication over paper records . EMRs can make relevant patient data readily available to estimate disease risk  and improve preventive care service delivery through alerts that identify patients due for screening . However, many providers override and ignore alerts due to “alert fatigue” where high volumes of irrelevant alerts limit usefulness . As well, only 3–10% of providers use these advanced EMR features , which further limits the potential benefits of EMR use on preventive care service delivery.
These purported benefits are especially important for preventive care services since primary care providers do not have sufficient time to complete all recommended services . Yet, recent time motion studies have shown that EMR use consumes significant amounts of clinical time, detracting from patient interaction [10, 11] and higher EMR use is correlated with physician burnout . Burnout may arise from low professional satisfaction caused by poor EMR usability, workflows, and interrupted patient-physician interactions . However, another report suggests negligible impacts on clinician working time  and instead frequent task switching may be a major contributor toward perceptions of inefficiency and disruption .
With over 85% of Canadian physicians using EMRs in their clinical practice  it is therefore prudent to develop better EMR point-of-care tools that facilitate preventive care service deliver while not contributing to alert fatigue or task switching and improving efficiency. Accordingly, this study aims to assess the efficiency, provider decision-making accuracy, ease of use, and usefulness of a proof-of-concept, point-of-care, preventive care CDSS. This preventive care CDSS automatically summarizes recommended preventive care services on a single screen within the EMR, which may be easily opened during dedicated preventive care visits.
This study followed a pre-post test design in an artificial clinical setting. Participants assessed two artificial patient records and made clinical decisions regarding which preventive care services were due for each patient. One chart review was completed with and one without the preventive care, point of care CDSS. Clinical decision-making accuracy and the time to make these decisions were assessed. Participants also completed a Technology Acceptance Model (TAM) to understand their experience using the automated preventive care CDSS . The EMR used was PS Suites .
Participants were recruited from Bruyére Family Health Team in Ottawa, Ontario (n = 18) and completed this study in October 2018. Family Health Teams are interdisciplinary primary care clinics in Ontario, which comprise of physicians, nurses, nurse practitioners, social workers, dieticians and other allied health staff . Our clinic is distributed across two locations in Ottawa. All clinical staff were invited to participate via emailed instructions and data collection forms (Additional files 1–3). All resident physicians were recruited by scheduling time for participation during a lunch time teaching session.
Point of care CDSS design and artificial chart setup
The preventive care CDSS was designed by SL—a final year family medicine resident physician—within PS Suites EMR . This CDSS allows automated text insertion into a clinical note to facilitate clinical documentation and pulls relevant stored patient data, such as demographics, dates and results of laboratory tests, and patient appointment data . This CDSS was designed following a published preventive care encounter template  and the Canadian Task Force on Preventive Health Care guidelines  were used to guide development of the CDSS. The CDSS automatically generated a summary note within the EMR and displayed the data for the primary care provider. This summary indicated when each preventive screening test was last completed and the result of each test (when available in EMR’s structured data fields).
Next, two artificial patient charts were created. Both charts represented 65-year-old females and did not contain real patient data. The charts were filled with preventive care data including pap tests, mammograms, type 2 diabetes screening, bone mineral densities, colon cancer screening, and hypertension screening. A 65-year-old female was selected since this age group has the most recommended preventive care services [1, 20]. Irrelevant non-preventive care encounters, laboratory tests, and diagnostic imaging were added to the charts. This irrelevant data was added to create a more complex and robust artificial record, since providers must find relevant clinical data when performing preventive care. The dates of all artificial data (relevant and irrelevant) were varied to avoid participants detecting patterns between the two records, but should have resulted in the same clinical decisions across the two charts.
Participants reviewed the two artificial patient charts and recorded the preventive care metrics that were completed, which tests were due, and the time to complete each chart review. Participants recorded this information on the data collection form (Additional file 2), which also included tests that would not be indicated for 65-years-old female to ensure providers made clinically appropriate decisions. This exercise occurred during in a single session and participants all received the same instruction (no randomization to different groups) due to restricted timelines to complete this resident physician project.
Participants reviewed the first chart and did not have access to the CDSS (usual method). A non-functional version of the CDSS was inserted into the chart that did not extract the patient’s latest preventive care data. Providers had to manually review the chart to complete the exercise and record their findings.
Next, participants reviewed the second chart using the preventive care CDSS (Additional files 4 and 5). Providers could use the summary or search the chart for required information to complete the exercise and record their findings (new method).
Immediately after completing the exercise, participants completed a paper-based TAM survey (Additional file 3) to report the perceived usefulness and ease of use of the CDSS. Then the primary investigator assessed the accuracy of participants’ decision-making by marking participants’ recorded responses from the data collection forms (Additional file 2) according to Canadian preventive care guidelines. The primary investigator performed a thematic analysis of the free-text feedback, assessing the content of positive and negative feedback and categorizing each comment accordingly.
Next, annual time savings during dedicated preventive care visits was determined. The number of dedicated preventive care visits was determined using physician billing codes from the previous year (Dec 9, 2017 to Dec 9, 2018) at Bruyére Family Health Team. Billing codes for dedicated preventive care visits and were summated. This total number of preventive care visits was multiplied by the observed amount of time saved when using the CDSS.
To help explain the time differences, process mapping was used to estimate the minimum number of actions required to complete each simulated chart assessment. Counted actions included keystrokes, clicks, scrolling through search results, reading results, interpreting results, and decision-making. This was mapped by SL only due to project timelines and the time to complete each step was not recorded. SL is a resident physician and regularly performs dedicated preventive care visits. The actions participants performed during their reviews could not be captured based on time and personnel limitations.
The results of 17 participants were included in the statistical analysis. One participant was excluded due to incomplete data. Both the accuracy of clinical decision-making and the time required to complete the chart review were analyzed with a paired, two-tailed T test. Potential time savings for preventive care visits were calculated by multiplying the number of preventive care visits completed in the last year by the mean time savings observed. Minimum number of actions required were counted from the process mapping. Perceived usefulness and ease of use scores were calculated by determining both the median and mean. Ninety-five percent confidence intervals were calculated for the mean clinical decision-making accuracy, time savings, perceived usefulness, and ease of use scores. All statistical analysis was completed with Microsoft Excel.
This study was submitted to the University of Ottawa Department of Family Medicine for review and approval prior to commencing. The Department of Family Medicine determined this project was deemed to be quality improvement and therefore a Research Ethics Board application was not required. The Department of Family Medicine’s scholarly project guidelines were followed throughout this project. Participants were informed of the study purpose before verbally consenting to participate.
Of the 86 healthcare providers at the Bruyére Family Health Team, 18 participated in the study (20.6%). The 17 participants that were included in the analysis consisted of: 1 medical student, 11 residents, 1 nurse practitioner, 4 staff physicians. No registered nurses participated (Table 1).
Mean time to complete chart review
The mean time to complete the chart review with the CDSS was significantly faster than manual chart review (249.5 ± 45.6 s; 95% CI vs 445.1 ± 75.0 s; 95% CI. P < 0.001; Fig. 1). The mean time saved during chart review was 195.6 ± 60.4 s; 95% CI.
Mean clinical decision-making accuracy
Participants’ decision-making accuracy did not significantly differ between the CDSS and manual chart review (78.4 ± 7.6%; 95% CI vs 80.9 ± 7.0%; 95% CI. P = 0.41; Fig. 2).
Potential time savings for dedicated preventive care visits at Bruyére Family Health Team
In the 365 days preceding this study, a total of 1520 dedicated preventive care visits were completed by Bruyére Family Health Team. The estimated annual clinical time savings from using the preventive care CDSS for these visits could be 82.6 ± 25.5 h per year (95% CI).
Minimum number of actions required to complete chart review
The process mapping demonstrated manual chart review required 128 actions. In comparison, the CDSS required 58 actions to complete the chart review. Therefore, participants may have performed 70 fewer actions using the preventive care CDSS, explaining some of the time savings.
Perceived usefulness and ease of use
Participants indicated that the new preventive care tool was both useful (Table 2) and easy to use (Table 3) as represented by median scores of 7/7 across all TAM metrics. The mean scores of perceived usefulness and ease of use were also very high (Tables 2 and 3). The usefulness statement that received the highest score was that “Using this interface would make it easier to do my job” (mean 6.69 ± 0.38; 95% CI out of 7). The ease of use statement that received the highest score was that “Learning to operate this interface would be easy for me” (mean 6.50 ± 0.43; 95% CI out of 7).
Qualitative feedback indicated that the preventive care CDSS was perceived to be efficient (70.6%), comprehensive (64.7%), organized (58.8%), and assisted clinical decision making (41.1%) (Table 4). Participants expressed also expressed that the use of “months since last done” was difficult to interpret (41.1%), participants required prior knowledge of preventive care guidelines (23.5%), uncertainty regarding the accuracy of information presented in the CDSS as some participants wanted to see the original reports (17.6%), loss information depth as some measures like pneumococcal vaccination did not show the different types of vaccines (11.8%), and that it was cumbersome to use (11.7%) (Table 5).
This study demonstrates that preventive care CDSSs may save clinical time, can be easy to use, and can be useful for clinical practice. A meta-analysis of preventive care CDSSs has already established that CDSSs improve screening, but most CDSSs are not integrated into EMRs and focus on effectiveness instead of workflows and usability . The guiding principle behind this work is the Quadruple Aim for healthcare optimization—improving health outcomes, value, patient experience, and provider experience . This study aims to assess both workflow (task efficiency) and usability (ease of use and usefulness) to assess value and provider experience, respectively.
Value is defined as cost per capita  and improved task efficiency can be directly related to reduced cost per capita . Measuring the dollars per capita cost savings is outside the scope of this study. Instead, this study assessed task efficiency showing improvements through preventive care CDSS, which we defined as time savings and fewer actions to complete the given task. This study showed that this preventive care CDSS may provide improved task efficiency, which was accomplished by automatically extracting relevant preventive care data in a single paged summary. This study is unique in that task efficiency relating to CDSS is under-reported in the literature . Extrapolating this time saving across an entire year for an organization could save many hours of work. Physicians could then use this saved time to provide additional healthcare services , reclaim lost patient interaction time , or tackle copious other EMR tasks . Importantly this improved task efficiency was achieved without impacting decision-making accuracy. Therefore, we suggest that clinical usage of this or similar CDSSs may improve the value of preventive care service delivery by improving task efficiency.
Improving physician EMR efficiency is important since physician burnout is negatively correlated with EMR adoption [10, 12, 24]. Although it is unclear why this association exists, EMR workflows may be the primary culprit. Time motion studies have inconsistently demonstrated the effect of EMR usage on clinician working hours, with reports indicating no impact  and others significant extension . It is possible instead that high rates of task switching contribute to perceptions of inefficiency . This CDSS summarizes all data in a single screen that is integrated into the EMR, which we propose may reduce task switching and cognitive loads, leading to high perceived usefulness and ease of use scores. Highly structured EMRs have many data entry fields, buttons, and drop downs, requiring additional user input processes. Each additional action may exacerbate inefficiencies, extend computer interaction times, and reduce direct physician–patient interaction . Accordingly, leveraging this structured data in useful ways via CDSSs may limit the number of user input actions, which do not add value to patient care. In this case those actions are primarily manual chart searches.
In addition to saving time, the use of automated CDSSs may improve the provider experience of care delivery. Participants reported very high usefulness and ease of use scores, which is one measure of provider experience. Participants reported that the CDSS was efficient, comprehensive, organized, and helpful for clinical decision-making. These findings align with the key factors that providers desire in electronic tools including: useful features, easy to use interfaces, efficient designs, valuable information, and practical workflows .
Further, this study has also identified several areas for CDSS improvement. Participants did not like the CDSS format, which showed how many months ago a preventive care service was delivered. Instead, participants wanted the specific date of or the number of years since the latest preventive care service, which suggests our approach deviated from participant expectations or usual thought processes. Therefore, we recommend that future CDSS development should understand the intended users’ needs and expectations or more in depth workflow analysis.
As well, participant commented that they would like reminders of the recommended preventive care time intervals. These reminders were deliberately excluded from this CDSS because there is evidence that user interfaces that make decisions for users reduce contemplation, leading to poorer decision-making . Avoiding overly prescriptive decision support has also been recommended as users may inherently trust the information provided to them .
This inherent trust in the CDSS could be problematic, which was highlighted in a few participants’ comments about data accuracy. This concern is valid as the accuracy and completeness of EMR data has been shown to be problematic . Reasons for these problems may be that data is missing, incorrectly recorded, out dated [28, 29], has variable terminology, or is misattributed to specific conditions and diagnoses . This CDSS aims to circumvents data quality issues by using data that is automatically important into the EMR by most Ontario laboratories  or categorized by the EMR to minimize inaccuracies . This automation may minimize human error and improve data quality. Though challenges can arise when tests have multiple data points in the EMR. For example, Papanicolaou smears data are stored in a variety of ways , so the CDSS must account for this issue as was done here.
This study was restricted to one Academic Family Health Team and the CDSS only works with one EMR. Therefore, the generalization of these results to other clinics and EMRs may present challenges. Other clinics may have better or worse workflows altering the time savings and usefulness afforded by this CDSS. As well, this study was conducted in a simulated environment and the benefits highlighted here may not translate to clinical practice.
Since participants were not randomized to do manual chart review or use the CDSS first, participants may have been primed by the first scenario of manual chart review. Accordingly, this may explain some of the time savings in the automated chart review.
This study demonstrated that CDSSs can leverage EMR data to provide significant time savings without impairing clinical decision-making accuracy. Similar CDSSs may improve EMR workflows and improve the provider experience of care delivery. In part, these time savings and improved experiences may be due to reduced user input requirements and provision of clinically relevant data. Further investigations and CDSS development are required to determine if these findings are applicable to real patient encounters.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Shimizu T, Bouchard M, Mavriplis C. Update on age-appropriate preventive measures and screening for Canadian primary care providers. Can Fam Physician. 2016;62(2):131–8.
Yarnall KSH, Pollak KI, Østbye T, Krause KM, Michener JL. Primary care: Is there enough time for prevention? Am J Public Health. 2003;93(4):635–41.
Dubey V, Mathew R, Iglar K, Moineddin R, Glazier R. Improving preventive service delivery at adult complete health check-ups: the Preventive health Evidence-based Recommendation Form (PERFORM) cluster randomized controlled trial. BMC Fam Pract. 2006;7(1):44.
Senore C, Inadomi J, Segnan N, Bellisario C, Hassan C. Optimising colorectal cancer screening acceptance: a review. Gut. 2015;64(7):1158–77.
Letters to the Public About Cancer Screening [Internet]. Cancer Care Ontario. 2017 [cited 2021 Oct 8]. Available from: https://www.cancercareontario.ca/en/get-checked-cancer/letters-to-public
Canada Health Infoway. The Emerging Benefits of Electronic Medical Record Use in Community-Based Care: Full Report [Internet]. 2013 [cited 2021 Oct 8]. Available from: https://www.infoway-inforoute.ca/en/component/edocman/1224-the-emerging-benefits-of-electronic-medical-record-use-in-community-based-care-full-report/view-document?Itemid=0
Klein Woolthuis EP, de Grauw WJ, van Gerwen WH, van den Hoogen HJ, van de Lisdonk EH, Metsemakers JF, et al. Identifying people at risk for undiagnosed type 2 diabetes using the GP’s electronic medical record. Fam Pract. 2007;24(3):230–6.
Dahrouge S, Hogg WE, Russell G, Tuna M, Geneau R, Muldoon LK, et al. Impact of remuneration and organizational factors on completing preventive manoeuvres in primary care practices. CMAJ. 2012;184(2):E135–43.
Lee EK, Mejia AF, Senior T, Jose J. Improving patient safety through medical alert management: an automated decision tool to reduce alert fatigue. AMIA Annu Symp Proc. 2010;2010:417–21.
Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med. 2016;165(11):753.
Joukes E, Abu-Hanna A, Cornet R, de Keizer NF. Time spent on dedicated patient care and documentation tasks before and after the introduction of a structured and standardized electronic health record. Appl Clin Inform. 2018;9(1):46–53.
Gawande A. Why Doctors Hate Their Computers. 2018 Nov 5 [cited 2019 Jan 17]; Available from: https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers
Friedberg MW, Chen PG, Van Busum KR, Aunon F, Pham C, Caloyeras J, et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy. Rand Health Q [Internet]. 2014 Dec 1 [cited 2020 Apr 14];3(4). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5051918/
Pizziferri L, Kittler AF, Volk LA, Honour MM, Gupta S, Wang S, et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform. 2005;38(3):176–88.
Zheng K, Haftel HM, Hirschl RB, O’Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc JAMIA. 2010;17(4):454–61.
Collier R. Electronic health records contributing to physician burnout. CMAJ Can Med Assoc J. 2017;189(45):E1405–6.
Davis F, Bagozzi R, Warshaw P. User acceptance of computer technology: a comparison of two theoretical Models. Manag Sci. 1989;35:982–1003.
Telus Health. PS Suite User Manual [Internet]. 2019 [cited 2020 Jan 8]. Available from: https://help.pssuiteemr.com/5.15/on/en/Content/01_Introduction/Download_PDFs.htm
Government of Ontario M of H and L-TC. Family Health Teams - Ministry Programs - Health Care Professionals - MOHLTC [Internet]. Government of Ontario, Ministry of Health and Long-Term Care; [cited 2021 Oct 14]. Available from: https://www.health.gov.on.ca/en/pro/programs/fht/
Canadian Task Force on Preventive Health Care | Published Guidelines [Internet]. [cited 2019 Jan 19]. Available from: https://canadiantaskforce.ca/guidelines/published-guidelines/
Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems. Ann Intern Med. 2012;157(1):29–43.
Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573–6.
Berwick DM, Nolan TW, Whittington J. The triple aim: care, health and cost. Health Aff (Millwood). 2008;27(3):759–69.
Gardner RL, Cooper E, Haskell J, Harris DA, Poplau S, Kroth PJ, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26(2):106–14.
Krall MA, Sittig DF. Clinician’s assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp. 2002;400–4.
van Nimwegen C, van Oostendorp H, Burgos D, Koper R. Does an Interface with Less Assistance Provoke More Thoughtful Behavior? In: Proceedings of the 7th International Conference on Learning Sciences [Internet]. Bloomington, Indiana: International Society of the Learning Sciences; 2006 [cited 2019 Jan 19]. p. 785–91. (ICLS ’06). Available from: http://dl.acm.org/citation.cfm?id=1150034.1150148
Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17.
Terry AL, Stewart M, Cejic S, Marshall JN, de Lusignan S, Chesworth BM, et al. A basic model for assessing primary health care electronic medical record data quality. BMC Med Inform Decis Mak. 2019;19(1):30.
Coleman N, Halas G, Peeler W, Casaclang N, Williamson T, Katz A. From patient care to research: a validation study examining the factors contributing to data quality in a primary care electronic medical record database. BMC Fam Pract. 2015;16:11.
eHealth Ontario. OLIS [Internet]. [cited 2020 Apr 14]. Available from: https://www.ehealthontario.on.ca/en/for-healthcare-professionals/ontario-laboratories-information-system-olis
MIS Quarterly [Internet]. [cited 2018 Feb 28]. Available from: https://www.misq.org/perceived-usefulness-perceived-ease-of-use-and-user-acceptance-of-information-technology.html
We would like to thank the University of Ottawa Department of Family Medicine for their support and review of the project during development. We would also like to thank the Bruyère Academic Family Health Team for the use of their electronic medical records and the residents and staff at Bruyère Academic Family Health Team for their time and participation.
No funding was obtained for completion of this study.
Ethics approval and consent to participate
This study was submitted to the University of Ottawa Department of Family Medicine for review and approval prior to commencing. The Department of Family Medicine determined this project was deemed to be quality improvement and therefore a Research Ethics Board application was not required.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Instructions to participants.
Form completed during simulated patient chart analysis.
Perceived Usefulness and Ease of Use Survey completed after the simulated exercise.
Sample Clinical Decision Support System Output.
Clinical Decision Support System Code.
About this article
Cite this article
Laing, S., Mercer, J. Improved preventive care clinical decision-making efficiency: leveraging a point-of-care clinical decision support system. BMC Med Inform Decis Mak 21, 315 (2021). https://doi.org/10.1186/s12911-021-01675-8
- Clinical-decision support
- Preventive care
- Quality improvement