Impact of a computerized system for evidence-based diabetes care on completeness of records: a before–after study

Background Physicians practicing in ambulatory care are adopting electronic health record (EHR) systems. Governments promote this adoption with financial incentives, some hinged on improvements in care. These systems can improve care but most demonstrations of successful systems come from a few highly computerized academic environments. Those findings may not be generalizable to typical ambulatory settings, where evidence of success is largely anecdotal, with little or no use of rigorous methods. The purpose of our pilot study was to evaluate the impact of a diabetes specific chronic disease management system (CDMS) on recording of information pertinent to guideline-concordant diabetes care and to plan for larger, more conclusive studies. Methods Using a before–after study design we analyzed the medical record of approximately 10 patients from each of 3 diabetes specialists (total = 31) who were seen both before and after the implementation of a CDMS. We used a checklist of key clinical data to compare the completeness of information recorded in the CDMS record to both the clinical note sent to the primary care physician based on that same encounter and the clinical note sent to the primary care physician based on the visit that occurred prior to the implementation of the CDMS, accounting for provider effects with Generalized Estimating Equations. Results The CDMS record outperformed by a substantial margin dictated notes created for the same encounter. Only 10.1% (95% CI, 7.7% to 12.3%) of the clinically important data were missing from the CDMS chart compared to 25.8% (95% CI, 20.5% to 31.1%) from the clinical note prepared at the time (p < 0.001) and 26.3% (95% CI, 19.5% to 33.0%) from the clinical note prepared before the CDMS was implemented (p < 0.001). There was no significant difference between dictated notes created for the CDMS-assisted encounter and those created for usual care encounters (absolute mean difference, 0.8%; 95% CI, −8.5% to 6.8%). Conclusions The CDMS chart captured information important for the management of diabetes more often than dictated notes created with or without its use but we were unable to detect a difference in completeness between notes dictated in CDMS-associated and usual-care encounters. Our sample of patients and providers was small, and completeness of records may not reflect quality of care.


Background
Successful diabetes care requires the active participation and informed self-management of engaged, educated, and motivated patients. The healthcare team, in turn, requires timely access at the point of care to comprehensive patient information, including but not limited to weight, blood pressure, current medications, glycosylated haemoglobin (HbA1c), serum creatinine, LDL cholesterol, and urine albumin to creatinine ratio. Easy access to this information helps clinicians and patients to tailor and optimize care. Collection of standardized and well-defined data on each patient also allows for provider-level and clinic-level summary data for quality improvement.
An electronic, diabetes-specific, chronic disease management system (CDMS) that collects limited data during the clinical encounter and presents these data during future clinical encounters may help meet the needs of diabetic patients and their healthcare team.
However, creating effective clinical information systems remains difficult. Because most systems are developed and tested in a few highly computerized environments with a long history of informatics excellence, the findings may not be generalizable to the more technologically naïve settings where most people receive care. Recent attention on health information technology as a means for better healthcare has made it increasingly important to design and test systems in a variety of settings.

The P-PROMPT CDMS in the Diabetes Care and Research Centre
With the help of diabetes specialists, Fig.P Software Inc. tailored its web-based chronic disease management system, the P-PROMPT CDMS, specifically for diabetes care and deployed it in the Hamilton Health Sciences Diabetes Care and Research Centre (DCRC), a hospitalbased, ambulatory care subspecialty clinic. The system's record structure is based on clinical practice in the DCRC and on the Canadian Diabetes Association practice guidelines for 2008, updated by us to 2010 to reflect more recent evidence. In a typical patient visit to the DCRC, physicians interview and examine patients in a room with an internet-connected computer. They always dictate a note summarizing the patient visit. Some practitioners tend to dictate their summary immediately after the patient leaves, while others defer this until the end of the day. Once transcribed, this dictated note is stored in the patient's paper records and communicated to their family doctor. Prior to the DCRC's recent adoption of the CDMS, this note, and any laboratory test results attached to it, comprised the longitudinal record of each patient. Figures 1, 2, 3 show screenshots of the CDMS interface. During or after a patient visit, physicians enter directly into the CDMS information important for the management of diabetes, including each patient's demographic information, existing diagnoses, disease registry membership, and complete medication profile that includes current and past medications, along with reasons for discontinuation. The system is linked to the hospital's electronic laboratory records and summarizes the results of laboratory tests in a dashboard that displays longitudinal trends graphically; results from tests performed outside of the hospital must be entered manually. Finally, the CDMS can provide point-of-care, guideline-based, patient-specific decision support for practitioners and tailored self-management support messages intended for patients.
Before carrying out a large study to test the impact of the CDMS on diabetes care in the DCRC, we conducted a small pilot to help with sample size calculations. For this study we asked, "Does the CDMS impact the recording of clinical information pertinent to evidence-based diabetes care?" Figure 4 summarizes the study design. This was a retrospective chart review of recent patient record entries (electronic charts and transcripts of dictated notes with any attached laboratory reports) generated as a result of CDMS-associated visits and previous transcripts of dictated notes (and attached laboratory reports) created without the help of the CDMS for the same patients. The Hamilton Health Sciences Research Ethics Board provided ethics approval for this study.

Eligibility of patient records
Patient records were eligible for review for patients who had their first complete CDMS-associated assessment at the clinic between January 1st 2011 and April 1st 2011 and previously without the help of the CDMS between June 1st 2009 and June 1st 2010, with no intervening DCRC physician visit (ignoring visits made to other health professionals).

Primary outcome
16 items of particular interest included the patient's weight; blood pressure; use of statins when appropriate; use of Angiotensin Converting Enzyme (ACE) inhibitors or Angiotensin II Receptor Blockers (ARBs) if appropriate; HbA1c; serum creatinine; LDL cholesterol; urine albumin to creatinine ratio (UACR); eye and foot examinations within the past year; any occurrence of severe hypoglycaemic episodes; number of non-severe hypoglycaemic episodes per month; cardiovascular events since the previous visit; and perfusion, light touch sensation, and skin integrity of the feet.
We counted the number of items in each record and calculated a missing fraction using Formula 1 For example, a record with 12 present items would have a missing fraction of 25%. We found the mean missing fraction by first finding the missing fraction  for each patient's records and calculating their mean. We considered items to be present if clearly stated in the record or not relevant for a given patient. We used values for blood pressure and UACR to judge whether a patient's use of an ACE inhibitor or ARB should have been noted in that patient's record, as the relevance of these medications depends on those values. ACE inhibitors or ARBs are appropriate if the UACR is greater than 2.0, or if the patient is known to be hypertensive or blood pressure measured in clinic is above 130/80 mmHg. Statin medications are appropriate if LDL cholesterol levels exceed 2.0 mmol/L, if a diabetic patient is male and over 45 years of age or female and over 50 years of age, or if the patient has ever experienced a stroke, myocardial infarction, or other major vascular event.
Our outcome of interest was the mean absolute difference in missing fraction between pairs of record types (see Analysis). Note that the CDMS does not directly assist dictation. CDMS-associated dictation refers to dictated notes that correspond to patient visits in which the CDMS was used.

Measurement protocol
From a larger group of 7 physicians in the DCRC, we selected for participation in this pilot study 3 attending physicians with a special interest in medical informatics. First, each clinician screened his patients' records sequentially for inclusion according to the eligibility criteria, beginning with the record updated most recently and moving to records updated progressively earlier until 10 eligible patient records were identified. Second, documentation pertaining to 2 separate clinical encounters was assessed within each record: the documentation generated during the recent CDMS-associated encounter and that which was generated during the next most recent encounter that did not include the CDMS, for a total of 61 encounters documented in 31 patient records. a Records were assessed in two phases: 1) review of all dictated notes and 2) review of electronic charts from the CDMS (depending on whether the CDMS was used or not) created during the corresponding patient encounter. We ordered these review phases to minimize any impact of the CDMS chart assessment on assessment of the dictated note.

Analysis
We estimated the mean differences and standard deviations between patient records for the proportion of missing items. We assessed 3 different record types for each patient (transcript of dictation from visit in which the CDMS was used [CDMS-associated transcript], transcript of dictation from usual care [non-CDMS] visit, and CDMS chart). For each patient, we calculated the absolute difference in the missing fraction in each of the following pairs: 1. CDMS-associated transcript of dictation VS usual care transcript of dictation 2. CDMS chart VS CDMS-associated transcript of dictation 3. CDMS chart VS usual care transcript of dictation All patients received care from 1 of 3 providers. We used Generalized Estimating Equations (GEE) [26] to perform our comparisons. GEE allowed us to construct multi-level linear models with a structure that takes into account similarity between records created by the same provider. We measured this similarity using the Intraclass Correlation Coefficient (ICC) [26]. The ICC ranges from 0 to 1; values close to 0 mean that there is little relationship between the records of the same provider while values closer to 1 suggest a strong relationship. An ICC of 1 indicates that completeness for a given type of record is constant for records created by the same provider, making the effective sample size of the study 3 instead of 31. We performed all analyses using the STATA Statistical Software package, version 11.2 [27]. Table 1 describes the data; Table 2 presents results of the statistical comparisons and, along with Figure 5, summarizes missing fractions corresponding to each record type overall and by provider. The CDMS chart outperformed the CDMS-associated dictated note by a substantial margin (mean missing fraction, 10 22.4%]). However, there was no difference between the usual-care dictated note and the CDMS-associated dictated note (absolute mean difference, 0.8%; 95% CI, −8.5% to 6.8%). Figure 6 presents the number of times each of the 16 items was considered missing across the 3 record types.

Results
We did not conduct statistical tests to determine the greatest contributors to the missing fraction, but serum creatinine, UACR, hypoglycemia, and perfusion of feet appear to contribute most often.
The ICC for the difference in missing fraction between CDMS charts and CDMS-associated dictation transcripts created by the same practitioner was 0.15 (95 CI, 0.00 to 0.55), 0.25 (0.00 to 0.75) for the difference in missing fraction between CDMS charts and usual care transcripts, and 0 (0.00 to 0.20) for the difference in missing fraction between CDMS-associated transcript and usual care transcript. These estimates are very imprecise because only three providers were included in the study.

Discussion
We discovered that CDMS charts capture information important for the management of diabetes more often than dictated notes created with or without its use. There was no difference between the two types of dictated notes, suggesting that use of the CDMS neither improved nor hindered the quality of the dictated note among diabetes specialists. We chose the target sample size of 30 patients to help us calculate sample size for future studies of this system, and did not explicitly power the study to detect differences in completeness. With so few patients, we cannot dismiss the possibility of positive or negative impact. Indeed, confidence intervals ranged from substantial improvement to substantial decline in completeness of dictated notes with use of the CDMS.
Completeness of records is only a distant surrogate marker of clinical care quality. It is possible that the items in our assessment form were biased toward the CDMS chart. The CDMS may elicit responses to items that the practitioner deems irrelevant for the dictated summary of a particular clinical encounter. For example, an unremarkable foot exam may not be mentioned in the dictation but the practitioner may still indicate in the CDMS chart (which provides a relevant drop-down menu) that it was performed. In this situation, the patient does not gain additional benefit despite having a more complete CDMS chart.
While the dictated notes did not change with CDMS use, storing more complete records in a structured electronic format allows for other quality improvement interventions. For example, the CDMS includes pointof-care, guideline-based, patient-specific decision support for practitioners and tailored self-management support messages intended for patients. A number of systems that engage diabetic patients and their practitioners have demonstrated benefit in randomized trials [8][9][10]12]. The CDMS is also being enhanced to provide periodic clinician-level performance feedback, which was previously found to improve the quality of care when combined with patient-specific decision support [28].  Providers were happy with the ability to track patients' progress longitudinally, and with access to past test results and medication profiles. We can expect some benefit to patient health if particular management decisions require that the practitioner refer to a detailed historical care record. Studies much larger and lengthier than are typically feasible with local computerized interventions would be necessary to detect these benefits, which may be rare but still meaningful given the high prevalence of diabetes.
A specialized diabetes clinic appears to lend itself well to the use of an electronic record and disease Figure 5 Does the CDMS improve recording of key parameters? Key parameters were missing less often from the CDMS chart than from dictated notes. Error bars indicate 95% confidence intervals. Figure 6 Which key parameters are missed and how often? CDMS, Chronic disease management system; BP, Blood pressure; ACEi, Angiotensin Converting Enzyme inhibitor; ARB, Angiotensin II Receptor Blocker; HbA1C, Glycosylated haemoglobin; Cr, Creatinine; LDL, Low density lipoprotein cholesterol; UACR, Urinary albumin to creatinine ratio; CV, Cardiovascular. management system tailored to the needs of diabetes specialists and their patients. Our findings, however, may not be generalizable to primary care, where most diabetic patients are managed. Primary care physicians address a wide range of ailments within a single patient due to multimorbidity [29] and across patients owing to a broad scope of practice. It may, therefore, be difficult for general practitioners to adopt a detailed system for the management of just one disease.

Conclusions
In this small, retrospective, before-after pilot study, the CDMS chart captured information important for the management of diabetes more often than dictated notes created with or without its use, but we were unable to detect a difference in completeness between notes dictated in CDMS-associated and usual-care encounters. Larger studies will assess the impact of this system on completeness of records and other, better indexes of care quality. Endnote a One usual care dictated note was not available.

Competing interests
One usual care dictated note was not available.RJS is the owner of Fig.P Software Incorporated, which develops the chronic disease management system discussed in this study. Other authors declare no competing interests.