This article has Open Peer Review reports available.
Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective
© Tu et al. 2015
Received: 14 July 2014
Accepted: 28 July 2015
Published: 13 August 2015
With the introduction and implementation of a variety of government programs and policies to encourage adoption of electronic medical records (EMRs), EMRs are being increasingly adopted in North America. We sought to evaluate the completeness of a variety of EMR fields to determine if family physicians were comprehensively using their EMRs and the suitability of use of the data for secondary purposes in Ontario, Canada.
We examined EMR data from a convenience sample of family physicians distributed throughout Ontario within the Electronic Medical Record Administrative data Linked Database (EMRALD) as extracted in the summer of 2012. We identified all physicians with at least one year of EMR use. Measures were developed and rates of physician documentation of clinical encounters, electronic prescriptions, laboratory tests, blood pressure and weight, referrals, consultation letters, and all fields in the cumulative patient profile were calculated as a function of physician and patient time since starting on the EMR.
Of the 167 physicians with at least one year of EMR use, we identified 186,237 patients. Overall, the fields with the highest level of completeness were for visit documentations and prescriptions (>70 %). Improvements were observed with increasing trends of completeness overtime for almost all EMR fields according to increasing physician time on EMR. Assessment of the influence of patient time on EMR demonstrated an increasing likelihood of the population of EMR fields overtime, with the largest improvements occurring between the first and second years.
All of the data fields examined appear to be reasonably complete within the first year of adoption with the biggest increase occurring the first to second year. Using all of the basic functions of the EMR appears to be occurring in the current environment of EMR adoption in Ontario. Thus the data appears to be suitable for secondary use.
Although North American countries have previously been lagging behind in the uptake of electronic medical records [EMRs] into their clinical practice , in recent years, with the introduction of government policies and mandates, physicians have been increasingly adopting EMRs into their clinical practice [2–5]. Canada Health Infoway strives towards the goal of one electronic health record for all Canadians, however in Canada, healthcare is organized at the provincial level and therefore each province has its own EMR adoption program and policies. Ontario was the first Canadian province to develop an EMR adoption program over a decade ago and although was initially slow in uptake , approximately 80 % of family physicians in Ontario are now on, or planning to go on an EMR in the near future . Ontario’s EMR adoption program has targeted family physicians and provides physicians subsidization for adopting an approved EMR software into their clinical practice for capture of clinical activity that occurs within the family physician office and relevant information pertaining to the family physician management of patient health care. There are additional financial incentives if physicians can demonstrate that they are using their EMR to schedule patient appointments, enter encounter notes, enter problem lists, record prescriptions, generate automatic alerts/reminders to support care delivery and receive laboratory results electronically .
Government motivation to encourage EMR use includes expectations of both improvements in efficiency and quality of care. However evidence to support this notion is limited and whether or not EMR use results in improved patient care and outcomes is unclear [8–11]. Advanced features such as reminders, clinical decision support, and practice-based surveillance are considered to be amongst the most effective tools within the EMR to improve care [12–14] but cannot be used without having completed fields for the variables which trigger these tools. Understanding the extent to which physicians are using their EMRs is more important for improving care than simply assessing the presence or absence of an EMR. Previous studies that have looked at using electronic health record data for quality measurement  or comparative effectiveness research  have recognized the complexity of this type of evaluation and have provided conceptual frameworks on how to assess the data and issues to consider. Although useful in concept they do not provide real world analysis of the completeness of data. In addition a recent literature review assessing the reliability and validity of electronic health record quality measures found that most studies have been performed on data from large academic sites .
Few studies have looked at the time necessary to have reasonably complete EMR records. One study looked at data reliability but was confined to assessments for preventative services (pap smears, mammograms and influenza vaccinations). They found that data extracted from EMRs had limited reliability in the initial phase of EMR implementation but that during the second year after the introduction of the EMR, data reliability improved substantially .
It is unknown if the current government sponsored programs for EMR adoption including financial subsidies and peer leader support, are sufficient for supporting successful EMR implementation and it is unknown if physicians are comprehensively using their EMRs for all aspects of patient care. Evaluating the data quality of EMRs is not only important for policy makers and health administrators, but for researchers who seek to use EMRs for secondary purposes, such as research. Therefore, we evaluated the data population of a variety of EMR fields as a measure for completeness of EMR usage to assess if current policies are sufficient for EMR implementation and the suitability of the EMR data for secondary purposes. We also assessed the duration of time after EMR implementation to have complete EMR records and provide benchmarks for evaluation of the completeness of a variety of EMR fields.
We performed a retrospective review of data within the Electronic Medical Record Administrative data Linked Database (EMRALD) at the Institute for Clinical Evaluative Sciences [ICES]. EMRALD contains data from family physicians in Ontario that use Practice Solutions® EMR, the market leading EMR software vendor in Ontario, Canada . All clinically relevant information from the EMRs are extracted. ICES is a ‘prescribed entity’ under provincial privacy legislation which provides the legal authority to collect individual level health information as it has the policies and procedures in place to protect patient privacy and confidentiality .
Data were extracted in the summer of 2012 from physicians who had been using their EMR for at least one year from both urban and rural locations distributed throughout Ontario. All physicians had a list of patients ‘rostered’ (enrolled) under them for which both the physician and the patient have a signed agreement with the provincial government identifying the physician that is responsible for the patient’s primary care service delivery . The characteristics of physicians included in this study were compared to all family physicians in Ontario using the ICES Physician Database [IPDB].
Only patients that had a valid health insurance number, date of birth, and were rostered to the contributing physicians were included. Active patients were defined as rostered patients that had a physician visit within the one year prior to the date of data extraction or in the one year time interval examined. The characteristics of patients included in this study were compared to all patients rostered in Ontario and all Ontario residents. Neighborhood income quintiles and rural residence were determined through linking postal codes to Canadian census data . General assessments of the burden of comorbidities were calculated using number of Ambulatory Care Groups (ACGs)  and presence of chronic conditions were determined using previously validated administrative data algorithms for identifying patients with specific disease conditions [24–27].
Measures of EMR utilization
Fields for EMR usage assessed and their measures
EMR usage field
Practice style independent fields
Billings in the EMR for an office visit with a corresponding progress note entry on the same day
Blood pressure and weight recording
Blood pressure or weight recorded in the structured variable
Completeness of the Cumulative Patient Profile [CPP]
Populated allergies, immunizations, active treatment, risk factors, personal traits, family history and medical history [history of past health and problem list included together] fields
Practice style dependent fields
Laboratory test results provided through an electronic link.
At least one laboratory test
Using the EMR to generate and record prescriptions
At least one prescription
Using the EMR for generating referral letters from the family physician to the specialist
Entry in the EMR in the referral’s field
EMRs including specialist consultant reports.
Entry in the EMR in the specialist’s consulatation letter field
Blood pressures and weights in the EMR are automatically captured in structured variable fields if entered using the nomenclature bp: and wt: respectively. The advantage for clinicians of having structured variable fields is that the measures can be searched and graphed to see changes over time. The advantage for researchers is that these measures can easily be identified and analysed. Physicians that have low recordings of these measures may not have been doing these measures, not recording these measures, or not recording these measures with the proper nomenclature such that the measurement populated the structured variable field.
Laboratory results can come into the EMR either through an automated electronic feed from the laboratory, manually entered in, or scanned into the EMR as a report. Only laboratory results entered into the EMR in a structured variable field, as what occurs in the instance of an automated electronic feed, are readily accessible both for analysis and for clinical care. Information stored in a structured variable field can be quickly searched and graphed to look at trends over time whereas information in scanned documents tends not to be searchable.
Some physicians who are not fully using their EMRs may opt to write their prescriptions on a prescription pad or a referral on a paper referral form. However doing so would result in incomplete EMR patient records and may not allow for use of advanced EMR functions. Thus we assessed the percent of active patients that had a prescription or referral letter. Physicians with really low numbers of ‘active’ patients with a prescription or a referral letter were thought to be using paper to perform these functions rather than their EMR.
Currently there is no province-wide program that facilitates automatic electronic transfer of consultation letters from specialists and hospital discharge summaries. Mechanisms to electronically transfer hospital discharge summaries are now starting to roll out but at the time of the study these mechanisms were not in place. We therefore looked at the number of consult letters compared to the number of rostered patients, as an approximate measure, to see if these external documents that were coming into physician offices were being entered into the EMR.
The CPP includes the categories: family history, medical history including history of past health and problem list (a list of current active problems), allergies, immunizations, active treatments, risk factors and personal traits. Within the EMR there are two separate fields for the history of past health and for a list of active problems. These two fields were grouped together since physicians do not use the two separate fields in a consistent manner.
Measuring time on EMR
Completeness of EMR fields by physician time on EMR was assessed by looking at each year since initiation of the EMR. The initiation date of the EMR was defined as the earliest date with at least 10 progress notes with a corresponding bill for a visit that were recorded on the same day. Only physicians that had a full year of data in the relevant year were included in the denominators for assessment. For example only the patients of the physicians that had at least 4 years of data were included in the assessment of the completeness of the EMR fields for the physician time on EMR 4 year category. The analysis for patient time on EMR was done by assessing both the EMR fields and the CPP fields in the one year prior to the most recent date of data extraction for patients that had a record in the EMR for <1 year, 1-2 years, 2-3 years and >3 years. Patient time on the EMR was calculated from first date of a progress note and a physician bill on the same date to the date of the most recent data extraction. The analysis for the EMR fields for the patient time on EMR were confined to just those patients that were ‘active’ (had a visit in the last year since extraction).
Physician characteristics by duration of EMR use were descriptively analyzed. We estimated the average completeness by duration of EMR use for each field with 95 % confidence intervals [CI] constructed around the average frequency of data completeness. Analyses were performed using SAS version 9.2 [SAS Institute, Cary, North Carolina].
We looked to identify benchmarks of completeness for each of the EMR fields, as benchmarks of this type have not previously been reported. The EMRALD team including several family physicians/EMR users discussed and agreed upon benchmarks for optimal completeness. The rationale behind these benchmarks included considerations for clinical use and secondary use of EMR data.
For the practice style independent fields we agreed that visit documentation (documentation for the purposes of clinical record keeping) and allergies were clinically the most important for patient safety and thus a benchmark of 95 % data completeness was set. For the other practice style independent fields a benchmark of 80 % completeness was agreed to be sufficient, feasible and clinically meaningful. For practice style dependent fields we recognized that physicians may simply prescribe fewer medications, order fewer lab tests, or refer their patients less than their colleagues. Thus it was decided that benchmarks for these parameters was best suited to be data driven, to identify outliers with really low population of these fields potentially indicating systemic EMR issues rather than physician practice style issues. The benchmark for these fields was set at the mean for the measure less one standard deviation.
Physicians were plotted on frequency distribution graphs and benchmarks were drawn to gain an understanding of where physicians lay in terms of meeting benchmarks.
This study received ethics approval from the Sunnybrook Health Science Centre Research Ethics Board. Individual level physician consent was not required for this study. As a prescribed entity under Ontario’s Personal Health Information Protection Act, ICES is authorized to collect and use personal health information from health organizations and clinics without consent for the purposes of health system evaluation and monitoring. Additionally, ICES is prohibited, under its agreements with data providers, from contacting individuals whose information has been entrusted to ICES. This contractual obligation restricts any opportunity to seek individuals’ consent for use of their information.
Comparison of EMRALD study cohort patients, rostered patients in Ontario and all residents of Ontario as of March 31, 2012
Rostered Ontario patients
All Ontario residents
Number of people
Neighborhood Income Quintile
1 - Lowest Income
5 - Highest Income
Number of Adjusted Clinical Groups [ACGs]b
Any chronic condition
Previous Acute Myocardial Infarction
Congestive Heart Failure
Chronic Obstructive Pulmonary Disease
Mental health issue
Comparison of study cohort physicians and all primary care physicians in Ontario as of March 31, 2012
EMRALD cohort physicians
All primary care physicians in Ontarioa
Under 35 years
Medical training location
International [including US]
Visits in the Emergency Department
More than 25 % of practice/bills
Less than 25 % of practice/bills
Practising in a patient enrolment model group
Full time affiliation
Physician Age on March 31, 2012
Years in practice
Years since graduation
Physician characteristics by duration of EMR use
Duration of EMR use
At least 1 year
At least 2 years
At least 3 years
At least 4 years
At least 5 years
Number of physicians
Percent female physicians
Mean physician age in years [SD]
Mean number of years in practice in Ontario [SD]
Percent in rural practice location
Mean duration of EMR use in years [SD]
Total number of patients
We developed measures to evaluate the extent to which physicians are using the EMR for patient care in Ontario. Overall it appears that physicians are making use of their EMRs within the first year and certainly by the second year post-adoption. Thus, it appears that the current levels of support to use the basic EMR functions are sufficient in achieving adoption of the EMR.
Increased physician and patient duration on the EMR improved data completeness, with the biggest improvements seen between the first and second year. This is similar to a previous study that found recording of preventative screening tests for cancer were not adequately captured in the first year of starting an EMR but improved substantially when looking in the two year window post EMR adoption . Another study found that having two or more years of electronic health record experience was independently associated with reported benefits of having an electronic health record . This time period required for completeness is also relevant to secondary users of the data when making assessment of the quality of care provided or ‘fit for purpose’ assessments, as incomplete EMR data may impede the use of clinical decision support embedded within the EMR, or result in inaccurate analysis of data for quality of care assessment or other types of analysis [28, 29]. Although duration of time on the EMR has not been shown to result in higher performance on quality indicators , our study illustrates that both physician and patient time on EMR influence the likelihood of data completeness. The results of this study have led us to develop data quality checks within the EMRALD database to look for outlying physicians to exclude from our studies if a particular field of the EMR is going to be utilized in a study analysis. Our results presented here should have applicability in other settings as EMRALD participants are not just academic sites but draw largely from community practising family physicians.
One of the important features of advanced EMR functions is to automatically generate feedback and reminders at the point of care and to ensure that they are a part of the healthcare providers workflow . Most of these clinical aids rely on completion of EMR data fields in order to accurately report back to the physician. The first step to using advanced functions of the EMR, such as reminders or clinical decision support, is having completed fields for the variables which trigger these tools. While we studied completeness of fields as a measure for EMR usage, we recognize that completeness is only one dimension of data quality . However, assessments of other dimensions of data quality cannot occur without a first assessment of completeness and previous studies looking at the suitability of using EMR data to recruit patients into clinical trials have found data completeness to be an essential component in the assessment of using EMR data for secondary purposes . Whether these completeness measures would hold true with other EMR systems is a potential limitation but we provide real world measures that can be used for comparison with data from other EMR systems.
A previous study has shown that the recording of the presence of some conditions is less frequent in the EMR compared to self-report . Another limitation of this study is that although we were able to determine if there was information recorded in each of the CPP fields we were unable to determine the completeness or quality of recording within each field. Other limitations include the inability to measure the time duration from installment of the EMR to actual usage of the EMR, but given that government funding support is time limited and user fees are charged by EMR vendors from time of installation, it is unlikely that the time from installation to the time of actual use are of significant duration. Another limitation is that we were unable to assess the impact of the local practice environment and/or the presence of allied health professionals on completeness of EMR fields. It is unknown who in a practice was responsible for completing fields such as the CPP or entering in blood pressures. Unfortunately we could also not tell when CPP fields were entered, as we only receive a snap shot of the CPP as it stands on the date of extraction and each item in the CPP is not time stamped. We could also not assess the quality of the data that was entered for example, recording of dates of disease onset. Last, although our patient characteristics were similar to rostered patients in Ontario in terms of presence of chronic diseases and co-morbid conditions, our EMRALD sample of patients were from higher income quintiles and had proportionally more patients living in rural areas compared to the Ontario population. In terms of income quintiles it is not known if this bias is because patients of higher income quintiles may be more likely to seek health care at the primary care level. With respect to a higher proportion of patients residing in rural areas, this proportion constantly changes as new physicians contribute to EMRALD and recruitment is on an ongoing basis. Similarly, our physician characteristics did not exactly match the rest of the family physicians in Ontario. Nonetheless our results showed consistent improvements with completion of fields with increasing duration of time on the EMR despite heterogeneity of physician characteristics.
In this paper we outline a pragmatic process which we have used to assess completeness of a variety of EMR fields to determine if physicians are adequately using their EMRs and to assess suitability for secondary use. Certainly we know that current EMRs in Ontario are deficient in documenting elements such as hospitalizations and emergency room visits . However, using these described methods in a broad range of EMR fields should be sufficient to allow use of primary care EMR data in a variety of different types of studies as we have employed these methods for data quality checking and have been able to perform a wide range of studies with EMRALD. The types of studies we’ve done have included: assessment of wait times from family physician referral to specialist , audit and feedback of quality indicators for chronic disease management , validation of administrative data algorithms [38, 39] and within EMR algorithms [40–42] to identify patients with a variety of medical conditions.
Additionally the methods that we have developed here can be used to identify physicians who perform poor on measures of data completeness and are in need of further assistance. Programs could be developed to identify those who fall below their peers and offer support to increase data quality and completeness to get them to more optimal usage. Since we did not examine the use of advanced functions of EMR systems that have been indicated to be important for EMR benefits realization, future work could examine the best point after adoption for them to be implemented, and their impact on patient management, performance on quality indicators and patient outcomes. All users of EMR data, including physicians, administrators, researchers, and policy makers should be acutely aware of the need for understanding data quality and completeness prior to utilizing it for purposes secondary to direct patient care. Furthermore, analysis of both patient time and physician time on EMR data are also important considerations for using EMRs for research as there is a greater likelihood of detecting data with increasing time of a patient’s contribution of data to the database.
This work was funded by a service agreement with Canada Health Infoway.
This study was supported by the Institute for Clinical Evaluative Sciences [ICES], which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care [MOHLTC]. The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.
Dr. Karen Tu is supported by a Canadian Institutes of Health Research Fellowship Award in Primary Care and a Research Scholar Award from the Department of Family and Community Medicine, University of Toronto.
Dr. Noah Ivers is supported by New Investigator Awards from the Canadian Institutes of Health Research the Department of Family and Community Medicine, University of Toronto.
Dr. Debra Butt and Dr. Liisa Jaakkimainen are supported by a Research Scholar Award from the Department of Family and Community Medicine, University of Toronto.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Schoen C, Osborn R, Doty MM, Squires D, Peugh J, Applebaum S. A survey of primary care physicians in eleven countries, 2009: perspectives on care, costs, and experiences. Health Aff (Millwood). 2009;28(6):w1171–83.View ArticleGoogle Scholar
- Patel V, Jamoom E, Hsiao CJ, Furukawa MF, Buntin M. Variation in electronic health record adoption and readiness for meaningful use: 2008-2011. J Gen Intern Med. 2013;28(7):957–64.View ArticlePubMedPubMed CentralGoogle Scholar
- Xierali IM, Hsiao CJ, Puffer JC, Green LA, Rinaldo JC, Bazemore AW, et al. The rise of electronic health record adoption among family physicians. Ann Fam Med. 2013;11(1):14–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Canada of Family Physicans of Canada, Canadian Medical Association, Royal College of PHysicians and Surgeons of Canada. National Physician Survey 2007. Results. 2007 [accessed 2014 December 11]; Available from: http://nationalphysiciansurvey.ca/surveys/2007-survey/.
- College of Family Physicians of Canada CMA, Royal College of Physicians and Surgeons of Canada. National Physician Survey 2014. Results. 2014. [accessed 2014 December 11]; Available from: http://nationalphysiciansurvey.ca/surveys/2014-survey/survey-results-2/.
- Keshavjee K. EMR Implementation in Ontario: A position paper to increase EMR implementation in Ontario. Intel of Canada White Paper; 2007.Google Scholar
- Ontario MD EMR Enhanced Use Program. [accessed 2014 December 11]; Available from: https://www.ontariomd.ca/portal/server.pt/community/emr_funding/emr_enhanced_use_program/.
- Jang J, Yu SH, Kim CB, Moon Y, Kim S. The effects of an electronic medical record on the completeness of documentation in the anesthesia record. Int J Med Inform. 2013;82(8):702–7.View ArticlePubMedGoogle Scholar
- King J, Patel V, Jamoom EW, Furukawa MF. Clinical benefits of electronic health record use: national findings. Health Serv Res. 2014;49(1 Pt 2):392–404.View ArticlePubMedGoogle Scholar
- Kern LM, Barrón Y, Dhopeshwarkar RV, Edwards A, Kaushal R, Investigators H. Electronic health records and ambulatory quality of care. J Gen Intern Med. 2013;28(4):496–503.View ArticlePubMedGoogle Scholar
- Klumpp TR. Electronic medical records and quality of cancer care. Curr Oncol Rep. 2013;15(6):588–94.View ArticlePubMedGoogle Scholar
- Tolar M, Balka E. Beyond individual patient care: enhanced use of EMR data in a primary care setting. Stud Health Technol Inform. 2011;164:143–7.PubMedGoogle Scholar
- Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.View ArticlePubMedGoogle Scholar
- Abramson EL, Malhotra S, Osorio SN, Edwards A, Cheriff A, Cole C, et al. A long-term follow-up evaluation of electronic health record prescribing safety. J Am Med Inform Assoc. 2013;20(e1):e52–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Kahn MG, Raebel MA, Glanz JM, Riedlinger K, Steiner JF. A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012;50(Suppl):S21–9.View ArticlePubMedGoogle Scholar
- Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013;51(8 Suppl 3):S30–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Chan KS, Fowles JB, Weiner JP. Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010;67(5):503–27.View ArticlePubMedGoogle Scholar
- Greiver M, Barnsley J, Glazier RH, Harvey BJ, Moineddin R. Measuring data reliability for preventive services in electronic medical records. BMC Health Serv Res. 2012;12:116.View ArticlePubMedPubMed CentralGoogle Scholar
- Vendor and Offering Market Share Reports. [accessed 2014 December 11]; Avaliable from: https://www.ontariomd.ca/portal/server.pt/community/emr_offerings/vendor_market_share/.
- Privacy at ICES. [accessed 2015 July 30]; Available from: http://www.ices.on.ca/Data-and-Privacy/Privacy%20at%20ICES.
- Ministry of Health and Long-Term Care M. Health Force Ontario: Family Practice Models. 2013 [accessed 2015 July 30]; Available from: http://www.healthforceontario.ca/en/Home/Physicians/Training_%7C_Practising_in_Ontario/Physician_Roles/Family_Practice_Models.
- Wilkins R. Automated geographic coding based on the Statistics Canada postal code conversion files, including postal codes through March 2009. Ottawa: Ontario, Canada; 2009.Google Scholar
- The John Hopkins University Bloomberg School of Public Health HSRDC. The John Hopkins Adjusted Clinical Groups® (ACG) Case-Mix System Version 6.0 Release Notes. The Johns Hopkins University, 2003 April. Report No.: Contract No.: June 30.Google Scholar
- Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512–6.View ArticlePubMedGoogle Scholar
- Tu K, Campbell N, Chen Z, Cauch-Dudek KJ, McAlister FA. Accuracy of administrative databases in identifying patients with hypertension. Open Med. 2007;1(1):E18–26.PubMedPubMed CentralGoogle Scholar
- Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying patients with physician-diagnosed asthma in health administrative databases. Can Respir J. 2009;16(6):183–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying individuals with physcian diagnosed COPD in health administrative databases. COPD. 2009;6(5):388–94.View ArticlePubMedGoogle Scholar
- Liaw ST, Chen HY, Maneze D, Taggart J, Dennis S, Vagholkar S, et al. Health reform: is routinely collected electronic information fit for purpose? Emerg Med Australas. 2012;24(1):57–63.View ArticlePubMedGoogle Scholar
- Burge F, Lawson B, Van Aarsen K, Putnam W. Assessing the feasibility of extracting clinical information to create quality indicators from primary healthcare practice EMRs. Healthc Q. 2013;16(3):34–41.View ArticlePubMedGoogle Scholar
- Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ, Bates DW, et al. The relationship between electronic health record use and quality of care over time. J Am Med Inform Assoc. 2009;16(4):457–64.View ArticlePubMedPubMed CentralGoogle Scholar
- Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.Google Scholar
- Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20:144–151.Google Scholar
- Köpcke F, Trinczek B, Majeed RW, Schreiweis B, Wenk J, Leusch T, Ganslandt T, Ohmann C, Bergh B, Röhrig R, Dugas M, Prokosch HU. Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence. BMC Med Inform Decis Mak 2013;13:37.Google Scholar
- Violán C, Foguet-Boreu Q, Hermosilla-Pérez E, Valderas JM, Bolíbar B, Fabregas-Escurriola M, et al. Comparison of the information provided by electronic health records data and a population health survey to estimate prevalence of selected health conditions and multimorbidity. BMC Public Health. 2013;13(1):251.View ArticlePubMedPubMed CentralGoogle Scholar
- Tu K, Mitiku TF, Ivers NM, Guo H, Lu H, Jaakkimainen L, et al. Evaluation of Electronic Medical Record Administrative data Linked Database (EMRALD). Am J Manag Care. 2014;20(1):e15–21.PubMedGoogle Scholar
- Jaakkimainen L, Tu K, Barnsley J, Salkeld E, Lu H, Glazier R, et al. Waiting to see the specialist: patient and provider characteristics of wait times from primary to specialty care. BMC Fam Pract. 2014;15(1):16.View ArticlePubMedPubMed CentralGoogle Scholar
- Ivers NM, Tu K, Young J, Francis JJ, Barnsley J, Shah BR, et al. Feedback GAP: pragmatic, cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci. 2013;8:142.View ArticlePubMedPubMed CentralGoogle Scholar
- Tu K, Wang M, Jaakkimainen RL, Butt D, Ivers NM, Young J, et al. Assessing the validity of using administrative data to identify patients with epilepsy. Epilepsia. 2014;55(2):335–43.View ArticlePubMedGoogle Scholar
- Tu K, Wang M, Young J, Green D, Ivers NM, Butt D, et al. Validity of administrative data for identifying patients who have had a stroke or transient ischemic attack using EMRALD as a reference standard. Can J Cardiol. 2013;29(11):1388–94.View ArticlePubMedGoogle Scholar
- Tu K, Manuel D, Lam K, Kavanagh D, Mitiku TF, Guo H. Diabetics can be identified in an electronic medical record using laboratory tests and prescriptions. J Clin Epidemiol. 2011;64(4):431–5.View ArticlePubMedGoogle Scholar
- Ivers N, Pylypenko B, Tu K. Identifying patients with ischemic heart disease in an electronic medical record. J Prim Care Community Health. 2011;2(1):49–53.View ArticlePubMedGoogle Scholar
- Krysko KM, Ivers NM, Young J, O’Connor P, Tu K. Identifying individuals with multiple sclerosis in an electronic medical record. Mult Scler. 2015;21(2):217–24.View ArticlePubMedGoogle Scholar