This article has Open Peer Review reports available.
Integrating data from an online diabetes prevention program into an electronic health record and clinical workflow, a design phase usability study
© The Author(s). 2016
Received: 20 January 2016
Accepted: 3 July 2016
Published: 11 July 2016
Health information is increasingly being digitally stored and exchanged. The public is regularly collecting and storing health-related data on their own electronic devices and in the cloud. Diabetes prevention is an increasingly important preventive health measure, and diet and exercise are key components of this. Patients are turning to online programs to help them lose weight. Despite primary care physicians being important in patients’ weight loss success, there is no exchange of information between the primary care provider (PCP) and these online weight loss programs. There is an emerging opportunity to integrate this data directly into the electronic health record (EHR), but little is known about what information to share or how to share it most effectively. This study aims to characterize the preferences of providers concerning the integration of externally generated lifestyle modification data into a primary care EHR workflow.
We performed a qualitative study using two rounds of semi-structured interviews with primary care providers. We used an iterative design process involving primary care providers, health information technology software developers and health services researchers to develop the interface.
Using grounded-theory thematic analysis 4 themes emerged from the interviews: 1) barriers to establishing healthy lifestyles, 2) features of a lifestyle modification program, 3) reporting of outcomes to the primary care provider, and 4) integration with primary care. These themes guided the rapid-cycle agile design process of an interface of data from an online diabetes prevention program into the primary care EHR workflow.
The integration of external health-related data into the EHR must be embedded into the provider workflow in order to be useful to the provider and beneficial for the patient. Accomplishing this requires evaluation of that clinical workflow during software design. The development of this novel interface used rapid cycle iterative design, early involvement by providers, and usability testing methodology. This provides a framework for how to integrate external data into provider workflow in efficient and effective ways. There is now the potential to realize the importance of having this data available in the clinical setting for patient engagement and health outcomes.
Information is the currency of healthcare. With the explosion of electronic health records, that information is increasingly digitally stored and exchanged . The public is digitizing personal health data faster than ever before. People regularly collect and store health-related data electronically, on their own devices and in the cloud, with direct implications for their health . From steps on a pedometer to calories consumed to biometric data to personal reminders, there is more health-related data outside of healthcare systems than in it [1, 3]. Healthcare has historically relied on patients to provide this information to their providers manually, however there is an emerging opportunity to integrate this data directly into the electronic health record. To be clinically effective, however, this data must be integrated into the increasingly complex healthcare team workflow [4, 5]. Only with optimal integration of this data – the who, what, when, where, and how of human-computer interactions – will the promise of improved healthcare delivery effectiveness and efficiency from health information technology be realized.
The Health Information Technology for Economic and Clinical Health (HITECH) act mandates the ability to electronically capture and share health data, and much effort is being put into achieving this goal [6, 7]. However, there has been little focus on the usability of the data [8–14]. The most successful software systems have paid great attention to the look and feel of their product. In the same way, attention must be paid to the integration, workflow implications and visualization of data in the electronic health record (EHR). And the data does not have to just come from other healthcare systems – it can come directly from the patient or from other software systems that the patient interacts with.
It is projected that by 2030, more than 2 billion people worldwide will be overweight and 1 billion will be obese . As a close corollary, by 2025 it is estimated there will be 380 million people living with type 2 diabetes . In the United States alone, diabetes cost more than $200 billion in the year 2007 . While the burden of disease is great, so also is the opportunity. As many as 90% of the cases of type 2 diabetes are attributable to diet and lifestyle, and thus potentially preventable . Not surprisingly, a number of initiatives have been developed to target behavior change to prevent obesity and diabetes. Most notably, the Diabetes Prevention Program (DPP) and its derivatives have been largely successful in controlled studies [19–21]. But these interventions have questions of reproducibility and scalability. Likewise, consensus statements for the treatment of obesity and diabetes place an importance on exercise and diet but primary care providers are frustrated with how to counsel their patients to achieve these lofty goals [22, 23]. One potential solution has been to utilize online tools to engage patients in lifestyle modification programs. The Goal-focused Online Access to Lifestyle Support (GOALS) program is an online adaptation of the DPP that aims to curb the burden of diabetes through online education and coaching. The program incorporates physical activity tracking, diet reporting, and educational modules with personalized lifestyle coaching. Thus far, the program has been shown to help patients lose weight and improve blood pressure control . Despite their successes, currently there is limited meaningful communication between providers and behavior modification programs such as DPP or GOALS. This is a lost opportunity since physician counseling has been shown to have a significant impact on patients’ weight loss and is expressed as a key motivator [25, 26]. With the optimization of electronic health records (EHRs) closing the digital divide, there now exists the opportunity to integrate updates from external lifestyle counseling seamlessly into the primary care visit using patient-generated data. However, as it stands, there remains a void in the ability of EHRs to integrate this lifestyle modification data in a useful way.
Effective design and implementation of EHR tools require studying the usability of these tools. Formal testing of these EHR features and tools with end-users has become essential to enhancing usability and outcomes, and meaningfully using EHRs [27–30]. Human computer interaction related to using the EHR has been studied using think-aloud, near live and live usability testing as well as surveys and interviews [31–36]. These use both quantitative and qualitative methodology to identify usability issues prior to adoption of the software. Agile, rapid-cycle design of the software that incorporates end-user feedback early in the design phase can decrease development time, provide transparency to the design process, and help ensure that the end-product reflects current practice patterns [37–39]. To date, little is known about the preferences of providers concerning the type, amount, and location of behavior change data or regarding its optimal workflow interface with the EHR.
Employing the use-case of the GOALS online diabetes prevention program, we performed a qualitative study to characterize the preferences of providers concerning the integration of externally generated lifestyle modification data into a primary care EHR workflow and an iterative design process to develop that interface.
We used a qualitative study design through semi-structured interviews with primary care providers to understand their clinical workflow and preferences towards integration of external health-related data in the EHR. The study was conducted in 2 phases. Phase 1 consisted of interviewing primary care providers about their perspectives on integrating lifestyle data into the EHR and showing them a rough mockup for early feedback. Phase 2 brought the providers back for more usability evaluation with a fully developed prototype. We used rapid-cycle design methods to refine the prototype interface between each usability evaluation.
Study setting and participants
We recruited primary care providers from the Section of General Internal Medicine and Department of Family Medicine at Boston University School of Medicine (BUSM) to participate in formative interviews. BUSM faculty’s main practices are located at Boston Medical Center (BMC), which is a large safety-net academic medical center in Boston, MA. The two adult primary care practices care for over 50,000 patients, with the vast majority coming from surrounding urban neighborhoods and having public insurance coverage.
Primary care providers were purposively selected to represent a cross-section of the faculty based on years in practice, technology avidity, change avidity, percent clinical effort, and clinical leadership role. Candidate providers were approached by study staff via email to participate in in-person interviews. They were recruited to participate in September 2014.
Providers who completed the first in-person interview were again approached in September 2015 to complete a second in-person interview to review a new version of the interface.
All interviews were recorded for later review and coding. Providers were not compensated for their time.
This study was approved by the Institutional Review Board at Boston University School of Medicine. Informed consent was verbally obtained from all interview participants.
Analysis and rapid-cycle iteration
The study team employed grounded-theory thematic analysis of the provider interviews to identify major themes related to the design of the user interface of external health-related data into the EHR. Independently, the interviewer and another reviewer listened to the recorded interviews to identify these themes. Each reviewer recorded ideas while actively listening to the recorded interviews. The number of interviewees that mentioned each idea was tabulated. The idea lists and tabulations from each reviewer were compared; only minor discrepancies were found and these were resolved by listening to recordings again. Ideas were grouped into larger themes during iterative conversations between the two reviewers. A final list of 4 themes related to the 5 areas covered in the initial interviews was provided to the study leaders and software developers to inform the software design.
Rapid-cycle design was employed between the study leaders and the software developers through weekly meetings over a 6 month period during which design components were reviewed for visual acceptance and usability based on initial provider interviews and study leaders’ own clinical practice. The result was a series of designs culminating in the prototype shown to providers during the second round of usability interviews; after this, minor changes were made to the final design to improve provider ease of use.
Nineteen providers were approached and ten agreed to be interviewed. All who agreed, completed the first in-person interview. Eight of the ten who completed the first interview were approached to complete the second-round interview. The remaining two providers were not approached as they were no longer faculty at Boston University School of Medicine. Five of the eight providers approached, completed the second in-person interview.
Completed Phase 1 interview only (N = 10)
Completed Phase 1 and Phase 2 interviews (N = 5)
Half-day clinic sessions/week (average)
Years in practice (average)
Comfort with EHR
Likelihood to look for new ways to experiment with technology
Comfort with computer usage overall
Comfort with medical practice change
Design themes (n = 10 interviewees)
Detailed idea (number of mentions)
Barriers to establishing healthy lifestyles
• Limited access to healthy food (6)
• Limited access to exercise options (6)
• Health literacy (6)
• Multiple competing priorities (4)
• Cost (5)
• Lack of time to cook well, exercise (4)
• Cooking and food storage facilities, homelessness (3)
• Motivation (4)
• Culture/family (2)
Features of lifestyle modification program
• In-clinic contact for warm handoff (8)
• Community-based program (5)
• Cultural, language sensitivity (4)
• Economic sensitivity (2)
• Education provided pictorially (2)
• Physical activity facilitation – pedometer (2)
• Incentives to participate (2)
• Integrate with PCP for synergy in messaging (3)
• Social experience/support, peer network (1)
• Help patient set small realistic goals (1)
• Makes it easy to keep coming back (1)
• Remote component – phone/online (1)
• Provide stress management/coping skills counseling (1)
• Outcome/evidence based (1)
• Scalable/few resources/automated in some way (1)
• Identify patient and MD goals – work toward mutual goals (1)
Reporting of outcomes to primary care provider
• Patient identified goals (5)
• Patient’s level of engagement (7)
• Biometric data – weight, blood pressure, hemoglobin A1c (6)
• Patient’s barriers, red flags that would make success near impossible (6)
• Overall sense of progress (7)
• How patient relates choices to health outcomes (1)
• Food group breakdown versus fat/carbohydrate breakdown (2)
• Patient’s action plan (1)
Integration with primary care
• Quarterly updates (6)
• Access and use in real-time with patients (7)
• Provider alerts for new data (2)
• Display in the EHR: graphical display, access coaching notes, patient goals and barriers (10)
• Train ancillary staff to use data with patients (7)
• Way to acknowledge receipt (1)
Barriers to establishing healthy lifestyles
“There’s a systemic problem of food environment; we can do a little as primary care docs but it’s a more systemic problem”
“Everybody can walk but people’s neighborhoods aren’t safe…people are afraid to walk in their neighborhoods sometimes; another challenge with our patients is affordability of healthy food and place to exercise and…depression; they have so much stress in their lives that food [becomes the answer for their stress]”
“Some of [the barriers] is [sic] health literacy; desire is there but they just don’t understand what to do”
“[Patients] need a lot of guidance and hand holding and it’s hard to do that in a primary care visit”
“From a healthcare point of view, we’re really not set up to do that whole process [of counseling around lifestyle changes]; this is a very medical model of care”
Features of lifestyle modification program
“Embedded here, would be wonderful, to have a nutritionist here on the floor to see patients before or after us [doctors]; help patients go more reliably”
“It’s great we’re doing population management, but it’s still on us…We have to get them while they’re here, if it was someone down the hall, it needs to be at the time of the visit to make the provider more efficient and to motivate the patient to do it and so the patient gets what they need at the time of the visit”
“Most successful studies employed remote counseling; in-person arm didn’t do any better than the remote arm because people don’t show up for appointments; I think remote is key”
“Patients have resources and ideas that could be really helpful for other patients; like a buddy system or sharing in group visits”
“Culturally relevant cooking classes are a big resource we have”
“I dream of a walking group in all the relevant neighborhoods that had people who spoke the right languages”
Reporting of program outcomes to the primary care provider
“Which goals were the patient working on; some sense of where the patient was at in terms of motivation and next steps”
“You can look at process or outcome; if they are participating and how often they’re doing it and have we seen any results in their weight in general”
“The big issue is that so few of my patients are engaged in any way, I just want to know that they’re engaged, someone is checking in on them, they’re having progress forward”
“I would love to know what’s happening so when I see the patient I can reinforce, like a warm handoff from the program; would know we were giving consistent advice and weren’t shooting each other in the foot”
“We know that [patients] trust [in their providers] improves some outcomes so maybe [having the primary care provider more connected] would, too”
Integration with primary care
“My role [as the primary care doctor] is to get them to go to these programs, reinforce; for patients that don’t need the whole package, I provide basics”
“I could seeing going to this [the integrated data display] rather than just the vital signs; looking at participation, how they’re doing against their goals, and what the next steps are; it’s more information than I have now”
“As soon as you log in to the chart is there a way to know that the patient is in the program…because then I could see myself wanting to use it. I can see myself wanting to use the extra information to talk with the patient”
“Would be nice if it was in the EMR, would connect to the [entire] patient [record]”
“Quarterly would be plenty; I don’t think we’re going to see results so quickly”
“You see a goal – they’re green on weight, red on calories; easy to see quickly, to use as a teaching tool for the patient”
“Ideally something that you could see easily in the context of your next visit, so you don’t have to go searching for it; would be helpful if the key point was in an easy, findable spot; I’d want a summary that didn’t take more than a click to get to”
“If you can slice it off, ‘I have this great program and you can talk with this person,’ laying the foundation and then having someone to follow up on that”
“Use this during a planned chronic care visit, but patients often save their urgent issues for their planned visits…so we never get to this”
“What helps people lose weight is having those reminders and someone that they’re coming back to, have some accountability. I don’t think we can underestimate our influence [as primary care providers] for a certain part of the population”
Iterative design process
Phase 2 usability interview suggestions (n = 5 interviewees)
Idea (number of mentions)
Graphs – seeing the trend over time is more useful than high-level dashboard) (5)
A lot of good information in graphs to use to support/guide motivational interviewing (4)
Text clear/explained well/visually pleasing (3)
Lessons completed give sense of engagement (2)
Points on graph give sense of engagement (2)
Provides more data than just from clinic visits (1)
Areas for Improvement
Where do the numbers on the dashboard come from/over what timeframe? Is it steps per day averaged over some time or just the last recorded? (4)
Either highlight time axis or make consistent from graph to graph (3)
Can we see high level content of lessons? What’s the difference between weekly & monthly lessons? (3)
Can you show BMI along with weight? (2)
Red “X” in lessons means not done or not done yet? What’s the goal number? (2)
Can we view this and document our clinic visit simultaneously? (1)
Can you drill down to graphs via clicking on the odometer? (1)
Bar bell – thought of lifting weights but probably means pounds given what is recorded (1)
Give the goal on the dashboard (rather than forcing user to do math) (1)
Can you create a toggle on the graphs to show a few different intervals? Total time, 3 months, since last visit? (1)
Steps can’t be less than zero – fix Y-axis so it doesn’t go negatively (1)
Can you give a sense of how often data should be recorded versus what is actually recorded? (1)
Give overall trend line on graphs to smooth out the noise? (1)
Would use at point of care – clinic visit, phone call (5)
Use by health coach/RN/MA in clinic (4)
Time constraints for physician use (4)
Would definitely use in patient care (motivating to review with PCP, even just having level of engagement is useful) (5)
Integrating health-related information into the clinical workflow is paramount in the ability to care for the entire patient; to be responsible for population health. Lifestyle modification programs are an invaluable part of preventive medicine,  but are often provided outside of the healthcare institution. Through semi-structured interviews, usability evaluations, and an iterative EHR interface design process, we developed an interactive widget, embedded in the providers’ usual EHR workflow, to give them seamless access to their patients' data generated in an online healthy lifestyle coaching program designed to reduce the incidence of diabetes.
Although EHRs contain a multitude of patient health data sources, they often do not easily pull in data from external systems in a clinically intuitive manner, particularly with non-traditional health data sources such as web-based lifestyle change programs [41, 42]. Patients are increasingly using online sites and mobile applications to help them get and stay healthy [43–45]. Healthcare providers have no insight into this experience or data. This novel interface provides an early experience on how to develop an interface that enables providers to easily follow patients’ progress and augment their current motivational efforts with patients. We have addressed a number of identified challenges of using externally generated health data in traditional healthcare: identifying pertinent information, integrating information into the health record, and aligning this information with the established clinical workflows .
The benefit of co-locating a group lifestyle modification program with primary care delivery has been shown, but this intervention lacked true integration of information from the program with primary care delivery . To achieve this integration, we employed user-centered rapid prototyping as has been done in other design work related to clinician acceptance of EHR enhancements [47–51]. The work to involve the end-user in the design phase is crucial to developing interventions that seamlessly integrate with existing clinical workflows, that are usable and useful [49–51].
This study has limitations given the use of a singular site and EHR for design work. While the small number of participants limits the certainty of the observations, the size of this study is similar to other rapid design development studies [48, 52–54]. These studies note that most user generated feedback about the EHR can be elicited in design iterations with a small number of participants [48, 52–54]. Thematic saturation was reached with these participants, suggesting that additional clinician input may have resulted in a similar design. The design went through multiple iterations, and resulted in an interface that clinicians were accepting of and interested in using in practice. The design is limited by the constraints of the commercial EHR and the type of data being integrated such that not all of the clinically desired EHR workflow changes were technically feasible. The design of the data integration will be directly applicable to any site using this EHR, however. The generalizability to other clinical settings and other EHRs may be more limited, although the general principles and approaches likely span these local differences. With implementation in a live clinical environment, additional refinements may occur to more fully enable clinical workflow and integrate seamlessly with the workflows of different clinical roles.
Although we have developed an integration of externally generated health-related data into the EHR workflow, there is still important work to be done to overcome operational barriers to using this new data. Time constraints during a patient-provider visit may limit the utility of the new data to the provider. Practices may do well to think of additional roles, outside of the primary care provider, which might be better suited to using this data in patient interactions, leveraging the evolving team approach to patient care and population health [55–57]. The versatility of the design interface will allow access to the data from a variety of entry points, supporting multiple clinical roles and workflows. The planned implementation in a clinical environment will test the hypotheses that having this data interface will improve patient engagement and health outcomes.
This report describes how iterative design leveraging rapid cycle usability testing approaches and involving providers, EHR designers, and health informatics researchers generated a clinically pragmatic EHR integration. This interface is sensitive to clinical workflows, aligns with clinical priorities, is intuitive and easy to use, and promises to help providers support and guide their patients. There is general consensus among study providers that having this external health-related data will help them and their patients; this has yet to be shown in a clinical setting. Studies demonstrating the value of this novel interface from an online diabetes prevention program into the EHR in diverse primary care settings will support the hypothesized importance of bringing this external data into the clinical realm for patient engagement and patient health outcomes.
DPP, Diabetes Prevention Program; EHR, electronic health record; GOALS: Goal-focused Online Access to Lifestyle Support.
This work was supported through a grant from the National Institute of Diabetes and Digestive and Kidney Diseases (R03 DK098162-02).
Availability of data and materials
Raw de-identified data can be requested from the corresponding author by any scientist wishing to use them for non-commercial purposes, without breaching participant confidentiality.
RM designed the interview guides and conducted the interviews, participated in the coding of the interview recordings, helped design the interface, and drafted the manuscript. JY participated in the interviews and the coding of the interview recordings, and helped draft the manuscript. DW participated in design of the interface, developed the software interface, and edited the manuscript. DM conceived the study, participated in the design of the interview guides and review of the interview themes, helped design the interface, and helped draft and edit the manuscript. All authors read and approved the final manuscript.
Dan Wilson is an employee of Moxe Health, a technology consulting firm that integrates software into electronic health records to improve interoperability. The other authors (RM, JY, DM) declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
This study was approved by the Institutional Review Board at Boston University School of Medicine. Informed consent was verbally obtained from all interview participants.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- EW J, N Y, E H. Adoption of certified electronic health record systems and electronic information sharing in physician offices: United States, 2013 and 2014. 2016; NCHS data brief, no 236:http://www.cdc.gov/nchs/data/databriefs/db236.htm. Accessed 15 Jan 2016.
- Gibbons MC, Wilson RF, Samal L, et al. Impact of consumer health informatics applications. Evid Rep Technol Assess (Full Rep). 2009(188):1-546.Google Scholar
- Glenn T, Monteith S. Privacy in the digital world: medical and health data outside of HIPAA protections. Curr Psychiatry Rep. 2014;16(11):494.View ArticlePubMedGoogle Scholar
- Girling R. The Elusive Promise of Electronic Health Records. 2014. http://medcitynews.com/2014/01/elusive-promise-electronic-health-records/. Accessed 15 Jan 2016.Google Scholar
- Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12(5):505–16.View ArticlePubMedPubMed CentralGoogle Scholar
- Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med. 2009;360(15):1477–9.View ArticlePubMedGoogle Scholar
- Blumenthal D, Tavenner M. The "meaningful use" regulation for electronic health records. N Engl J Med. 2010;363(6):501–4.View ArticlePubMedGoogle Scholar
- Eslami S, Abu-Hanna A, de Keizer NF. Evaluation of outpatient computerized physician medication order entry systems: a systematic review. J Am Med Inform Assoc. 2007;14(4):400–6.View ArticlePubMedPubMed CentralGoogle Scholar
- Bates DW, Kuperman GJ, Wang S, et al. Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality. J Am Med Inform Assoc. 2003;10(6):523–30.View ArticlePubMedPubMed CentralGoogle Scholar
- Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch Intern Med. 2011;171(10):897–903.PubMedPubMed CentralGoogle Scholar
- Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38(3):189–99.View ArticlePubMedGoogle Scholar
- Mollon B, Chong J, Holbrook AM, Sung M, Thabane L, Foster G. Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med Inform Decis Mak. 2009;9:11.View ArticlePubMedPubMed CentralGoogle Scholar
- Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc. 2005;12(4):438–47.View ArticlePubMedPubMed CentralGoogle Scholar
- Corrao NJ, Robinson AG, Swiernik MA, Naeim A. Importance of testing for usability when selecting and implementing an electronic health or medical record system. J Oncol Pract. 2010;6(3):120–4.View ArticlePubMedPubMed CentralGoogle Scholar
- Kelly T, Yang W, Chen CS, Reynolds K, He J. Global burden of obesity in 2005 and projections to 2030. Int J Obes (Lond). 2008;32(9):1431–7.View ArticleGoogle Scholar
- van Dieren S, Beulens JW, van der Schouw YT, Grobbee DE, Neal B. The global burden of diabetes and its complications: an emerging pandemic. Eur J Cardiovasc Prev Rehabil. 2010;17 Suppl 1:S3–8.PubMedGoogle Scholar
- Dall TM, Zhang Y, Chen YJ, Quick WW, Yang WG, Fogli J. The economic burden of diabetes. Health Aff (Millwood). 2010;29(2):297–303.View ArticleGoogle Scholar
- Hu FB, Manson JE, Stampfer MJ, et al. Diet, lifestyle, and the risk of type 2 diabetes mellitus in women. N Engl J Med. 2001;345(11):790–7.View ArticlePubMedGoogle Scholar
- Tuomilehto J, Lindström J, Eriksson JG, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344(18):1343–50.View ArticlePubMedGoogle Scholar
- Weinger K, Beverly EA, Lee Y, Sitnokov L, Ganda OP, Caballero AE. The effect of a structured behavioral intervention on poorly controlled diabetes: a randomized controlled trial. Arch Intern Med. 2011;171(22):1990–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Li G, Zhang P, Wang J, et al. The long-term effect of lifestyle interventions to prevent diabetes in the China Da Qing Diabetes Prevention Study: a 20-year follow-up study. Lancet. 2008;371(9626):1783–9.View ArticlePubMedGoogle Scholar
- Association AD. Standards of medical care in diabetes-2015 abridged for primary care providers. Clin Diabetes. 2015;33(2):97–111.View ArticleGoogle Scholar
- Nelson KM, Reiber G, Boyko EJ, III N. Diet and exercise among adults with type 2 diabetes: findings from the third national health and nutrition examination survey (NHANES III). Diabetes Care. 2002;25(10):1722–8.View ArticlePubMedGoogle Scholar
- McTigue KM, Conroy MB, Hess R, et al. Using the internet to translate an evidence-based lifestyle intervention into practice. Telemed J E Health. 2009;15(9):851–8.View ArticlePubMedGoogle Scholar
- Rose SA, Poynter PS, Anderson JW, Noar SM, Conigliaro J. Physician weight loss advice and patient weight loss behavior change: a literature review and meta-analysis of survey data. Int J Obes (Lond). 2013;37(1):118–28.View ArticleGoogle Scholar
- West DS, DiLillo V, Bursac Z, Gore SA, Greene PG. Motivational interviewing improves weight loss in women with type 2 diabetes. Diabetes Care. 2007;30(5):1081–7.View ArticlePubMedGoogle Scholar
- Saleem JJ, Patterson ES, Militello L, et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc. 2007;14(5):632–40.View ArticlePubMedPubMed CentralGoogle Scholar
- Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc. 2009;2009:548–52.PubMedPubMed CentralGoogle Scholar
- Zahabi M, Kaber DB, Swangnetr M. Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation. Hum Factors. 2015;57(5):805–34.View ArticlePubMedGoogle Scholar
- Borycki E, Kushniruk A, Nohr C, et al. Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches. Contribution of the IMIA Working Group Health Informatics for Patient Safety. Yearb Med Inform. 2013;8:20–7.PubMedGoogle Scholar
- Kannry J, McCullagh L, Kushniruk A, Mann D, Edonyabo D, McGinn T. A Framework for Usable and Effective Clinical Decision Support: Experience from the iCPR Randomized Clinical Trial. EGEMS (Wash DC). 2015;3(2):1150.Google Scholar
- Payne TH, Hines LE, Chan RC, et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc. 2015;22(6):1243–50.View ArticlePubMedGoogle Scholar
- Mann D, Knaus M, McCullagh L, et al. Measures of user experience in a streptococcal pharyngitis and pneumonia clinical decision support tools. Appl Clin Inform. 2014;5(3):824–35.View ArticlePubMedPubMed CentralGoogle Scholar
- Marcilly R, Vasseur F, Ammenwerth E, Beuscart-Zephir MC. Methods uncovering usability issues in medication-related alerting functions: results from a systematic review. Stud Health Technol Inform. 2014;205:885–9.PubMedGoogle Scholar
- Walji MF, Kalenderian E, Piotrowski M, et al. Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR. Int J Med Inform. 2014;83(5):361–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Li AC, Kannry JL, Kushniruk A, et al. Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support. Int J Med Inform. 2012;81(11):761–72.View ArticlePubMedGoogle Scholar
- Beck K, Beedle M, Bennekum A, et al. Manifesto For Agile Software Development. 2001. www.agilemanifesto.org. Accessed 1 Dec 2015.Google Scholar
- Ekas L. Five Benefits of Agile Practices in Software-Intensive Systems Development. The Invisible Thread. www.ibm.com/developerworks/community/blogs/invisiblethread/entry/five_benefits_of_agile_practices?lang=en: IBM; 2012. Accessed 15 Jan 2016.
- Montoya M. Agile: A Prescription for Improved Healthcare Technology and Delivery. www.cprime.com/2012/07/agile-development-in-healthcare-technology-industry/: cPrime; 2012. Accessed 15 Jan 2016.
- Moyer VA. Force USPST. Screening for and management of obesity in adults: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2012;157(5):373–8.PubMedGoogle Scholar
- Archer A, Bolser B, Crocker J, Miller J, Parman CC, Warner D. Managing unsolicited health information in the electronic health record. J AHIMA. 2013;84(10):70–3.PubMedGoogle Scholar
- Sharp J. Why patient-generated health data is critical to the process of care. 2015. http://medcitynews.com/2015/11/why-patient-generated-health-data-is-critical-to-the-process-of-care/. Accessed 15 Jan 2016.Google Scholar
- Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010;12(1):e4.View ArticlePubMedPubMed CentralGoogle Scholar
- Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res. 2004;6(4):e40.View ArticlePubMedPubMed CentralGoogle Scholar
- Fox S, Duggan M, Rainie L, Purcell K. The Diagnosis Difference. Pew Research Center;2013.Google Scholar
- Blonstein AC, Yank V, Stafford RS, Wilson SR, Rosas LG, Ma J. Translating an evidence-based lifestyle intervention program into primary care: lessons learned. Health Promot Pract. 2013;14(4):491–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Croon RD, Klerkx J, Duval E. Designing a Useful and Usable Mobile EMR Application through a Participatory Design Methodology: A Case Study. Paper presented at: Healthcare Informatics (ICHI), 2014 IEEE International Conference on; 15-17 Sept. 2014, 2014.Google Scholar
- Croon RD, Klerkx J, Duval E. Design and Evaluation of an Interactive Proof-of-Concept Dashboard for General Practitioners. Paper presented at: Healthcare Informatics (ICHI), 2015 International Conference on; 21-23 Oct. 2015, 2015.Google Scholar
- Mann DM, Kannry JL, Edonyabo D, et al. Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care. Implement Sci. 2011;6:109.View ArticlePubMedPubMed CentralGoogle Scholar
- Mann DM, Lin JJ. Increasing efficacy of primary care-based counseling for diabetes prevention: rationale and design of the ADAPT (Avoiding Diabetes Thru Action Plan Targeting) trial. Implement Sci. 2012;7:6.View ArticlePubMedPubMed CentralGoogle Scholar
- Mann DM, Kudesia V, Reddy S, Weng M, Imler D, Quintiliani L. Development of DASH Mobile: a mHealth lifestyle change intervention for the management of hypertension. Stud Health Technol Inform. 2013;192:973.PubMedGoogle Scholar
- Kushniruk AW, Borycki EM. Low-cost rapid usability engineering: designing and customizing usable healthcare information systems. Healthc Q. 2006;9(4):98–100. 102.PubMedGoogle Scholar
- Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform. 2004;37(1):56–76.View ArticlePubMedGoogle Scholar
- Borycki EM, Kushniruk AW, Kuwata S, Kannry J. Use of Simulation Approaches in the Study of Clinician Workflow. AMIA Annu Symp Proc. 2006;2006:61–65.Google Scholar
- Grumbach K, Bodenheimer T. Can health care teams improve primary care practice? JAMA. 2004;291(10):1246–51.View ArticlePubMedGoogle Scholar
- Vaghefi I, Hughes JB, Law S, Lortie M, Leaver C, Lapointe L. Understanding the Impact of Electronic Medical Record Use on Practice-Based Population Health Management: A Mixed-Method Study. JMIR Med Inform. 2016;4(2):e10.View ArticlePubMedPubMed CentralGoogle Scholar
- Chen EH, Bodenheimer T. Improving population health through team-based panel management: comment on "Electronic medical record reminders and panel management to improve primary care of elderly patients". Arch Intern Med. 2011;171(17):1558–9.View ArticlePubMedGoogle Scholar