Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Utilizing a user-centered approach to develop and assess pharmacogenomic clinical decision support for thiopurine methyltransferase

BMC Medical Informatics and Decision Making volume 19, Article number: 194 (2019) Cite this article

Abstract

Background

A pharmacogenomic clinical decision support tool (PGx-CDS) for thiopurine medications can help physicians incorporate pharmacogenomic results into prescribing decisions by providing up-to-date, real-time decision support. However, the PGx-CDS user interface may introduce errors and promote alert fatigue. The objective of this study was to develop and evaluate a prototype of a PGx-CDS user interface for thiopurine medications with user-centered design methods.

Methods

This study had two phases: In phase I, we conducted qualitative interviews to assess providers’ information needs. Interview transcripts were analyzed through a combination of inductive and deductive qualitative analysis to develop design requirements for a PGx-CDS user interface. Using these requirements, we developed a user interface prototype and evaluated its usability (phase II).

Results

In total, 14 providers participated: 10 were interviewed in phase I, and seven providers completed usability testing in phase II (3 providers participated in both phases). Most (90%) participants were interested in PGx-CDS systems to help improve medication efficacy and patient safety. Interviews yielded 11 themes sorted into two main categories: 1) health care providers’ views on PGx-CDS and 2) important design features for PGx-CDS. We organized these findings into guidance for PGx-CDS content and display. Usability testing of the PGx-CDS prototype showed high provider satisfaction.

Conclusion

This is one of the first studies to utilize a user-centered design approach to develop and assess a PGx-CDS interface prototype for Thiopurine Methyltransferase (TPMT). This study provides guidance for the development of a PGx-CDS, and particularly for biomarkers such as TPMT.

Peer Review reports

Background

Pharmacogenomics, the study of the effect of genetic variation on drug response, is being studied more extensively and is receiving more attention from both clinicians and researchers [1, 2]. Providers’ incorporation of pharmacogenomic results into prescribing decisions can improve both treatment efficacy and safety by accounting for the predicted drug response of individual patients [2,3,4]. The pharmacogenomic biomarker for thiopurine methyltransferase (TPMT) has been studied extensively due to the drug’s serious side effects, such as neutropenia and myelosuppression [5,6,7,8]. Patients with leukemia or inflammatory bowel disease (IBD) who carry a TPMT mutation, especially those with biallelic mutations, can experience myelosuppression when undergoing treatment with thiopurine medications such as azathioprine and mercaptopurine. Myelosuppression is a potentially dangerous decrease in the production of white blood cells that can lead to death [9]. Therefore, testing for a TPMT mutation has been recommended before starting thiopurine therapy [10,11,12].

An essential factor for successful pharmacogenomic implementation into clinical practice is the delivery of information via clinical decision support (CDS) tools. CDS can be incorporated into the electronic health record (EHR) through an interruptive alert, or via other mechanisms as part of a patient record. Pharmacogenomic clinical decision support (PGx-CDS) tools for TPMT-based therapy may improve implementation of pharmacogenomics by providing up-to-date, evidence-based guidelines and real-time decision support.

Many health care systems are starting to implement PGx-CDS for several biomarkers [10, 13,14,15]. Within the last decade, 16 different PGx-CDSs have been developed and implemented into EHRs to guide pharmacogenomic practice [16]. However, many PGx-CDSs appear to have been developed without a formal assessment of clinicians’ needs or a systematic set of design requirements. Clinical data, including genetic results, lab values, diagnoses, patient information, and drug interactions, are all variables used to make prescribing decisions. Since these data are complex and are delivered in a fast-paced environment, CDS guidance needs to be organized and presented effectively to aggregate diffuse information for prescribers and aid their medication prescribing decisions [15, 17]. In addition, poor interface design of PGx-CDS can introduce errors and promote alert fatigue, hindering the effectiveness of pharmacogenomic prescribing [15, 18]. Therefore, the objective of this study was to apply user-centered design methods to develop and evaluate a prototype of a PGx-CDS user interface for thiopurine medications to assist providers with their prescribing decisions.

Methods

Study design and setting

This study was conducted at a large, midwestern Veterans Affairs (VA) Medical Center and a co-located, major academic healthcare system (MAHS). For EHRs, the VA uses the Computerized Patient Record System (CPRS) while the MAHS uses a commercial EHR. At the time of this study, neither healthcare system used any PGx-CDS in their prescribing processes.

This study was conducted in two phases. In Phase I, we conducted qualitative interviews to assess the information needs of physicians to develop interface design requirements for PGx-CDS for TPMT. Phase I interviews were conducted at each physician’s facility. Results from Phase I were applied in Phase II to develop a prototype and evaluate the design of PGx-CDS through usability testing with physicians. Phase II usability testing was performed at the VA Health Services Research and Development (HSR&D) Human-Computer Interaction and Simulation Laboratory [19] or via remote usability testing, based on participants’ preferences. This study was approved by the Indiana University IRB (protocol 1602878929R003.)

Phase I: qualitative interview

Participant recruitment

Physicians from both healthcare systems were recruited for a 30-min, semi-structured interview conducted by a pharmacist researcher (KN). Physicians were eligible to participate if they had prescribed azathioprine or mercaptopurine in their clinical practice. Participating physicians were divided into two groups of specialty pracitce: gastroenterology and oncology with each group further divided into subspecialty of adult and pediatric practice. Each participant was verbally consented prior to the interview (the IRB approved the exemption for written consent for this phase).

Procedure and data collection

Participants were interviewed individually. The full interview guide is available in the Additional file 1: S4. Interview questions. Examples of questions include:

  1. 1.

    What is your perception and interpretation of the meaning of TPMT pharmacogenomic clinical decision support tools?

  2. 2.

    If the computer system provided decision support for pharmacogenomics, would you use it in your clinical practice? Why or why not?

    1. a.

      How might such a tool be helpful to you?

    2. b.

      What barriers do you foresee for incorporating the tool into your work?

    3. c.

      Would you like such a system to work similarly to an alert system?

    4. d.

      What information would you like to see presented in a PGx-CDS?

Interviews were audio recorded and transcribed for analysis [20]. Two independent analysts, a pharmacist researcher, and a human-computer interaction expert analyzed transcripts through a combination of inductive and deductive qualitative analysis to identify initial theme [21]. A priori codes were derived from the interview questions. Analysts met regularly to discuss and agree upon themes that were identified inductively. Similar to other literature, analysts independently coded the transcripts then met weekly to discuss discrepancies until reaching consensus [22]. A third individual, a human factors expert, was consulted if discrepancies could not be resolved. Data were analyzed and managed using NVivo 10 (QSR International).

Phase II: prototype development and usability testing

Prototype development

Data from qualitative interviews and available literature [23,24,25] were used to inform the development of an electronic prototype of the PGx-CDS in a mock EHR (Additional file 1: S5.) The prototype was developed by a research pharmacist and a human-computer interaction expert. We created a list of key features to design a PGx-CDS for TPMT using both collected interview data and human factors principles (Table 1) These key features in Table 1 focused mainly on the display while interview data provided the content of the PGx-CDS prototype. In phase II of this study, we evaluated both the display and the content of the PGx-CDS prototype via usability testing. To facilitate iterative evaluation and redesign, the prototype was not implemented in a “live” EHR. This approach, which has been used in other studies [34, 35], allowed the PGx-CDS tool to be evaluated safely, separate from patient care, providing insight on how it can be improved prior to implementation. Instead, Axure RP 7 (Axure Software Solutions, Inc., San Diego, CA) was used to design the mock-up EHR as well as the prototype (Fig. 1) This was a cost-effective, rapid approach to help physicians visualize the potential design of a future PGx-CDS interface. Participants could interact with the prototype as if they were working with an actual PGx-CDS from the EHR. This prototype provided a realistic impression of what the PGx-CDS could look like in clinical practice. This also helped us identify early errors in the design. Finally, in clinical practice, a TPMT related PGx-CDS would need to support many types of prescribers for patients being treated for leukemia, IBD, or rheumatoid arthritis. Therefore, we chose to apply a universal design approach [36] for PGx-CDS with the goal of creating a CDS that can effectively support a wide range of primary care and specialty prescribers. A universal design approach is generally robust and often requires less training and education of end users [37].

Table 1 Key features of PGx-CDS display design
Full size table
Fig. 1
figure 1

Screenshot of a PGx-CDS prototype interface. This is the 2nd version we used to conduct usability testing in phase II

Full size image

Scenario development

Four standardized, fictitious patient scenarios were developed by a team consisting of a research pharmacist, a gastroenterologist, and an oncologist to test the usability of the PGx-CDS. Two scenarios were developed for oncology participants to represent leukemia patients, one with a homozygous and one with a heterozygous mutation, and two scenarios were developed for gastroenterologists (Additional file 1: S3) to represent patients with IBD. Patient scenarios were further refined based on feedback from a gastroenterology specialist and pilot testing with a primary care provider.

Participant recruitment

Physicians from each healthcare system were recruited for a 30-min usability session. Eligibility criteria were the same as phase I: physicians were eligible to participate if they ever prescribed azathioprine or mercaptopurine in their clinical practice. Participants were consented in written prior to their sessions.

Simulation procedure

One of two moderators, each with experience moderating usability sessions, conducted each session and read a scripted introduction (see Additional file 1: S1, moderator script) to maintain consistency across participants [35]. Participants were informed that the goal was to improve the design of the PGx-CDS. Participants were asked to complete two patient scenarios that were relevant to their practice and complete tasks as though they were prescribing for real patients (Additional file 1: S2.) A physical barrier separated the moderator from participants to help reduce the potential for bias [19]. Also, the moderator refrained from offering guidance to the participant during the usability session.

All participants completed tasks in the same sequence: one patient scenario with a homozygous mutation followed by one patient scenario with a heterozygous mutation. There was no time limit for each scenario. Participants were asked to ‘think aloud’ [38] by verbalizing their thoughts, reactions, and points of confusion as they interacted with the PGx-CDS while completing the scenarios. Afterward, participants were asked to respond to an online 19-item Computer System Usability Questionnaire (CSUQ) [39] via REDCap software [40] to gather their level of satisfaction with the PGx-CDS tool. Finally, the moderator concluded the session by conducting a debrief interview with the participant to collect additional feedback about the interaction.

We used an iterative method to test our prototype because our goals were to develop and rapidly refine a provider-centered PGx-CDS to help providers with their thiopurine prescribing decisions. Specifically, any usability errors identified during testing were used to enhance the design of the PGx-CDS prototype. A similar, iterative prototyping method has been successfully applied in other research studies of health information technology [41, 42].

Data collection and outcomes measures

Qualitative usability data were collected via the “think aloud” technique with participants’ verbalizations captured by [38] Morae 3.3 (TechSmith Corp.) screen video and audio recordings. Additionally, qualitative data included information from debriefing interviews. Quantitative usability data were collected on four established measures of usability [41]: learnability, usability errors, efficiency, and satisfaction. Learnability and usability errors were evaluated by reviewing Morae® video recordings. Efficiency was measured from the time-stamped video recordings and included the amount of time the physicians spent completing each scenario as well as time spent on the PGx-CDS prototype interface. Finally, after working on the usability scenario, participants were asked to complete the CSUQ 19-item questionnaire [39] to evaluate their satisfaction with the PGx-CDS. Each participant completed the questionnaire once. In addition to capturing their overall perceptions of satisfaction of PGx-CDS, this questionnaire assessed their satisfaction for three main domains: system usefulness, information quality, and interface quality.

Results

Participants’ characteristics

Across both phases, 14 physicians (12 male; five from the VA and nine from the other MAHS) participated. Women accounted for 14% of the participants. The average age of participants was 39 years (range, 30 to 48 years). Table 2 summarizes the demographic information of participants.

Table 2 Demographic information. Physicians represented two specialties: oncology and gastroenterology
Full size table

Emergent themes from interviews

Interviews yielded 11 themes: the need for PGx-CDS for TPMT; impact of PGx-CDS on clinical workflow; lab testing preferences; perceived barriers to PGx implementation; PGx-CDS content; PGx-CDS display; references within PGx-CDS; genetic result content; display of patients’ genetic results; PGx care coordination; and examples of related software and CDS systems. Table 3 lists all 11 themes, overarching comments from participants, and example quotes. A table with definitions of each theme and comments from each specialty group is shown in the Additional file 1: S6.

Table 3 Themes identified from interviews
Full size table

Some themes have been discussed elsewhere in the literature for other types of CDS [23,24,25, 43], therefore, the Results section below focuses on themes new to the literature that can benefit the implementation of PGx-CDS. These themes are presented into two overarching categories: 1) physicians’ views about PGx-CDS, and 2) key design features needed for PGx-CDS.

Physicians’ views on pharmacogenomic clinical decision support

During interviews, all ten physicians expressed a high degree of interest in a PGx-CDS tool to help them with prescribing decisions for TPMT. They expected that PGx-CDS would help detect patients’ history of TPMT genetic mutation(s) when they prescribe either mercaptopurine or azathioprine and then provide guidance on how to “adjust the dose”. However, they emphasized that PGx-CDS needs to be incorporated into the current EHR rather than as a standalone software system and must operate in real-time to be effective.

Additionally, both specialty groups described the potential value of PGx-CDS in reducing their reliance on pharmacy-related resources. Oncologists indicated that the implementation of PGx-CDS might reduce the burden and workload for their pharmacy service. Specifically, physicians could use the PGx-CDS to look for new dosing recommendations and references rather than depend heavily on consulting the pharmacy service. In addition, one physician described that PGx-CDS can help assist with his/her work because “there is a lot of [mental] fatigue [with] mercaptopurine dosing [because of irregular dosing regimens], but to have something [like a PGx-CDS] that would function to coordinate and assimilate information into one place, [it] would actually decrease [my] fatigue over time” (Pediatric oncologist C). Similarly, GI specialists who rarely prescribed TMPT medications or encountered this genetic mutation in patients shared that if PGx-CDSs were readily available in the EHR, they would “be more inclined to use [the CDS]” rather than consulting outside clinical references or primary medical literature.

Important design features needed for clinical decision support

Content

Based on interview data, information that needs to be included in the PGx-CDS interface for TPMT decision-making are below, along with numbers of participants that expressed a need

  • genotype (homozygous mutation or heterozygous mutation) (n = 5)

  • phenotype (poor metabolizer, regular metabolizer, or high metabolizer) (n = 4)

  • the meaning of the correlation between genotype and phenotype (i.e., whether the patient is at high/low/medium risk for side effects) (n = 6)

  • potential medication risks for patients (i.e., myelosuppression, neutropenia); recommendation for dosing adjustment; (n = 9)

  • Reference citations to support the dosing recommendation. Importantly, the prescriber needs to be able to trust the PGx-CDS and “validate the recommendation with their own internal thought processes.” (n = 8)

There are several additional features described as desirable by physicians, with evidence that some PGx-CDS design needs are unique to the type of physician specialist. Pediatric oncologists (n = 3) wanted to have clarification regarding the dosing calculation - how the new dosing recommendation was calculated since it can be based on the patient’s weight or body surface area - for medication dose adjustment. One pediatric oncologist also wanted PGx-CDS to provide guidance about the specific dosing scheme that is appropriate for the individual patient, since dosing regimens for azathioprine and mercaptopurine are not the same for each day of the patients’ medication regimen. For example, in 1 week, a patient can be taking one tab of azathioprine 50 mg for 4 days (M, W, F, Sun) and 1.5 tabs of azathioprine 50 mg for 3 days (T, Th, Sat) for a total of 375 mg/week. This complicated dosing regimen can become more confusing when additional, PGx-related dosing adjustment is needed. To account for the patient’s genetic profile, the PGx-CDS needs to provide clear guidance about this regimen. Also, a physician wanted PGx-CDS to help create a long-term dosing agenda for patients with complex regimens. This physician indicated that this type of support might save time for both the prescriber and pharmacist when counseling patients on the new dosing regimen.

Additionally, while pediatric oncologists (n = 2) expressed a preference to have a dose adjustment based on patients’ body surface area, gastroenterologists recommended that the PGx-CDS provide dose adjustment based on patients’ weight. Also, several physicians from both groups (n = 3) shared that since thiopurine medications are used for maintaining therapy when treating patients with either leukemia or IBD, the PGx-CDS should also recommend specific lab tests (e.g., white blood cells, hemoglobin) that should be ordered for monitoring medications. Physicians confirmed that lab values related to TPMT medications also are important information to present in the PGx-CDS. Additionally, physicians expressed a desire for the PGx-CDS to present the following options for medication management: neutrophil count as a lab value; ability to turn on/off certain options within the PGx-CDS content; and ability for the PGx-CDS to automatically generate recommendations for a new dosing regimen within the medication ordering screen of the EHR. Finally, if a patient presents with a TPMT homozygous mutation, gastroenterologist providers (n = 3) stated that they would be more likely to prescribe a different biologic agent rather than adjust the medication dose, therefore information like Tuberculosis test and vaccine information should be available to the prescriber through PGx-CDS so they can choose the alternate biologic medication more effectively.

Display

Physicians (n = 3) from both specialties expressed the importance of having clear and concise information on the PGx-CDS: “less is more.” Also, since the TPMT mutation can cause serious side effects for patients, they envisioned an interruptive, PGx-CDS pop-up in the middle of the screen to help them pay more attention.

When participants were asked whether they want to have an option to send the supporting reference(s) for PGx-CDS recommendations to their email account, and most (n = 9) were interested, including all five gastroenterologists. In contrast, oncologists generally wanted more opportunity to review patient information in real-time since “pediatric oncology is different than the primary care setting, we have more time”. Therefore, they preferred to have all information, including reference content, presented on the PGx-CDS interface to help them make prescribing decisions.

Usability testing

Efficiency

Efficiency was measured by the time each physician spent to complete the medication orders (see Fig. 2) All participants (n = 7) successfully completed all assigned patient scenarios for the usability tasks without help from the moderator.

Fig. 2
figure 2

Efficiency of using PGx-CDS to complete a medication order

Full size image

Satisfaction

Satisfaction ratings were high across participants (Table 4). One participant (#5), however, rated the ‘interface quality’ relatively low (mean 3.3).

Table 4 Satisfaction scores for the PGx-CDS interfacea. Data for individual participants are presented as means
Full size table

During the usability sessions, participants across specialties expressed high satisfaction with this prototype. They were especially satisfied with the visual design of the alert (n = 5). One adult GI specialist mentioned, “I like that the [warning] is red to alert me about the TPMT mutation. It catches my attention.” Similarly, an adult oncologist stated, “[the] alert is visually well designed, it gets your attention, things that [are] important are in red, it gives you some options [to make decision].”

Usability errors

We highlight four usability errors that were uncovered during usability testing. First, several participants (n = 3) mentioned they were unclear whether the “Accept” button approved the PGx-CDS recommendation or approved their original medication orders. A pediatric oncologist stated, “I assume ‘Accept’ will accept a new order but it is not clear. ‘Accept new order’ will be more clear – pediatric oncologist E.” Another participant suggested changing the label to “Accept recommendation. – Adult oncologist C” Also, the initial button labeled “Others” also was unclear for several participants (n = 3). An adult GI specialist mentioned, “I like the ‘Other’ button but it does not necessarily tell me what that is.- Adult GI C” Thus, we replaced it with “Immunization and contact info” to aid clarity during usability testing. Second, the earlier version of the PGx-CDS prototype did not have a cancel button to allow participants to get out of CDS without further action. This hinders the control and freedom of prescribers when ordering medications. We added this option to the prototype during the usability testing period after two participants completed usability testing. Third, when participants used the “Override” button from the CDS interface, they stated that it lacks a warning message to prompt them about their potential dangerous actions of overriding the recommendation. An adult GI specialist commented, “[PGx-CDS] should have a second level of warning such as, ‘This patient is at a very high risk of severe leukopenia, are you sure you want to expose patient to that risk?’- Adult GI D.” Another adult GI specialist stated, “For [the] override, you should build something like, ‘Are you sure you want to override this recommendation?’. It can be malpractice if overriden.- Adult GI D” Finally, two providers mentioned that the CDS interface did not provide a patient’s weight or body surface area (BSA) to help providers confirm the dosing calculation. A pediatric oncologist mentioned, “This CDS is based on weight, we used BSA to calculate the dose. – Pediatric oncologist F” This suggestion was noted for future improvement.

Discussion

This is one of the first studies to employ a user-centered design approach to develop a clinical decision support tool for pharmacogenomics, specifically for TPMT medications. Key contributions of this research include the following: 1) application of human factors engineering to design a PGx-CDS for TPMT, 2) analysis of prescribers’ information needs and developed a PGx-CDS prototype using information collected from practicing providers, and 3) evaluation of a prototype PGx-CDS via usability testing, which could inform future implementation of PGx-CDS in EHRs to aid in the care of patients. While a user-centered design approach has been used to develop many other types of CDS based on the literature [44, 45], it appears that user-centered design approaches have been underutilized for few PGx-CDSs, including TPMT biomarker for azathioprine, or mercaptopurine. As discussed in the introduction, pharmacogenomic clinical services are still at an early stage of development. It is important to collect prescribers’ information needs and design a PGx-CDS system that supports prescribers’ workflow for pharmacogenomic prescribing. Therefore, the development of new PGx-CDSs using a combination of human factors engineering and information collected from end-users is critical for the successful adoption and implementation of pharmacogenomic services.

This study aligns with some information found in prior literature. For instance, our results agree with two studies which found that PGx-CDS should present dosing guidelines and recommendations to users rather than just information about the potential harm of the adverse interaction [15, 23]. Since the numbers of biomarkers and availability of genetic tests are increasing, it is unsustainable to expect providers to immediately recall guideline recommendations and how to interpret all pharmacogenomic results. Our results also agree with Devine et al. that PGx-CDS should include a summary of the literature related to PGx mutations, dosing recommendations, and the ability for end-users to verify the recommendation sources [15].

Despite these commonalities, our study was unique compared to prior research in that it applied a human factors approach to develop the PGx-CDS prototype for TPMT. Specifically, the development of our PGx-CDS was informed by “the four pillars of successful user-interface development,” published by Shneiderman et al. [46] to increase the chance of successful implementation. These four pillars are 1) user interface requirements, 2) guidelines documents and processes, 3) user-interface software tools, and 4) expert reviews and usability testing. The ‘user-interface software tools’ pillar was described in our Methods section. Below, we use the remaining three pillars to organize the discussion of our study results.

User interface requirements

As described by Jacko [47], clearly specifying user requirements is key to successful interface development. In this study, the process of a hybrid analysis of physician interviews allowed us to identify specific aspects that they perceived as important design features for PGx-CDS. We organized these design features into PGx-CDS “content” and “display.” The information we gather about PGx-CDS content and display allowed us to include essential information into PGx-CDS interface prototype that providers needed to make prescribing decisions. Additionally, supporting references for the PGx-CDS recommendations can help gain providers’ trust when they make prescribing decisions. These features were then separated further for distinct comments between two subspecialties. This allowed us to add options to the PGx-CDS prototype design to help support the needs of both physician specialties. For example, the PGx-CDS interface (Fig. 1) gave physicians options to directly access the full reference details for recommendations (preferred by oncologists) or to send those references to the physicians’ work email addresses (preferred by gastroenterologists). The PGx-CDS interface we designed received high satisfaction scores from both types of physician specialists in ‘information quality’ and ‘interface quality’ (Table 4), which is likely because we gathered and applied design requirements directly from users via interviews.

Guidelines documents and processes

We used our findings to create a summary of PGx-CDS design recommendations (Table 1) to assist with new PGx-CDS development for TPMT. This table was generated from both interview data and human factors principles to align the PGx-CDS with physicians’ workflow and needs. These design recommendations are not only applicable for TPMT but may also inform the development of PGx-CDS for other pharmacogenomic biomarkers. In addition, they are not EHR dependent. CDS developers can follow these recommendations to develop PGx-CDS interfaces that align with the requirements and constraints of each specific EHR system.

Expert review and usability testing

We conducted formal usability testing with physicians to evaluate the design of our novel PGx-CDS prototype interface. Overall, providers were able to learn how to use the new PGx-CDS effectively and were highly satisfied with the interface and design of the PGx-CDS for TPMT. Satisfaction rates (Table 4) were high in all three aspects of the PGx-CDS prototype (system usefulness, information quality, and interface quality). These findings demonstrate the enthusiasm of physicians for this type of PGx-CDS and the value of applying a user-centered approach in the development of a new PGx-CDS prototype. Providers will be more likely to adopt a new CDS when they are satisfied with the design of the system [48].

This study has several limitations. First, it is possible that our sample is affected by selection bias since we did not randomly select participants. Providers who were more interested in PGx-CDS may have been more willing to participate in the interview and usability evaluation than those not interested in PGx-CDS. This bias may have contributed to our positive findings for satisfaction with the PGx-CDS. We took steps to reduce other potential biases by standardizing interview and usability procedures. Second, providers were informed that patient scenarios were fictitious, and they might not respond to PGx-CDS recommendations in the same manner as they would for real patients. Third, we applied the think-aloud technique, which can sometimes lengthen time measurements. Therefore, the time physicians spent on the PGx-CDS tasks may be greater in our study than in actual clinical practice. Fourth, only 14% of our participants were women in this study. While we attempted to recruit as many women participants as possible from all eligible participants, the small number of female providers is a limitation of our study. Finally, we only recruited seven participants for usability evaluation. The small sample size limited the number of usability findings. Nevertheless, work by Nielsen et al. indicates that 5 participants can identify up to 80% of major usability errors [49].

Future directions

The results of this study can provide further guidance in the development and enhancement of other PGx-CDS interfaces, such as PGx-CDS for clopidogrel, or CYP2D6. The key features we identified in this project (Table 1) can serve as a reference for new studies to evaluate PGx-CDS interface design as well as the development of new PGx-CDSs. We plan to continue this research line with PGx-CDSs development in an actual functional system at a well-established institute in the US. Specifically, we will evaluate and compare the current implemented PGx-CDS interfaces at that institute with newly developed PGx-CDS tools that utilized the guidance from this project. Those result can help identify the strengths and weaknesses of different PGx-CDS designs in clinical practice. Also, our findings can inform related approaches to develop other PGx-CDS workflows in the EHR, such as passive PGx-CDS that appears to the side of the screen, for example, or a mix of passive and interruptive PGx-CDS. Finally, user-centered approaches can be used to develop new PGx-CDS that utilize not only pharmacogenomics results but also other factors such as the patient’s profile in the EHR and drug-drug interactions from their medication list to provide further guidance in pharmacogenomic medication decision-making for prescribers.

Conclusion

This is one of the first studies to apply user-centered design to develop a PGx-CDS tool for physicians who prescribe TPMT medications. This research resulted in a satisfying PGx-CDS interface prototype for physicians, which may be useful to patients receiving TPMT medication treatment. Our results suggest that providers are interested and in need of PGx-CDS for TPMT to improve treatment safety and efficacy. This study provides further guidance to help with the development of clinical decision support for pharmacogenomics, especially for the TPMT biomarker.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

BSA:

Body surface area

CDS:

Clinical decision support

CPRS:

Computerized Patient Record System

CSUQ:

Computer System Usability Questionnaire

EHR:

Electronic health record

GI:

Gastroenterology

HSR&D:

Health Services Research and Development

IBD:

Inflammatory bowel disease

MAHS:

Major academic healthcare system

PGx-CDS:

Pharmacogenomic clinical decision support

TPMT:

Thiopurine methyltransferase

VA:

Veterans Affairs

References

  1. NIH. What is pharmacogenomics. 2018. Available from: https://ghr.nlm.nih.gov/primer/genomicresearch/pharmacogenomics. Cited 2018 10/3.

    Google Scholar 

  2. Johnson JA. Pharmacogenetics: potential for individualized drug therapy through genetics. Trends Genet. 2003;19(11):660–6.

    Article  CAS  PubMed  Google Scholar 

  3. NIH. What is pharmacogenomics. 2017. Available from: https://ghr.nlm.nih.gov/primer/genomicresearch/pharmacogenomics. Cited 2017 01/09.

    Google Scholar 

  4. CPIT. Center for pharmacogenomics and individualized therapy. Available from: https://pharmacy.unc.edu/research/centers/cpit/. Cited 2015 02/05.

  5. Liu YP, et al. Association between thiopurine S-methyltransferase polymorphisms and thiopurine-induced adverse drug reactions in patients with inflammatory bowel disease: a meta-analysis. PLoS One. 2015;10(3):e0121745.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  6. Liu YP, et al. Association between thiopurine S-methyltransferase polymorphisms and azathioprine-induced adverse drug reactions in patients with autoimmune diseases: a meta-analysis. PLoS One. 2015;10(12):e0144234.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  7. El-Rashedy FH, et al. Clinical implication of thiopurine methyltransferase polymorphism in children with acute lymphoblastic leukemia: a preliminary Egyptian study. Indian J Med Paediatr Oncol. 2015;36(4):265–70.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Weinshilboum RM, Sladek SL. Mercaptopurine pharmacogenetics: monogenic inheritance of erythrocyte thiopurine methyltransferase activity. Am J Hum Genet. 1980;32(5):651–62.

    CAS  PubMed  PubMed Central  Google Scholar 

  9. NCI. NCI dictionary of cancer terms. 2016. Available from: http://www.cancer.gov/publications/dictionaries/cancer-terms?cdrid=44173. Cited 2016 02/05.

    Google Scholar 

  10. Bell GC, et al. Development and use of active clinical decision support for preemptive pharmacogenomics. J Am Med Inform Assoc. 2014;21(e1):e93–9.

    Article  PubMed  Google Scholar 

  11. Sanderson JD. TPMT testing before starting azathioprine or mercaptopurine: surely just do it? Gastroenterology. 2015;149(4):850–3.

    Article  PubMed  Google Scholar 

  12. Lennard L. Implementation of TPMT testing. Br J Clin Pharmacol. 2014;77(4):704–14.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Johnson JA, et al. Institutional profile: University of Florida and Shands Hospital personalized medicine program: clinical implementation of pharmacogenetics. Pharmacogenomics. 2013;14(7):723–6.

    Article  CAS  PubMed  Google Scholar 

  14. Shuldiner AR, et al. Implementation of pharmacogenetics: the University of Maryland personalized anti-platelet pharmacogenetics program. Am J Med Genet C Semin Med Genet. 2014;166C(1):76–84.

    Article  PubMed  Google Scholar 

  15. Devine EB, et al. Usability evaluation of pharmacogenomics clinical decision support aids and clinical knowledge resources in a computerized provider order entry system: a mixed methods approach. Int J Med Inform. 2014;83(7):473–83.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Hinderer M, et al. Integrating clinical decision support systems for pharmacogenomic testing into clinical routine - a scoping review of designs of user-system interactions in recent system development. BMC Med Inform Decis Mak. 2017;17(1):81.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Kawamoto K, et al. A national clinical decision support infrastructure to enable the widespread and consistent practice of genomic and personalized medicine. BMC Med Inform Decis Mak. 2009;9:17.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Horsky J, et al. Design of decision support interventions for medication prescribing. Int J Med Inform. 2013;82(6):492–503.

    Article  PubMed  Google Scholar 

  19. Russ AL, et al. Design and implementation of a hospital-based usability laboratory: insights from a Department of Veterans Affairs laboratory for health information technology. Jt Comm J Qual Patient Saf. 2012;38(12):531–40.

    Article  PubMed  Google Scholar 

  20. Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.

    Article  PubMed  Google Scholar 

  21. Swain J. A hybrid approach to thematic analysis in qualitative research: using a practical example. London: SAGE Research Methods Cases; 2018.

  22. Saldaña J. The coding manual for qualitative researchers; 2016.

    Google Scholar 

  23. Welch BM, Kawamoto K. The need for clinical decision support integrated with the electronic health record for the clinical application of whole genome sequencing information. J Pers Med. 2013;3(4):306–25.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Johansen Taber KA, Dickinson BD. Pharmacogenomic knowledge gaps and educational resource needs among physicians in selected specialties. Pharmgenomics Pers Med. 2014;7:145–62.

    PubMed  PubMed Central  Google Scholar 

  25. Bates DW, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523–30.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Wickens CD, et al. Introduction to human factors engineering. 2nd ed. New Jersey: Prentice-Hall, Inc.; 2003.

  27. Salvendy G. Handbook of human factors and ergonomics; 2006.

    Book  Google Scholar 

  28. Phansalkar S, et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc. 2010;17(5):493–501.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Wogalter MS, Kalsher MJ, Rashid R. Effect of signal word and source attribution on judgments of warning credibility and compliance likelihood. Int J Ind Ergon. 1999;24(2):185–92.

    Article  Google Scholar 

  30. Smith-Jackson TL. Warnings and risk communication by Michael S. Wogalter, David M. DeJoy, & Kenneth R. Laughery 1999, 365 pages, $109.95, vol. 10(1). Philadelphia: Taylor & Francis ISBN 0-7484-0266-7; 2002. p. 29.

    Google Scholar 

  31. Sanders MS, McCormick EJ. Human factors in engineering and design, vol. xiii. 7th ed. New York: Mcgraw-Hill Book Company; 1993. p. 790.

    Google Scholar 

  32. Marino CJ, Mahan RR. Configural displays can improve nutrition-related. decisions: an application of the proximity compatibility principle. Hum Factors. 2005;47(1):121–30.

    Article  PubMed  Google Scholar 

  33. Zhang J, et al. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform. 2003;36(1–2):23–30.

    Article  PubMed  Google Scholar 

  34. Saleem JJ, et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc. 2007;14(5):632–40.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Russ AL, et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc. 2014;21(e2):e287–96.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Story MF, et al. The universal design file : designing for people of all ages and abilities. Raleigh; Washington, DC: School of Design, the Center for Universal Design, NC State University; U.S. Dept. of Education, Office of Educational Research and Improvement, Educational Resources Information Center; 1998.

    Google Scholar 

  37. Russ AL, et al. Prescribers’ interactions with medication alerts at the point of prescribing: a multi-method, in situ investigation of the human-computer interaction. Int J Med Inform. 2012;81(4):232–43.

    Article  PubMed  Google Scholar 

  38. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. 2009;78(5):340–53.

    Article  PubMed  Google Scholar 

  39. Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum-Comput Interact. 1995;7:57–78.

    Article  Google Scholar 

  40. Harris PA, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  PubMed  Google Scholar 

  41. Russ AL, et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users’ detection of medication discrepancies. J Biomed Inform. 2018;82:178–86.

    Article  PubMed  Google Scholar 

  42. Melton BL, et al. Iterative development and evaluation of a pharmacogenomic-guided clinical decision support system for warfarin dosing. Appl Clin Inform. 2016;7(4):1088–106.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Khelifi M, et al. Design recommendations for pharmacogenomics clinical decision support systems. AMIA Jt Summits Transl Sci Proc. 2017;2017:237–46.

    PubMed  PubMed Central  Google Scholar 

  44. Brunner J, et al. User-centered design to improve clinical decision support in primary care. Int J Med Inform. 2017;104:56–64.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Melnick ER, et al. Patient-centered decision support: formative usability evaluation of integrated clinical decision support with a patient decision aid for minor head injury in the emergency department. J Med Internet Res. 2017;19(5):e174.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Shneiderman B, et al. Designing the user interface: strategies for effective human-computer interaction. New Jersey: Addison-Wesley Publishing Company; 2009. p. 624.

  47. Jacko JA. Human-computer interaction handbook: fundamentals, evolving technologies, and emerging applications. 3rd ed. New Jersey: CRC Press, Inc.; 2012. p. 1518.

  48. HealthIT.gov. Plan for successful CDS development, design, and deployment. Available from: https://www.healthit.gov/sites/default/files/3-4-3-successful-cds.pdf. Accessed 07 Aug 2019.

  49. Nielsen J. Usability engineering. New Jersey: Morgan Kaufmann Publishers Inc.; 1993. p. 358.

Download references

Acknowledgements

We gratefully acknowledge Brian Porter and Diana Natividad for assisting with participant recruitment, Erica Evans’s assistance with basic manuscript editing, and Rachel Gruber’s assistance with manuscript submission and formatting. The view expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or U.S. government.

Funding

This work was supported in part by VA HSR&D grant CIN 13–416 at the Richard L. Roudebush Veterans Affairs Medical Center in Indianapolis, Indiana and A HSR&D Local Initiated Project Award. Dr. Nguyen was supported by HSR&D HFP 04–148 funding for medical informatics fellowship. Both VA funds provided resources and space for Dr. Nguyen to design, collect, analyze and intepretate data of this manuscript. Dr. Russ was supported in part by a VA HSR&D Career Development Award, CDA 11–214 to help design and analyze the study. Manuscript preparation and revision was made possible in part by support from the Regenstrief Institute, Center of Health Services Research and University of Florida, College of Pharmacy.

Author information

Authors and Affiliations

  1. Department of Pharmacotherapy and Translational Research, University of Florida, College of Pharmacy, 1225 Center Drive, Gainesville, FL, 32610, USA

    Khoa A. Nguyen

  2. Center for Health Services Research, Regenstrief Institute Inc., 1101 W 10th St, Indianapolis, IN, USA

    Khoa A. Nguyen, David A. Haggstrom & Thomas F. Imperiale

  3. Center for Health Information and Communication, Department of Veterans Affairs (VA), Veterans Health Administration, Health Services Research and Development Service (CIN 13-416), Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN, 46202, USA

    Khoa A. Nguyen, Himalaya Patel, David A. Haggstrom, Thomas F. Imperiale & Alissa L. Russ

  4. Department of Pharmacy Practice, College of Pharmacy, Purdue University, 640 Eskenazi Avenue, Indianapolis, IN, USA

    Khoa A. Nguyen, Alan J. Zillich & Alissa L. Russ

  5. Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, USA

    David A. Haggstrom & Thomas F. Imperiale

Authors
  1. Khoa A. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Himalaya Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David A. Haggstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alan J. Zillich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thomas F. Imperiale
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alissa L. Russ
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KAN developed the idea for this study with input from ALR, AJZ, TFI, and DAH. KAN led the development of the PGx-CDS design with input from ALR and HP. KAN drafted the interview guide and experimental scripts, developed the usability session design, and conducted the usability analysis, with guidance from ALR. KAN conducted all interviews. KAN also developed the patient scenarios and created the software versions of the prototype. KAN and HP conducted usability sessions and associated evaluations. KAN and HP analyzed interview transcripts with guidance from ALR. KAN analyzed video data for usability issues and summarized the findings with input from HP, ALR, AJZ, DAH, and TFI. KAN drafted the initial manuscript; all authors provided scientific critiques to produce the final version of the article. All authors have read and approved the manuscript and its revisions.

Corresponding author

Correspondence to Khoa A. Nguyen.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Indiana University Institutional Review Board (IU IRB,) Protocol 1602878929. Participants in phase I were verbally consented as approved by the IU IRB while participants in phase II were consented in written prior to their procedures.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: S1.

Moderator script. S2. Task sheet for oncologist participants. S3. Example of a case for gastroenterologists. S4. Interview questions. S5. Screenshot of a mock-up electronic health record. S6. Definitions and comments from each group.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, K.A., Patel, H., Haggstrom, D.A. et al. Utilizing a user-centered approach to develop and assess pharmacogenomic clinical decision support for thiopurine methyltransferase. BMC Med Inform Decis Mak 19, 194 (2019). https://doi.org/10.1186/s12911-019-0919-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-019-0919-4

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us