Skip to main content
Download PDF

Patient perceptions of an electronic-health-record-based rheumatoid arthritis outcomes dashboard: a mixed-methods study

BMC Medical Informatics and Decision Making volume 24, Article number: 302 (2024) Cite this article

Abstract

Background

Outcome measures are crucial to support a treat-to-target approach to rheumatoid arthritis (RA) care, yet their integration into clinical practice remains inconsistent. We developed an Electronic Heath Record-integrated, patient-facing side-car application to display RA outcomes (disease activity, functional status, pain scores), medications, and lab results during clinical visits (“RA PRO Dashboard”). The study aimed to evaluate patient perceptions and attitudes towards the implementation of a novel patient-facing dashboard during clinical visits using a mixed-methods approach.

Methods

RA patients whose clinicians used the dashboard at least once during their clinical visit were invited to complete a survey regarding its usefulness in care. We also conducted semi-structured interviews with a subset of patients to assess their perceptions of the dashboard. The interviews were transcribed verbatim and analyzed thematically using deductive and inductive techniques. Emerging themes and subthemes were organized into four domains of the Ecological Model of Health.

Results

Out of 173 survey respondents, 79% were interested in seeing the dashboard again at a future visit, 71% felt it improved their understanding of their disease, and 65% believed it helped with decision-making about their RA care. Many patients reported that the dashboard helped them discuss their RA symptoms (76%) and medications (72%) with their clinician. Interviews with 29 RA patients revealed 10 key themes: the dashboard was perceived as a valuable visual tool that improved patients’ understanding of RA outcome measures, enhanced their involvement in care, and increased their trust in clinicians and the clinic. Common reported limitations included concerns about reliability of RA outcome questionnaires for some RA patients and inconsistent collection and explanation of these measures by clinicians.

Conclusions

In both the quantitative and qualitative components of the study, patients reported that the dashboard improved their understanding of their RA, enhanced patient-clinician communication, supported shared decision-making, and increased patient engagement in care. These findings support the use of dashboards or similar data visualization tools in RA care and can be used in future interventions to address challenges in data collection and patient education.

Peer Review reports

Background

Shared decision-making, health literacy, and effective communication around outcome measures are important components of a treat-to-target approach in rheumatoid arthritis (RA) [1,2,3], a chronic autoimmune disease characterized by significant pain, joint swelling and stiffness, inflammation, and fatigue [4]. Standardized RA outcome measures provide reliable indicators of patients’ experiences of their disease activity (DA), functional status, symptoms, and pain levels over time [3, 5]. Three outcome measures have been identified as part of an RA “core set”: 1) Patient global assessment of disease utilized as a component of a composite DA measure, 2) Pain, and 3) Physical function (PF) [6]. Thus, routine collection and assessment of these outcomes is recommended by the American College of Rheumatology (ACR) to monitor changes over time, improve patient outcomes, guide clinical decision making, and promote patient-centered care [7,8,9,10].

Despite recommendations for regular collection of RA outcome measures and their proven benefits for patient-clinician communication and patient health outcomes, use of these measures is inconsistent in clinical care [11,12,13,14]. A few studies have reported on perceived barriers to the integration of RA outcome measures into clinical practice by clinicians, including concerns about the complexity of outcome measure questionnaires, and the belief that incorporation of these measures might hinder interactions with patients [15] or detract from a patient-focused encounter [16].

Several studies have shown that use of electronic dashboards that display outcome measures during clinical visits positively impacted shared decision-making, improved collection and integration of patient reported outcomes (PROs) into the clinical workflow, enhanced symptom control, and increased health-related quality of life [17,18,19,20]. In rheumatology, several dashboards have been developed to help improve quality of care [21, 22] and the DANBIO register in Denmark features a dashboard for clinical practice that has been utilized to collect PROs from patients and track RA outcomes over time [23]. However, these dashboards were primarily designed for clinician use, as opposed to being patient-facing. A recent study described adaptations made to the EPIC native rheumatology module to implement a patient-facing visualization that tracked PROs for patients with RA and juvenile idiopathic arthritis [24]: this dashboard helped users make sense of health information, communicate more effectively, and make decisions.

At our institution, we recently used a human-centered design process to develop and deploy an Electronic Heath Record (EHR)-integrated, patient-facing side-car application to display RA outcomes (disease activity, functional status, pain scores), medications, and lab results during clinical visits (“RA PRO Dashboard”). The dashboard was designed to be patient-facing and customizable in its display of data [25]. In this study, we aimed to evaluate patient perceptions and attitudes around RA outcomes and the implementation of this novel patient-facing dashboard using a mixed-methods approach guided by the Ecological Model of Health (EMH) framework. Roll-out of the dashboard provided a unique opportunity to query patients about the dashboard interface and about RA outcome measures more generally. To our knowledge, this is among the first studies to focus on patient perspectives on the benefits and limitations of using a digital health tool displaying RA outcome measures in clinical practice.

Methods

RA PRO dashboard development

We used principles of human-centered design [26, 27] to develop an EHR-integrated, patient-facing sidecar RA PRO dashboard application. As described previously, the dashboard pulls RA outcome data from the EHR that are collected routinely during clinical visits (CDAI, PROMIS-PF and pain) and displays graphs showing their trajectories over time, including established clinical targets, as well as medications and lab results (Fig. 1) [25]. The data shown on the dashboard are derived from existing structured fields in the EHR. Data from 2014 onwards are included (as are values from the same day’s visit). The dashboard is designed to be shared by clinicians with patients, either on the computer screen during in-person clinical visits or using a share-screen function during telehealth visits.

Fig. 1
figure 1

Screenshot of the RA PRO dashboard. Nasrallah C, Wilson C, Hamblin A, Young C, Jacobsohn L, Nakamura MC, Gross A, Matloubian M, Ashouri J, Yazdany J, Schmajuk G. Using the technology acceptance model to assess clinician perceptions and experiences with a rheumatoid arthritis outcomes dashboard: qualitative study. BMC Med Inform Decis Mak. 2024 May 27;24(1):140. https://doi.org/10.1186/s12911-024-02530-2. PMID: 38802865; PMCID: PMC11129391

Full size image

Clinical setting

The dashboard was implemented at an academic rheumatology clinic in northern California which serves approximately 500 patients with RA. The clinic is staffed by a rheumatology nurse practitioner and approximately 12 attending physicians who see patients between 1 and 4 half-days per week. Attendings also supervise 6 rheumatology trainees, who rotate through the clinic for 1–3 years.

Patient-reported components of RA outcome measures are typically collected by medical assistants (MAs) when the patient checks in for their in-person visit, or when they initially log on to a telehealth visit. Patients are asked to complete 1) a pain question to assess arthritis pain over the past seven days using a visual-analog scale of 0–100, 2) the PROMIS-PF questionnaire to assess PF, and 3) the patient global assessment of RA activity, which is used by clinicians to calculate the CDAI. After completing the forms, MAs input the raw scores into structured fields in the EHR, whereby the PROMIS-PF scores are converted into t-scores [28]. Once the clinician enters the evaluator global assessment and tender and swollen joint counts, the EHR uses the patient global assessment score to generate the DA score as part of the CDAI [29]. Adherence to these workflows is routine: PROMIS-PF and pain scores are documented in the EHR at nearly 80% of visits (both in-person and telehealth); CDAIs are captured at approximately 60% of in-person visits.

The RA PRO dashboard went live in August 2021 as part of a stepped-wedge pragmatic cluster-randomized trial. Clinicians were trained by study coordinators (CW, AH, CH, CY) on how to navigate the dashboard. Clinicians could engage with the dashboard or share it with their patients at their own discretion. During our study period (August 2021 through April 2023), 19.8% of RA visits had confirmed interactions with the dashboard, representing 318 unique patients who saw the dashboard at least one time during this period.

Study design

This study used a mixed-methods approach to evaluate the patients’ experiences, acceptability, and perceived usefulness of the RA PRO dashboard shared by their clinician during their visit. By combining quantitative and qualitative data, we aimed to gain a thorough understanding of patient perspectives and identify any suggestions for potential improvements to the dashboard. This approach allowed us to complete a robust analysis that compared numerical survey results with detailed patient feedback.

Data collection

Quantitative data collection

Eligibility criteria for participation in the patient surveys required that patients be ≥ 18 years of age, with ≥ 2 International Classification of Diseases (ICD) codes for RA at least 30 days apart, receiving care at the academic rheumatology clinic, proficient in English (primary spoken language documented as English in the EHR), and had seen the dashboard during their most recent rheumatology clinic visit. The inclusion criteria ensured that patients had direct experience with the dashboard prior to completing the survey. All patients who saw the RA PRO dashboard during the study period were invited to participate in the quantitative survey (n = 318).

Patients who were interested in participating in the survey were explained the aim of the study and verbally consented to participate. The survey included statements regarding patients’ experiences with the RA PRO dashboard, evaluating its acceptability and usefulness during the visit, and their willingness to see it in future visits, using a 4-point Likert-type scale (“Yes", “Somewhat”, “No”, “Unsure”; see specific survey items in Table 2). We included a balancing question: “Do you think using the dashboard changed the focus of your visit?” to understand whether using the RA PRO dashboard during the clinic visit may have adversely affected the patient-clinician interaction during the visit (e.g., shifted the focus from the patient to the computer). Therefore, this question was an “inverse” measure where a “No” response reflected positive impressions of the dashboard. Patients could complete the survey more than once if they had multiple visits in the clinic during the study period.

Qualitative data collection

Eligibility criteria for patient interviews required patients had completed at least one quantitative survey above. Purposive sampling was used to ensure that a range of usability perspectives were captured (i.e., patient quantitative survey responses were varied, and that patients were cared for by different clinicians in the clinic) [30].

Semi-structured interviews were conducted with a subset of patients who completed at least one dashboard survey (discussed above) aiming to gather in-depth insights into patients' experiences with the RA PRO dashboard and perceptions towards its usefulness and usability. To ensure that a range of usability perspectives were included, we invited 141 patients treated by various clinicians to participate in these interviews. First, study team members (CW, AH, CH, CY) reached out to eligible patients in-person at the clinic or by phone after their telehealth visit with their clinician and explained the purpose of the study, its voluntary nature, and asked them to contact the research team if interested in scheduling an interview. An experienced researcher trained in qualitative research methods (CN) conducted virtual interviews with patients via Zoom between November 2022 and April 2023 and team members (CW, CY) took notes. The interviews were guided by a semi-structured interview guide developed by CN (Additional File 1) that assessed patients’ perceptions towards RA outcome measures and explored their experience with the RA PRO dashboard shared by their clinician during their visit. The interview guide was tested with two rheumatologists (JY, GS), and piloted with the first few patients included in the study to ensure that the questions were understandable and relevant to end users. No major changes were required as a result of this testing. Interviews lasted between 20 to 45 min and were audio-recorded and transcribed verbatim. Later, transcripts were checked against recordings for quality assurance. Anonymity of patients was protected through de-identifying the transcripts. Study activities were approved by the Institutional Review Board at the University of California, San Francisco.

Theoretical framework

Our analysis was guided by the EMH, which provides a framework to assess multiple factors at the patient, interpersonal, clinician, and clinic levels that might influence patient perceptions of RA outcome measures and their experience with the RA PRO dashboard [31]. The EMH framework addresses the complexities, interactions, and interdependencies between individual, interpersonal, and organizational determinants of health. It has been widely used in public health settings to explore patient preferences, perceptions, and attitudes towards different health matters, to identify modifiable factors at the appropriate level, and ultimately to design targeted interventions to improve clinical inertia.

Analysis

Quantitative analysis

Survey data were analyzed descriptively using STATA 18 [32] including frequencies and mean scores for scale items. Results were organized as per the levels of the EMH, excluding the clinical level domain since survey responses did not capture information for that domain. For participants that completed more than one survey over the study period (n = 57), only their initial survey response was included in the primary analysis. A separate table was generated showing responses of the most recent survey for the 57 patients with more than one survey collected.

Qualitative analysis

Responses to open-ended interview questions were analyzed thematically using deductive and inductive techniques to identify key themes and subthemes [33]. Open-ended questions allowed respondents to elaborate on specific aspects of their experiences with the RA PRO dashboard, capturing nuanced feedback that complimented the quantitative data. Using Dedoose version 7.0.23 [34], an experienced qualitative researcher (CN) reviewed the transcripts to apply a set of deductive codes based on the EMH and created a preliminary set of inductive codes to capture emergent ideas within and across the interviews. Applied codes were later discussed, edited, and organized into a structured codebook with definitions for each code. Three members of the study team (CW, AH, CH) independently applied the codes to each transcript (2 coders per transcript) and discrepancies were resolved via consensus meetings with a fourth coder (CN). Data were then reviewed by deductive and inductive codes to identify emerging themes and subthemes, which were organized into the domains of the EMH using a systematic and iterative process [35] (Fig. 2). Then, all coded excerpts were summarized and sorted by their relevant themes and subthemes with quotes illustrating how each theme or subtheme served as a benefit or disadvantage of RA outcome measures in general and the RA PRO dashboard in specific as perceived by the patients. The study complied with the Consolidated Criteria for reporting Qualitative Research (COREQ) checklist (Additional File 2).

Fig. 2
figure 2

Generated themes and sub-themes as the ecological model

Full size image

Results

Survey findings

A total of 173 patients completed the survey. Participant demographics are summarized in Table 1. Responses to survey items are shown in Table 2. Visits preceding completion of the questionnaires were mostly in-person (89%). Respondents had visits with 15 different clinicians, although the majority saw one of six clinicians (Additional File 3).

Table 1 Socio-demographic characteristics of study participants
Full size table
Table 2 Summary of survey responses, categorized according to the Ecological Model of Health (N = 173)
Full size table

Patient level

Survey responses showed that more than two-thirds of patients (78.6%) wanted to see the dashboard again at a future visit. Regarding individual benefits of the dashboard, 70.5% of patients reported that it helped them understand more about their diseases, more than half (59.5%) understood why they were prescribed certain medications, and slightly less than half (42.8%) agreed that the dashboard helped them share information about their RA with others.

Interpersonal level

Three-quarters of patients felt that the dashboard helped them talk to their doctor about their RA or symptoms (75.7%) and medications (72.3%). 61.8% of patients considered that the dashboard helped them talk with their clinician about other things that are important to managing their RA, with slightly more than half of respondents (56.1%) reporting that the dashboard helped their overall patient-clinician communication. A significant majority of patients (64.7%) found the dashboard useful in improving their decision-making regarding RA care. With regards to the balancing question, a minority of patients (22.0%) reported a change in the focus of their clinical visits due to the dashboard.

Clinician level

Slightly less than half of patients (45.7%) felt that using the dashboard helped their clinicians better understand what is most important to them, and around a third of respondents (37.6%) considered that using the dashboard gave their clinician additional information that they would otherwise not have had.

Findings from the most recent survey responses among patients with more than one survey completed (n = 57) did not differ substantially from the primary analysis (Additional File 4).

Interview findings

141 patients who completed the quantitative surveys were invited to participate in patient interviews (characteristics in Additional File 5). We ceased patient recruitment and data collection after completing 29 interviews with patients treated by 9 unique physicians and reaching data saturation (later interviews were not bearing new perspectives on RA outcome measures or the RA PRO dashboard). Interview participant demographics are summarized in Table 1. The majority of patients interviewed were well-educated (79.3% had at least a college or graduate degree) and demonstrated a high level of confidence in completing medical forms independently (75.9%). Quantitative survey responses from the 29 patients who completed interviews were similar to those observed in the primary analysis sample of 173 (Additional File 6).

Ten key themes were identified, representing patients’ perceived benefits and limitations of collecting RA outcome measures and experiences with the RA PRO dashboard in clinical practice. Generated themes and subthemes were organized into the four levels of influence according to the EMH: Patient, interpersonal, clinician, and clinic level. Below, we summarize themes and subthemes within each level and provide exemplary quotes.

Patient level

Three main themes emerged at the individual patient level: self-awareness, visualizing disease trajectory, and perceived reliability of RA outcome questionnaires. Most patients reported that completing RA outcome questionnaires and discussing changes over time with their clinician by reviewing the RA PRO dashboard increased their understanding of their symptoms and improved their knowledge about their disease prognosis (Table 3.: Q1.1). In terms of knowledge about usability of RA outcome measures, patients’ understanding of the role of these measures in RA care depended on clinicians’ use of the dashboard. For instance, patients whose clinician used the RA PRO dashboard at every clinical visit to discuss updated RA outcome scores expressed high knowledge about the measures collected and understanding of target scores. They stated that the content of the dashboard provided them with “information that they couldn’t understand before” regarding score calculation and interpretation (Table 3.: Q1.2). However, patients whose clinician did not frequently use the dashboard or did not discuss RA outcome measure results showed limited understanding of the role, benefits, and use of these measures in RA care (Table 3.: Q1.3).

Table 3 Benefits and limitations of collecting RA outcome measures and using the RA PRO dashboard as per the Ecological Model of Health—Interview quotes
Full size table

Regular RA outcome measure assessment and visualization of the dashboard’s graphs showing disease trajectory helped patients understand trends in their DA and PF in relation to medications prescribed. While some patients found this motivating and encouraged adherence to their treatment plan when symptoms improved over time (Table 3.: Q1.4), others found it distressing when target scores were not achieved, or symptoms worsened (Table 3.: Q1.5).

Perceived reliability of RA outcome questionnaires was a third theme generated at the individual level pertaining to patients’ perceived limitations of outcome measures themselves. A few patients considered numeric scales used to measure RA outcomes as inaccurate, given that RA symptoms including arthritis pain fluctuate over time (Table 3.: Q1.6). Others felt the scales were too subjective, particularly when quantifying their symptoms with numeric ratings (Table 3.: Q1.7). They stated that these scales did not objectively capture the nuances of their RA experiences and that the interpretation of symptoms could vary significantly between individuals. One patient reported having difficulty answering questions on the PROMIS-PF questionnaire, citing that the statements were unclear, vague, or irrelevant to their age or medical condition (Table 3.: Q1.8).

Interpersonal level

At the interpersonal level, three themes were generated: patient-clinician communication, patient involvement in care, and focus of the clinical visit. In terms of patient-clinician communication, patients felt that using the RA dashboard to discuss outcome measures enhanced their understanding of their clinician’s perception of their disease, especially in cases where there was discordance between patient and clinician RA outcome scores (Table 3., Q:2.1). Further, patients who reported that their clinician collected, reviewed, and provided a thorough explanation of RA outcome scores at every visit were receptive to the dashboard and expressed a desire to review it with their clinician regularly. They considered RA outcome scoring and target measures (specifically CDAI) to be important topics of discussion during clinical visits (Table 3., Q:2.2). Conversely, a few patients reported that their clinician provided less detailed explanations of their RA outcome scores and quickly shared the dashboard with them during visits. They expressed frustration and confusion about the dashboard in general and usability of RA outcomes more specifically (Table 3., Q:2.3). Several patients felt that the use of the RA PRO dashboard helped to initiate new topics of discussion not usually asked about by clinicians that extended beyond traditional healthcare topics, such as factors that contributed to disease flares and their resulting effects (inflammatory diets, personal and professional goals and responsibilities, and sleep routines) (Table 3., Q:2.4). However, one patient, who explained that the clinician does not frequently use the dashboard or discuss RA outcomes during clinical visits, stated that these measures added no new information and provided redundant topics of discussion that might have naturally arisen during the usual clinical encounter (Table 3., Q:2.5).

Regarding involvement in their care, patients explained that collecting RA measures and reviewing the dashboard with their clinician prompted them to recall specific elements that triggered their symptoms, facilitating more detailed discussions with their clinician (Table, Q:2.6). Moreover, some patients reported that filling out RA outcome questionnaires encouraged them to reflect more thoughtfully on their symptoms, especially when responding to the pain question (Table 3., Q:2.7). Additionally, patients discussed that the data presented in the dashboard facilitated goal setting and conversations about goals of care with their clinician. These discussions encompassed various aspects such as treatment initiation, modification, medication side effects, DA targets, maintenance of physical functionality and symptom management, including pain control (Table 3., Q:2.8).

Lastly, one participant expressed that completing RA outcome forms prior to the visit and utilizing the dashboard to discuss scores and changes over time facilitated a more structured approach to clinical visits. This process streamlined communication, guided discussions towards pertinent health topics, and fostered productive health conversations (Table, Q:2.9).

Clinician level

The clinician level included two themes: screening tool and work efficiency. The majority of patients described the RA PRO dashboard as a supportive screening tool that aided their clinician in visualizing RA outcome scores during clinical visit. They emphasized that the dashboard offered an overarching view of their condition and assisted clinicians in consistently monitoring their DA, PF, and pain over time, enabling them to monitor patient symptoms effectively. This facilitated shared decision-making and improved treatment outcomes. (Table 3., Q:3.1).

Other patients explained that assessing RA outcome data prior to the visit and sharing the dashboard with patients to discuss updated scores might enhance clinicians’ work efficiency. They felt this approach led to more effective visits by enabling clinicians to prepare for consultations and granting them quicker access to patient data (Table 3., Q:3.2).

Clinic level

The clinic level includes two themes: quality of care and RA outcome measures workflow. Participants reported both benefits and limitations regarding the collection of RA outcome measures at the clinic. Specifically, patients highlighted the positive impact of collecting RA outcomes and using the dashboard on their perception of the clinic’s quality of care. They emphasized that it built trust in the department, showcased professionalism, and ensured continuity of care and a commitment to monitoring their disease (Table 3., Q:4.1). Despite their appreciation for collecting RA outcomes, many participants expressed a lack of interest in these measures due to the data collection process employed by the clinic. Some noted that the irregularity in collecting RA outcomes at each visit had a negative impact on their perception of the benefits and utility of these measures by their clinician. As a result, they requested more consistent surveying at each visit to enhance their engagement with the process (Table 3., Q:4.2). Finally, many patients considered completing RA outcome measures as an additional burden, especially when they did not receive explanations from the medical support staff upon receiving the forms. They particularly noted the lack of information regarding the measures being collected, their purpose, use, and benefits, which contributed to their perception of the process as burdensome (Table 3., Q:4.3).

Discussion

In this study, we used a mixed-methods approach incorporating both surveys and interviews to evaluate patient perceptions of a new health Information Technology (IT) tool that displays RA outcome measures in routine clinical practice. Guided by the EMH as an analysis framework, both the quantitative and qualitative components of the study found that patients endorsed and valued the use of the RA dashboard during clinic visits. In general, they viewed the RA PRO dashboard as a valuable visual tool that enhanced their understanding of their disease and assisted their physicians in gaining a clearer understanding of their priorities The interviews highlighted several key benefits of the dashboard, including its role in enhancing patients’ knowledge, facilitating communication about RA, medications, and other disease-related topics with their clinicians, improving clinical encounters by fostering patient involvement in RA care, and bolstering trust in clinicians and the clinic. Interviews underscored the significance patients attributed to RA outcome measures, emphasizing their view that routine assessment and use of these measures were crucial to RA care, as the questionnaires prompted them to reflect on their health. Although generally positive, both sources of data suggested that some but not all patients believed in the ability of the dashboard to improve inter-personal communication.

Our findings are consistent with the results of other studies assessing the positive impacts of dashboards on patient knowledge, shared decision-making, and patient-clinician communication in the fields of rheumatology, endocrinology, oncology, and psychiatry [17,18,19, 24, 36]. These patient-facing dashboards have been shown to ()facilitate patient communication with clinicians making it easier to discuss health concerns, treatment options, and outcomes. Our findings expand upon these findings, as patients stated that the RA PRO dashboard is not merely a tool for providing clinicians with easier and faster access to their RA outcomes data but empowers them to analyze and use their own data to guide self-management. Our findings also align with prior qualitative studies, which emphasized that patients value clear communication about the purpose of PROs, their integration in consultations, and guidance on how to complete PRO forms correctly [37].

Despite interest in the dashboard, our study highlights several barriers related to the logistics of collecting RA outcome measures. In line with previous studies, we found that inconsistencies in guidance on administering RA outcome questionnaires [38] and inadequate explanations about the importance of reporting these measures may reduce their perceived usefulness, hinder patient acceptance and engagement and lead to lower PRO completion rates due to respondent burden [39,40,41,42]. Addressing this challenge may require a streamlined workflow for the routine collection of RA outcomes, alongside continuous monitoring of adherence to the workflow, including front desk staff, MAs, nurses, and clinicians. Providing comprehensive training for medical support staff can enhance their understanding of the significance of outcome measures in RA care and improve their ability to communicate effectively with patients when administering questionnaires. Additional barriers identified in the study were associated with specific questions included in the RA outcome measure questionnaires, such as perceptions that some of the questions were irrelevant or unclear or lack of specificity in response options tailored to patients' symptoms [37, 43, 44]. These findings indicate that while the dashboard itself is seen as valuable, clinic workflow challenges for collecting the information and the burden of data collection for patients are barriers that require attention [45, 46]. Clear and standardized explanations of the outcome measure forms distributed to patients during check-in for clinical visits, including their purpose and significance in RA care, can also enhance patient understanding and engagement.

We did not find major differences in the quantitative survey results for patients with repeated surveys (and therefore likely more exposures to the RA PRO dashboard). However, in the qualitative interviews, we observed that clinicians’ perceived attitude toward RA outcome measures impacted patients' acceptance and perceived benefits of the dashboard. For instance, patients who received detailed explanations and feedback from their clinicians on their RA outcome scores and those who saw the dashboard at nearly every clinical visit generally showed more interest in seeing the dashboard in future visits. They also expressed higher awareness and knowledge about the importance of collecting and using these measures in RA care. Conversely, patients who reported that their clinicians did not discuss the RA outcome measures or share the dashboard with them expressed more barriers in understanding these measures and limited interest in seeing the dashboard at future visits. It is likely that limited communication with patients about RA outcome scores not only affects their understanding, but also undermines their willingness to engage in the collection of RA outcome measures. Clinician factors related to collection of RA outcomes, including concerns about the potential of the dashboard to lengthen visits, have been explored in our prior work [25]. These findings suggest that integrating the collection and use of RA outcome measures into routine practice, educating clinicians on the value of RA outcome data, and training them on how to interpret and discuss these measures with patients can help normalize their use and emphasize their importance in patient care [47]. Additionally, providing opportunities for patients to ask questions or seek clarification about RA outcome measure forms can further improve their perception of the relevance and utility of these measures in managing RA more effectively.

A key strength of this mixed-methods study was its use of the EHM as a framework, which allowed us to categorize the patients’ perceived benefits and limitations. Despite its strengths, this study is not without limitations. One limitation of this study is the potential for selection bias. The participants who engaged in the study may not be representative of the broader population of RA patients, particularly due to a majority of the interview participants having high levels of education and health literacy. High digital literacy may have biased the study toward positive perspectives of the dashboard. Since all patients included in the study had seen the dashboard at least once during clinical visits, our findings cannot be generalized to those who have never seen the dashboard before. Although we included a range of usability perspectives, the qualitative sample consisted of patients willing to participate in the interview process, which may not have captured the full range of patient views on outcome measures and usability of the RA PRO dashboard in clinical care. Our qualitative sample was small, and may not have captured all possible perspectives. Additionally, since this study was conducted at a single academic medical center, our results may not generalize to other settings.

Conclusions

In conclusion, this study highlights that most patients found an EHR-integrated RA dashboard that displays their disease outcomes over time useful for improving understanding of their disease, enhancing communication with their clinicians, aiding with shared decision-making, and involving them in care. Future work should test and develop methods to address challenges related to regular collection of data, and to the use and discussion of outcome measures by clinicians.

Data availability

De-identified data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy and ethical restrictions.

Abbreviations

RA:

Rheumatoid Arthritis

DA:

Disease Activity

PF:

Physical Function

ACR:

American College of Rheumatology

PRO:

Patient-Reported Outcomes

EMH:

Ecological Model of Health

CDAI:

Clinical Disease Activity Index

PROMIS-PF:

Patient-Reported Outcomes Measurement Information System Physical Function

EHR:

Electronic Heath Record

ICD:

International Classification of Diseases

MA:

Medical Assistant

COREQ:

Consolidated Criteria for reporting Qualitative Research

IT:

Information Technology

Q:

Quote

References

  1. Greene GJ, Beaumont JL, Bacalao EJ, Muftic A, Kaiser K, Eisenstein AR, et al. Integrating PROMIS Measures in a Treat-to-Target Approach to Standardize Patient-Centered Treatment of Rheumatoid Arthritis. J Rheumatol. 2023;50(8):1002–8.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Mathews AL, Coleska A, Burns PB, Chung KC. Evolution of Patient Decision-Making Regarding Medical Treatment of Rheumatoid Arthritis. Arthritis Care Res (Hoboken). 2016;68(3):318–24.

    Article  PubMed  Google Scholar 

  3. Nowell WB, Gavigan K, Kannowski CL, Cai Z, Hunter T, Venkatachalam S, et al. Which patient-reported outcomes do rheumatology patients find important to track digitally? A real-world longitudinal study in ArthritisPower. Arthritis Res Ther. 2021;23(1):53.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Shah B. Rheumatoid Arthritis: American College of Rheumatology; 2023 updated February 2023. Available from: https://rheumatology.org/patients/rheumatoid-arthritis.

  5. Fraenkel L, Bathon JM, England BR, St Clair EW, Arayssi T, Carandang K, et al. 2021 American College of Rheumatology Guideline for the Treatment of Rheumatoid Arthritis. Arthritis Care Res (Hoboken). 2021;73(7):924–39.

    Article  PubMed  Google Scholar 

  6. Orbai AM, Bingham CO 3rd. Patient reported outcomes in rheumatoid arthritis clinical trials. Curr Rheumatol Rep. 2015;17(4):28.

    Article  PubMed  Google Scholar 

  7. Gabriel SE, Michaud K. Epidemiological studies in incidence, prevalence, mortality, and comorbidity of the rheumatic diseases. Arthritis Res Ther. 2009;11(3):229.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Fautrel B, Alten R, Kirkham B, de la Torre I, Durand F, Barry J, et al. Call for action: how to improve use of patient-reported outcomes to guide clinical decision making in rheumatoid arthritis. Rheumatol Int. 2018;38(6):935–47.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.

    Article  PubMed  Google Scholar 

  10. Greenhalgh J. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18(1):115–23.

    Article  PubMed  Google Scholar 

  11. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Yazdany J, Robbins M, Schmajuk G, Desai S, Lacaille D, Neogi T, et al. Development of the American College of Rheumatology’s Rheumatoid Arthritis Electronic Clinical Quality Measures. Arthritis Care Res (Hoboken). 2016;68(11):1579–90.

    Article  PubMed  Google Scholar 

  13. Solomon DH, Losina E, Lu B, Zak A, Corrigan C, Lee SB, et al. Implementation of Treat-to-Target in Rheumatoid Arthritis Through a Learning Collaborative: Results of a Randomized Controlled Trial. Arthritis Rheumatol. 2017;69(7):1374–80.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Curtis JR, Sharma P, Arora T, Bharat A, Barnes I, Morrisey MA, et al. Physicians’ explanations for apparent gaps in the quality of rheumatology care: results from the US Medicare Physician Quality Reporting System. Arthritis Care Res (Hoboken). 2013;65(2):235–43.

    Article  PubMed  Google Scholar 

  15. Campbell R, Ju A, King MT, Rutherford C. Perceived benefits and limitations of using patient-reported outcome measures in clinical practice with individual patients: a systematic review of qualitative studies. Qual Life Res. 2022;31(6):1597–620.

    Article  PubMed  Google Scholar 

  16. Philpot LM, Barnes SA, Brown RM, Austin JA, James CS, Stanford RH, et al. Barriers and Benefits to the Use of Patient-Reported Outcome Measures in Routine Clinical Care: A Qualitative Study. Am J Med Qual. 2018;33(4):359–64.

    Article  PubMed  Google Scholar 

  17. Barnes GD, Sippola E, Ranusch A, Takamine L, Lanham M, Dorsch M, et al. Implementing an electronic health record dashboard for safe anticoagulant management: learning from qualitative interviews with existing and potential users to develop an implementation process. Implement Sci Commun. 2022;3(1):10.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Brettschneider C, Luhmann D, Raspe H. Informative value of Patient Reported Outcomes (PRO) in Health Technology Assessment (HTA). GMS Health Technol Assess. 2011;7:Doc01.

    PubMed  PubMed Central  Google Scholar 

  19. Hassett MJ, Cronin C, Tsou TC, Wedge J, Bian J, Dizon DS, et al. eSyM: An Electronic Health Record-Integrated Patient-Reported Outcomes-Based Cancer Symptom Management Program Used by Six Diverse Health Systems. JCO Clin Cancer Inform. 2022;6:e2100137.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Tsangaris E, Edelen M, Means J, Gregorowitsch M, O'Gorman J, Pattanaik R, Dominici L, Hassett M, Witkowski ML, Schrieber K, Frank E, Carnie M, Pusic A. User-centered design and agile development of a novel mobile health application and clinician dashboard to support the collection and reporting of patient-reported outcomes for breast cancer care. BMJ Surg Interv Health Technol. 2022;4(1):e000119. https://doi.org/10.1136/bmjsit-2021-000119. 

  21. Newman ED, et al. Improving the quality of care of patients with rheumatic disease using patient-centric electronic redesign software. Arthritis Care Res (Hoboken). 2015;67(4):546–53.

    Article  PubMed  Google Scholar 

  22. Eriksson JK, Askling J, Arkema EV. The Swedish Rheumatology Quality Register: optimisation of rheumatic disease assessments using register-enriched data. Clin Exp Rheumatol. 2014;32(5 Suppl 85):S-147–9.

    Google Scholar 

  23. Ibfelt EH, Jensen DV, Hetland ML. The Danish nationwide clinical register for patients with rheumatoid arthritis: DANBIO. Clin Epidemiol. 2016Oct;25(8):737–42. https://doi.org/10.2147/CLEP.S99490.PMID:27822121;PMCID:PMC5094524.

    Article  Google Scholar 

  24. Van Citters AD, Taxter AJ, Mathew SD, Lawson E, Eseddi J, Del Gaizo V, Ahmad J, Bajaj P, Courtnay S, Davila L, Donaldson B, Kimura Y, Lee T, Mecchella JN, Nelson EC, Pompa S, Tabussi D, Johnson LC. Enhancing Care Partnerships Using a Rheumatology Dashboard: Bringing Together What Matters Most to Both Patients and Clinicians. ACR Open Rheumatol. 2023;5(4):190–200. https://doi.org/10.1002/acr2.11533. Epub 2023 Feb 27. PMID: 36852527; PMCID: PMC10100696.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Nasrallah C, Wilson C, Hamblin A, Young C, Jacobsohn L, Nakamura MC, et al. Using the technology acceptance model to assess clinician perceptions and experiences with a rheumatoid arthritis outcomes dashboard: qualitative study. BMC Med Inform Decis Mak. 2024;24(1):140.

    Article  PubMed  PubMed Central  Google Scholar 

  26. McCreary L. Kaiser Permanente’s innovation on the front lines. Harv Bus Rev. 2010;88(9):92, 4–7, 126.

    PubMed  Google Scholar 

  27. Brown T. Design thinking. Harv Bus Rev. 2008;86(6):84–92, 141.

    PubMed  Google Scholar 

  28. Wahl E, Gross A, Chernitskiy V, Trupin L, Gensler L, Chaganti K, et al. Validity and Responsiveness of a 10-Item Patient-Reported Measure of Physical Function in a Rheumatoid Arthritis Clinic Population. Arthritis Care Res (Hoboken). 2017;69(3):338–46.

    Article  PubMed  Google Scholar 

  29. Aletaha D, Nell VP, Stamm T, Uffmann M, Pflugbeil S, Machold K, et al. Acute phase reactants add little to composite disease activity indices for rheumatoid arthritis: validation of a clinical activity score. Arthritis Res Ther. 2005;7(4):R796–806.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Adm Policy Ment Health. 2015;42(5):533–44. https://doi.org/10.1007/s10488-013-0528-y.PMID:24193818;PMCID:PMC4012002.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Stokols D. Translating social ecological theory into guidelines for community health promotion. Am J Health Promot. 1996;10(4):282–98.

    Article  CAS  PubMed  Google Scholar 

  32. Support ST. Citing Stata software, documentation, and FAQs. Release 18 ed. College Station, TX: StataCorp LLC; 2023.

    Google Scholar 

  33. Miles MB, Huberman AM. Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage; 1994.

  34. Dedoose. web application for managing, analyzing, and presenting qualitative and mixed method research data. 7.0.23 ed. Los Angeles, CA: SocioCultural Research Consultants, LLC; 2016.

  35. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

    Article  Google Scholar 

  36. Mohindra NA, Coughlin A, Kircher S, O’Daniel A, Barnard C, Cameron KA, et al. Implementing a Patient-Reported Outcome Dashboard in Oncology Telemedicine Encounters: Clinician and Patient Adoption and Acceptability. JCO Oncol Pract. 2024;20(3):409–18.

    Article  PubMed  Google Scholar 

  37. Primdahl J, Jensen DV, Meincke RH, Jensen KV, Ziegler C, Nielsen SW, et al. Patients’ views on routine collection of patient-reported outcomes in rheumatology outpatient care: a multicenter focus group study. Arthritis Care Res (Hoboken). 2020;72(9):1331–8.

    Article  PubMed  Google Scholar 

  38. Mercieca-Bebber R, King MT, Calvert MJ, Stockler MR, Friedlander M. The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Relat Outcome Meas. 2018;9:353–67.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Trillingsgaard CM, Nielsen BK, Hjollund NK, Lomborg K. Use of patient-reported outcomes in outpatient settings as a means of patient involvement and self-management support - a qualitative study of the patient perspective European Journal for Person Centered Healthcare. 2016;4(2):359–67.

    Google Scholar 

  40. Westergaard K, Skovgaard L, Magyari M, Kristiansen M. Patient perspectives on patient-reported outcomes in multiple sclerosis treatment trajectories: A qualitative study of why, what, and how? Mult Scler Relat Disord. 2022;58:103475.

    Article  PubMed  Google Scholar 

  41. Trautmann F, Hentschel L, Hornemann B, Rentsch A, Baumann M, Ehninger G, et al. Electronic real-time assessment of patient-reported outcomes in routine care-first findings and experiences from the implementation in a comprehensive cancer center. Support Care Cancer. 2016;24(7):3047–56.

    PubMed  Google Scholar 

  42. Duman-Lubberding S, van Uden-Kraan CF, Jansen F, Witte BI, Eerenstein SEJ, van Weert S, et al. Durable usage of patient-reported outcome measures in clinical practice to monitor health-related quality of life in head and neck cancer patients. Support Care Cancer. 2017;25(12):3775–83.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Scholle SH, Morton S, Homco J, Rodriguez K, Anderson D, Hahn E, et al. Implementation of the PROMIS-29 in Routine Care for People With Diabetes: Challenges and Opportunities. J Ambul Care Manage. 2018;41(4):274–87.

    Article  PubMed  Google Scholar 

  44. Thestrup Hansen S, Kjerholt M, Friis Christensen S, Brodersen J, Holge-Hazelton B. User experiences on implementation of patient reported outcome measures (PROMs)in a Haematological outpatient clinic. J Patient Rep Outcomes. 2020;4(1):87.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Aiyegbusi OL, Cruz Rivera S, Roydhouse J, Kamudoni P, Alder Y, Anderson N, et al. Recommendations to address respondent burden associated with patient-reported outcome assessment. Nat Med. 2024;30(3):650–9.

    Article  CAS  PubMed  Google Scholar 

  46. Atkinson TM, Schwartz CE, Goldstein L, Garcia I, Storfer DF, Li Y, et al. Perceptions of response burden associated with completion of patient-reported outcome assessments in oncology. Value Health. 2019;22(2):225–30.

    Article  PubMed  Google Scholar 

  47. Riemsma RP, Kirwan JR, Taal E, Rasker HJJ. Patient education for adults with rheumatoid arthritis. Cochrane Database Syst Rev. 2003(2):CD003688. https://doi.org/10.1002/14651858.CD003688. Accessed 1 Oct 2024.

Download references

Acknowledgements

Not Applicable

Funding

This project was supported by grant number 1R01HS025405 from the Agency for Healthcare Research and Quality (AHRQ). The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ.

Author information

Author notes

  1. Catherine Nasrallah and Cherish Wilson co-first authors and contributed equally to this work.

Authors and Affiliations

  1. Division of Rheumatology, Department of Medicine, University of California, 4150 Clement Street, Mailstop 111R, San Francisco, CA, 94121, USA

    Catherine Nasrallah, Cherish Wilson, Alicia Hamblin, Christine Hariz, Cammie Young, Jing Li, Jinoos Yazdany & Gabriela Schmajuk

  2. San Francisco Veterans Affairs Medical Center, San Francisco, CA, USA

    Gabriela Schmajuk

  3. Center for Vulnerable Populations and Zuckerberg San Francisco General Hospital, San Francisco, CA, USA

    Jinoos Yazdany

Authors
  1. Catherine Nasrallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cherish Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alicia Hamblin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christine Hariz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cammie Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jinoos Yazdany
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Gabriela Schmajuk
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

GS and JY conceived and designed the study, obtained research funding and supervised its conduct and takes responsibility for the paper. CW, AH, CH and CY collected survey data at the clinic and coded patient interviews. JL provided advice on quantitative study design and JL and CW analyzed the quantitative data. CN provided advice on qualitative study design, conducted interviews and analyzed the qualitative data. GS, CN, CW, CH drafted the manuscript. All authors read and approved the final manuscript. CN and CW contributed equally to this manuscript.

Corresponding author

Correspondence to Gabriela Schmajuk.

Ethics declarations

Ethics approval and consent to participate

The protocol and all documents used in this study were reviewed and approved by the University of California San Francisco (UCSF) Institutional Review Board. A verbal informed consent was obtained from all study participants prior to participation in the surveys and interviews. The informed consent form was reviewed and approved by the University of California San Francisco (UCSF) Institutional Review Board.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nasrallah, C., Wilson, C., Hamblin, A. et al. Patient perceptions of an electronic-health-record-based rheumatoid arthritis outcomes dashboard: a mixed-methods study. BMC Med Inform Decis Mak 24, 302 (2024). https://doi.org/10.1186/s12911-024-02696-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-024-02696-9

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us