Algorithm-based decision support for symptom self-management among adults with Cancer: results of usability testing

Abstract

Background

It is essential that cancer patients understand anticipated symptoms, how to self-manage these symptoms, and when to call their clinicians. However, patients are often ill-prepared to manage symptoms at home. Clinical decision support (CDS) is a potentially innovative way to provide information to patients where and when they need it. The purpose of this project was to design and evaluate a simulated model of an algorithm-based CDS program for self-management of cancer symptoms.

Methods

This study consisted of three phases; development of computable algorithms for self-management of cancer symptoms using a modified ADAPTE process, evaluation of a simulated model of the CDS program, and identification of design objectives and lessons learned from the evaluation of patient-centered CDS. In phase 1, algorithms for pain, constipation and nausea/vomiting were developed by an expert panel. In phase 2, we conducted usability testing of a simulated symptom assessment and management intervention for self-care (SAMI-Self-Care) CDS program involving focus groups, interviews and surveys with cancer patients, their caregivers and clinicians. The Acceptability E-scale measured acceptability of the program. In phase 3, we developed design objectives and identified barriers to uptake of patient-centered CDS based on the data gathered from stakeholders.

Results

In phase 1, algorithms were reviewed and approved through a consensus meeting and majority vote. In phase 2, 24 patients & caregivers and 13 clinicians participated in the formative evaluation. Iterative changes were made in a simulated SAMI-Self-Care CDS program. Acceptability scores were high among patients, caregivers and clinicians. In phase 3, we formulated CDS design objectives, which included: 1) ensure patient safety, 2) communicate clinical concepts effectively, 3) promote communication with clinicians, 4) support patient activation, and 5) facilitate navigation and use. We identified patient barriers and clinician concerns to using CDS for symptom self-management, which were consistent with the chronic care model, a theoretical framework used to enhance patient-clinician communication and patient self-management.

Conclusion

Patient safety and tool navigation were critical features of CDS for patient self-management. Insights gleaned from this study may be used to inform the development of CDS resources for symptom self-management in patients with other chronic conditions.

Background

Patient-centered healthcare is one of six aims to improve the United States healthcare system [1]. One facet of patient-centered care is engaging patients and encouraging self-management, especially in the context of chronic illness. Self-management programs that provide coaching and education, and are supported by timely information, facilitate patient engagement [2, 3].

Given the frequency of symptoms from cancer and its treatment, it is essential that patients (and their caregivers) understand how to self-manage symptoms and when to call their clinicians for advice. McCorkle and colleagues [3,4,5] noted that most patients with cancer try to self-manage their care and that developing system-level interventions to support self-management is essential for quality cancer care. Evidence-based strategies to assist patients with self-management include: education, telephone consultations, Internet tools for tracking disease-specific parameters, and coaching [3, 6,7,8,9,10,11,12,13]. Such interventions targeted information management, medications, psychological consequences of illness, lifestyle, social support, communication, accessing services, and setting goals [14,15,16]. Self-management interventions decreased the severity of pain [17], fatigue [18], and depression [19]. Web-based interventions, featuring self-monitoring, education and coaching of patients combined with summaries of patient-reported data delivered to clinicians, led to improvement in symptom distress and quality of life [12, 16, 20]. Evidence suggests that reporting symptoms to clinicians alone may not be sufficient. A project in which patients receiving chemotherapy reported on symptoms using an automated phone system failed to impact care. Even though symptoms of moderate to severe intensity were reported, clinicians did not provide patients with management guidance [10].

Considering the variability of clinician response to symptom reports, an optimal approach to enhance self-management includes providing patient-specific, real-time, actionable information. Supplying Clinical Decision Support (CDS) directly to patients may enhance their ability to self-manage symptoms. CDS “provides individuals with person-specific information, intelligently filtered or presented at appropriate times, to enhance health” [21]. Four studies have described rule-based CDS tools for cancer patients. [12, 20, 22, 23] In two studies, CDS was used to identify the presence of a symptom using an algorithm with a single decision node that generated general recommendations for symptom management [12, 22]. In studies by Berry and colleagues [24] and Weaver, [23] the focus was self-care support and identifying symptoms to report to clinicians. Berry and colleagues [12] found that Web-based coaching of patients to report symptom experiences verbally resulted in more frequent reports during clinic visits, but the investigators did not document how often a participant called the clinic to ask for help. Ruland et al. [20] developed a Web-based resource that assisted patients with assessing symptoms, finding information, communicating with clinic personnel and provided self-management advice. This tool was evaluated and showed a slight decrease in symptom distress for patients [20] but the CDS component was not evaluated independently for its effect. A follow-up study determined that communication with nurses was the most valued component [25].

The present study extends the literature by reporting on the design and formative evaluation of an algorithm-based simulated model of a patient-centered CDS program that facilitates cancer symptom self-management, provides advice on when patients should contact their clinicians, and includes coaching information about what to tell them. The goals of the project were to develop computable algorithms for pain, constipation and nausea/vomiting in phase 1, conduct iterative usability testing of the simulated CDS program called the Symptom Assessment and Management Intervention for Self-Care (SAMI-Self-Care) with patients, their caregivers and clinicians in phase 2, and develop design objectives and identify barriers to uptake of patient-centered CDS based on the data gathered from stakeholders, which included members of an expert panel, patients, caregivers, and their clinicians in phase 3.

Methods

We employed a convergent, parallel, mixed methods design, [26] in which we collected qualitative and quantitative data in parallel, analyzed it separately, and then merged all data during interpretation, to evaluate the acceptability of a simulated algorithm-based CDS program for cancer symptom self-management. This study was conducted at Dana-Farber Cancer Institute (DFCI) and approved by the Institutional Review Board (IRB), Protocol number-12-300.

Phase 1: Algorithm-based CDS intervention development

In order to develop the patient-centered CDS algorithms, we used a modified ADAPTE process [27,28,29] consisting of five steps; 1) identify expert panels to develop and evaluate the computable algorithms, 2) develop groups to work on each symptom and synthesize the literature, 3) convene groups to translate evidence-based information into computable algorithms for each symptom, 4) conduct peer review on the content of the algorithms before convening a multidisciplinary consensus meeting, and 5) hold a multidisciplinary panel meeting to review, modify and approve the algorithms.

The study team identified areas of expertise that were critical for developing and evaluating an algorithm-based CDS tool for symptom self-management. Individuals with the desired expertise were identified through existing professional relationships or through recommendations of colleagues. The identified individuals were invited to join the panel by the principal investigator (MEC) and included: stakeholders with expertise in clinical care (oncologists, palliative care experts, psychiatrists, and oncology nurses), information systems (experts in CDS, patient data collection, health communication, and graphic design), care delivery process (experts in workflow, quality improvement, and health equity), as well as patients and caregivers.

Our expert panel drew from evidence-based resources and worked with CDS experts to develop computable algorithms that would enable self-management of pain, constipation, and nausea/vomiting. Overall, the expert panel met four times for 4 h each time to review the algorithms developed by the research team. The symptoms chosen were the most common reasons for urgent care among cancer patients and identified as important targets for symptom management by patients and their caregivers in a previous study that explored patient preferences for CDS to enhance clinical care [30]. Our process of developing algorithms was based on our previous work that adapted evidence-based guidelines for clinicians [31]. Similar to our previous work, multi-disciplinary groups were formed for each symptom and serial meetings were held to develop the initial algorithms. Once the algorithms were completed, a group meeting with the members of the research team and the expert panel was held to review, modify and approve the algorithms. For each symptom, characteristics were identified to direct self-management. Characteristics of pain included descriptors of neuropathic or somatic pain, and a measure of intensity (moderate, severe). Features of nausea/vomiting included chemotherapy-induced, radiation-induced, chemoradiation-induced, acid reflux, vertigo, recent narcotic increase, anticipation of chemotherapy, and constipation. Characteristics of constipation included frequency and consistency of stool.

Phase 2: Usability testing

In order to refine the algorithm content and the simulated SAMI-Self-Care tool display and function iteratively, we used a collaborative-participatory approach. This approach engaged stakeholders in a shared co-designing process and helped ensure that the product will meet real-life needs and be adopted [32, 33]. The team formulated a scenario for a symptom management dilemma that would allow patient users to traverse the self-care management algorithms so that self-management recommendations could be generated. Cognitive testing was performed for each algorithm by reviewing the content displayed on an iPad with patients and their caregivers to ensure that these questions were clear, understandable, and relevant, and that the response options were appropriate prior to initiating the iterative usability testing. Cognitive testing of items is recommended as a standard approach to the development process as it allows one to ascertain whether participants understand items on a questionnaire and allows for iterative changes before finalizing questions [34].

This patient-vetted algorithm content was given to the Dana-Farber Harvard Comprehensive Cancer Center Health Communication Core. The Core developed a mockup of the SAMI-Self-Care using interactive PDF files to simulate the function of the CDS tool based on expert panel input and clinical informatics literature [35, 36]. The PDF files imitated the functionality of a Website and allowed participants to experience the “look and feel” of a potential operational CDS system on an iPad. This approach provided a practical and economical approach to usability testing, especially during formative stages of development when iterative refinement is necessary. The Health Communication Core also designed reports containing self-management recommendations to elicit feedback from participants regarding report content and appearance. A series of focus groups or interviews were conducted to elicit feedback about the CDS program. Using a multi-methods approach (i.e. focus groups and interviews) is appropriate when the intent is to gather information that is focused on specific goals and questions, which in this case was participant response about the usability of an algorithm-based CDS program [37]. We used a combination of methods to provide flexibility to participants to be able to participate in the usability sessions. We used the same facilitator to conduct the focus groups and interviews and used an interview guide to elicit information about the usability and acceptability of the SAMI-Self-Care CDS program.

Based on feedback from the expert panel, patients, caregivers, and clinicians, we iteratively created three successive versions of the SAMI-Self-Care program. Figures 1, 2, 3, 4 show the iterative development of the self-management report that was generated for participants based on responses to the questions and algorithm-based CDS. Feedback from the usability testing was used to inform the changes that were made to the CDS program and the self-management report (see results).

Fig. 1

Initial SAMI-Self Care report

Fig. 2

First revision for SAMI-Self Care report

Fig. 3

Second revision for SAMI-Self Care report

Fig. 4

Third revision for SAMI-Self Care report

Participant recruitment

Patients were identified through medical records at DFCI under a waiver of consent authorization, and names were submitted to their clinicians for approval to contact patients, as required by the IRB. Eligible patients were age ≥ 18, English speaking, and had received cancer treatment within the past 6-months. All approved patients were sent an initial letter describing the study and inviting them to participate. Patients received two follow-up calls if they didn’t respond to the letter. Patients identified caregivers to invite to participate in the study. Eligible caregivers were age ≥ 18 years and provided care to a patient who received cancer treatment within the past 6-months. Deliberate efforts were made to sample across a range of diagnoses, educational backgrounds, ages, genders, and races. All participants signed consent forms at the time of participation. Each participant received $50 for completing the study.

Clinicians in ambulatory oncology included physicians, nurse practitioners, physician assistants, and registered nurses. Clinicians were identified through clinic rosters. An invitation to participate in the study was emailed to eligible clinicians. Two follow-up emails were sent if no response was received. Clinicians were recruited for either focus groups or interviews depending on their schedules. Clinicians received an invitation letter describing the study but signed consent was waived by the IRB. Each clinician received $100 for completing the study.

Focus group and interview processes

Focus groups and interviews were held separately for clinicians and patients/caregivers. Focus groups included up to five participants; interviews included one or two participants and were held in a convenient conference room at DFCI. Meetings for clinicians lasted about 30 min whereas those for patients and caregivers lasted 45–60 min. One symptom was addressed per session. Using a script, the facilitator queried patients, caregivers and clinicians about the algorithm content of the SAMI-SC programs for constipation, pain, and nausea/vomiting, and asked about the “look and feel” of the materials for pain and nausea/vomiting. The topics for patients and caregivers focused on the: 1) visual appeal, 2) format and navigation, 3) understandability of written content and terminology, and 4) wording of self-management suggestions. The topics for the clinicians focused on 1) a review of the patient self-management algorithms, 2) review of the simulated SAMI-Self-Care CDS program, and 3) feedback about ways to improve the algorithms and/or simulated model. Patients/caregivers and clinicians reported demographic data using a standardized survey.

The number of sessions per symptom was determined by evidence of content saturation and absence of new common responses [38]. With patients/caregivers, we conducted six sessions for pain, seven sessions for nausea/vomiting, and three sessions for constipation. With clinicians, we conducted two sessions for pain, three sessions for nausea/vomiting, and one session for constipation. All discussions were audio-recorded and transcribed.

Survey instruments

The Acceptability E-scale measured the acceptability of the CDS program [39]. This scale has been used in adult patients with various types of cancers from medical oncology, radiation oncology and stem cell transplant to measure acceptability of Web-based systems, including computerized symptom and quality of life assessments, patient educational materials and patient-centered decision support. Psychometric testing was conducted in 627 adults with cancer [40, 41]. Cronbach alpha reliability coefficient was 0.75 and factor analysis revealed that the scale was unidimensional [39]. Five-point Likert-type items (1–5; larger number equals higher acceptability and a score of 3 indicates a neutral response) were presented to elicit feedback from both patient/caregiver and clinician participants regarding: 1) understandability of language, 2) helpfulness of suggestions provided for symptom self-management, 3) usefulness of reports, and 4) overall satisfaction with the program. The a priori target for acceptability of SAMI-Self-Care was a mean score of ≥4 on each item since this score indicated a positive acceptability response. In addition to the Acceptability E-scale, participants were asked what other types of information would have been helpful for symptom self-management. The scale took less than 2 min to complete and was written at a 5th grade reading level.

Analysis

Patient/caregiver and clinician demographic data were summarized as descriptive statistics. Acceptability E-scale results were summarized as item means and standard deviations.

Audio-recorded session content was transcribed and reviewed by three study team members (MEC, DFL, MMN), coded inductively, and grouped into common responses to identify revisions that needed to be made in the CDS tool. Common responses were then combined and reconciled though discussion. A list of suggested revisions to improve SAMI-Self-Care algorithms and the simulated model was created and ranked based on audio files and notes. Critical revisions were implemented immediately, while other less critical revisions were monitored for repetition. This process was repeated, after each iteration of CDS tool development, until no new responses or suggestions for changes emerged from the usability testing. Phase 2 analyses focused on the revisions that needed to be make to the CDS tool.

Phase 3: Design objectives and barriers to uptake of patient-centered CDS

Once all the data were collected and the usability testing completed, three investigators (MEC, DFL, MMN) reviewed all of the qualitative data in order to identify design objectives and barriers to uptake of the patient-centered CDS program. Eight sessions were needed to organize the data collected from the expert panel, patients, caregivers and clinicians and discuss the themes that were identified through inductive content analysis [42]. These data were then shared and discussed with other members of the research team to generate a final list of design objectives, patient barriers, and clinician concerns to use of the CDS program.

Analyses

Audio-recorded session content was transcribed and reviewed by three study team members (MEC, DFL, MMN), coded, and grouped into themes. Themes were then combined and reconciled though discussion. Inductive analyses of the coded qualitative data for phase 3 focused on interpreting patient barriers to using the CDS program, defining clinician concerns about use of CDS by patients/caregivers, and identifying strategies to overcome the areas of concern. It was noted during the analysis that the themes that emerged from the qualitative data were consistent with the Chronic Care Model [3, 43]. Thus, five design objectives detailed below (in Results) were developed to address these concerns. As a result, content analysis was then used to regroup the themes to fit within the Chronic Care Model.

Results

Phase 1: Rule-based CDS intervention development

Nine potential participants were approached for membership in the expert panel, and all accepted the invitation. Altogether, 16 members of the research team and expert panel contributed to algorithm development. Work sessions were conducted in person and by Web conferencing so that the flow charts could be displayed more easily for discussion. Branching logic was used to develop the algorithms and these were displayed as flow charts that were shared among all team members. Research staff provided support to access current guidelines that were available for the targeted symptoms and to assist with literature reviews. Each work group had someone assigned who was expert in pharmacological, non-pharmacological and self-management approaches so that a comprehensive approach to enable self-management would be embedded into the algorithms.

Once the development of the algorithms and usability testing was completed, a multidisciplinary consensus meeting was held with the expert panel, research team and co-investigators to review, modify and approve the final versions. In order to iteratively improve the program, the expert panel focused on several issues. The primary issue was patient safety. Safety was addressed by identifying potential serious causes of each symptom and directing patients to seek contact with their clinicians. Accordingly, all three algorithms began with the identification of “red flag symptoms” that caused patients a forced exit of the CDS tool and a directive to call a clinician immediately for guidance. A second issue was how to make the tool accessible to patients across a range of health literacies. Our solution was to provide information that patients could elect to review or skip based on their needs. A third issue was how to provide medication-related advice that was aligned with therapies recommended by a patient’s clinician. To address this issue, we asked the patient whether the medication we were recommending for the problem (e.g. senna for constipation) was approved by his/her clinician. If it was approved, we inquired whether the patient had taken that medication. If not, the patient was advised to take it. If the patient had not been prescribed the recommended therapy, he or she was advised to contact his/her clinician and inquire if this therapy could be appropriate. The resulting algorithms had a large number of decision nodes ranging from 51 to 257 (moderate and severe pain combined). A decision node is a point in the algorithm where the logic branches into two or more directions. A higher number of decision nodes reflect greater algorithmic complexity [44]. Table 1 provides information about the number of decision nodes and red flag questions that were present within each symptom algorithm. A simple majority vote was taken at the end of the discussion surrounding each symptom algorithm to determine whether agreement was reached regarding modifications and approval. Table 1 provides an overview of the number of red flags that were required for each symptoms and the complexity of the algorithms.

Table 1 Number of decision nodes and red flag questions in each symptom algorithm

Phase 2: Usability testing

Participant sample

One-hundred-and-three patients were screened to identify the 24 patients and caregivers for study inclusion (Table 2). Twenty-six patients were deemed ineligible by their clinicians due to the incapacitating nature of their condition and 14 did not meet other eligibility criteria or could not be contacted. Seventeen potential subjects declined participation; 22 indicated interest but were unable to attend scheduled sessions or did not respond to scheduling requests. Participating patients and caregivers were predominately Caucasian, almost evenly split between men and women and most had some college level education.

Table 2 Cancer Patient and Caregiver Demographics (N = 24)

Forty-four clinicians were contacted to identify 13 who participated in focus groups and interviews (Table 3). Of the clinicians who did not participate, 20 did not respond, 2 declined, and 9 indicated interest but were unable to attend scheduled sessions. Clinician participants were predominately female and Caucasian.

Table 3 Oncology Clinician Demographics (N = 13)

Focus groups and interviews

Patients and caregivers provided feedback about the visual appeal of the program, especially related to the color scheme, text density, and font size. Participant feedback sought to make the display inviting and easy to read. Some of the issues raised by clinicians who viewed the program were similar to issues identified by patients and their caregivers. For example, clinicians made suggestions about use of language, decreasing the density of text and providing graphic images to enhance understanding. For the most part, patients, caregivers and clinicians had similar comments to enhance the usability of the program. However, clinicians provided additional feedback about the content of the algorithms, and to ensure patient safety. They suggested adding questions at the beginning of the algorithms that would identify potentially dangerous symptoms that needed immediate attention and prompt patients to exit the program and call their clinicians. We also found that clinicians disagreed about what were best practices for self-management in pain management. Some clinicians felt that patients with pain levels of ≥ 9 should not self-manage and call their clinicians immediately, whereas others felt that it was acceptable to use the suggestions and then call their clinicians for refractory pain. Tables 4 and 5 provide an overview of comments provided by participants to improve the usability of the program.

Table 4 Results of Usability Testing for the SAMI-Self-Care CDS Program: Patient Perspectives

Table 5 Results of Usability Testing for the SAMI-Self-Care CDS Program: Clinician Perspectives

Acceptability surveys

Participants completed surveys regarding the acceptability of SAMI-Self-Care for “nausea/vomiting” and “pain” after each round of system development and viewing the simulated interface. Nine patients and their caregivers completed surveys, four assessing pain and five assessing nausea and vomiting. Two rounds of assessment for usability and acceptability were conducted with patients to reach or surpass the predetermined threshold for acceptability. In general, patients found the system easy to understand and helpful for self-management.

Thirteen clinicians completed surveys, nine assessed pain and three assessed nausea and vomiting, and one assessed constipation. Two rounds of assessment for usability and acceptability were conducted with clinicians. The simulated program for nausea/vomiting and constipation reached and surpassed the threshold for acceptability. However, the helpfulness of suggestions for the pain algorithm was scored lower than our target threshold of 4.0. For the most part, clinicians also viewed the system for pain as usable and helpful. Subsequent discussions revealed that some clinicians felt that patients with a pain score ≥ 9 should contact their clinicians and not pursue self-management. The algorithm was subsequently changed to reflect that patients with severe pain, should contact their clinicians, but time limitations prevented the modified algorithm from being reevaluated.

Phase 3: Design Objectives and Barriers to Patient-centered CDS

Design objectives

From patient, caregiver, and stakeholder feedback, we identified five design objectives that were relevant for the development of algorithm-based patient-centered CDS, which included: ensure patient safety, communicate clinical concepts effectively, promote communication with clinicians, support patient activation, and facilitate navigation (see Table 6). This table includes information about the design objective, specific stakeholder barriers related to this objective, solutions that were identified to address the barrier and changes that were made in the user interface. A discussion surrounding each of these design objectives is found in the following section followed by some barriers to patient-centered CDS that emerged from our findings.

Table 6 Design Objectives for Development of Patient-Centered CDS

Ensure patient safety

To ensure that the use of SAMI-Self-Care would not overlook a life-threatening condition, we identified a set of screening questions for each symptom that sought to direct patients with potentially dangerous conditions to stop using the algorithm and seek contact with their clinicians. To ensure that the advice provided through SAMI-Self-Care was sound, we derived algorithm logic and recommendations from evidence-based clinical guidelines. When appropriate, we also inquired if a recommended therapy had already been suggested by the patient’s clinician. If the therapy had not been suggested, we encouraged the patients to check with their clinician to request approval before initiating a new medication.

Communicate clinical concepts effectively

In order to communicate clinical concepts effectively, we conducted cognitive testing of content with patients to ensure an adequate level of understanding, and we added graphics to support concepts in the text when appropriate. We also designated a section of the display for explanations or lists so that patients who needed more information could easily access it. Formative testing of questions through the cognitive interviewing process helped us to recognize that we needed to make questions as explicit and detailed as possible so patients were not left to speculate about how to respond. Finally, we used established interface development practices for screen appearance, layout, and text fonts to enhance readability [45].

Promote communication with clinicians

To encourage communication with patients’ clinicians, we included explicit suggestions regarding when to contact their clinicians and a script for what to say to them. We also advised patients to notify their clinicians within a specified time frame about interventions they may have followed from SAMI-Self-Care.

Support patient activation

We sought to engage patients by providing personalized and actionable instructions. As appropriate, we asked patients about resource availability and their willingness to follow interventions prior to making recommendations. Through the educational display section, we provided content to help patients understand why certain questions were being asked and why specific recommendations were provided. Helpful features included suggestions for pharmacologic and non-pharmacologic therapies.

Facilitate navigation and use

To expedite SAMI-Self-Care use we identified methods to optimize traversal of the algorithms and minimize the burden of data entry (see Table 6). We included a progress bar for patients to know their position in the process, and we added capabilities to change responses or to pause a session. We also added display content to enable a patient to recognize the context of every interaction through tabs illustrating the symptom being addressed and text clarifying the context of each question. Finally, we ensured that all pathways could be traversed, were unique and not redundant.

Barriers that need to be addressed to promote patient-centered CDS

Barriers to promote patient-centered CDS regarding symptom self-management that emerged appeared to be consistent with the Chronic Care Model so the themes that emerged were reorganized around the components of this model which included: patient safety, cultural competency, care coordination, resource availability, and acceptance of technology.

Barriers pertaining to cultural competency were the most common and included challenges related to unfamiliar terminology, lack of reference to a clinical framework, and unusual health beliefs. Although many of the concerns identified by patients were shared by clinicians, a few items were unique to patients and their caregivers, including expressing uncertainty about some self-management suggestions and whether the actions would be acceptable to their clinicians, expressing negative health beliefs associated with the use of narcotic medications, and not having the suggested medications available at home. There also were concerns about communication with their clinicians, availability of technology in the community, and use of the technology.

Discussion

We developed a simulated algorithm-based CDS program to support symptom self-management among adults with cancer and their caregivers. Our CDS program is more complex than other previously developed programs and provides patents with specific information, tailored to their situation regarding when to call their clinician, what to tell them, and how to self-manage their symptoms. In order to accomplish this task, we have many more decisional nodes than previous patient-centered CDS systems [12, 22]. Acceptability testing results among patients were favorable, suggesting that patients found the SAMI-Self Care prototype a satisfactory approach to for symptom self-management. Providing CDS directly to patients may be a valuable tool. Future studies should explore the best modalities for providing access to these tools including the Web, tablet computers, or mobile phones [46]. We anticipate that the SAMI-Self-Care tool could be implemented on a Web-based platform that could be accessible across computers, tablets and smartphones. A link to this content could be inserted in the homepage of a patient’s oncology treatment group or sent out directly to patients through electronic mail.

We noted that the acceptability results from clinicians for pain were less favorable than for nausea/vomiting and constipation, and were below our desired threshold of 4.0 for helpfulness of the report and suggestions. This result required obtaining a larger sample size to gather responses about usability testing as compared to other symptoms. In spite of our efforts to adhere to pain guidelines, [47] some clinicians were reluctant to support self-management because they wanted to remain aware of their patients’ symptoms and to be directly involved in management. This finding is interesting as implementation of pain guidelines has been a challenge. Evidence suggests that pain management practices have not changed over the last twenty years and many patients with cancer continue to experience severe uncontrolled pain [48, 49]. Further study is needed to understand clinician attitudes toward pain management and barriers to encouraging patient self-care management. One concern that clinicians expressed is that they wanted to be aware of patients who had severe levels of pain. The addition of a system that alerts clinicians when patient symptoms pass a predetermined level of severity would address this concern. Cleeland and colleagues [13] found that automated symptom severity alerts after surgery for lung cancer reduced symptom distress. In a recent study that examined clinician preferences for CDS, the use of an alerting system for increased symptom severity was identified as a desirable component [50].

Our design objectives were specifically developed in this study to support development of effective patient-focused CDS. To date, CDS design objectives have focused on use by clinicians. Thus, even though our CDS system was designed for patients, we sought to include features known to be associated with effective CDS systems for clinicians [35, 36, 51]. The system features identified in our study that were similar to clinician CDS included: graphics to enhance understanding of content; explicit, actionable recommendations provided at the point of decision-making; presentation of advice that cultivated trust by providing an explanation of medical logic if needed [36]. In addition, similar to Bates and colleagues, [35] our Design Objectives focused on streamlining the user interaction by decreasing the number of questions needed to inform the algorithm, targeting anticipated needs with advice in real-time, providing supplemental information as needed but without interruption, and relying on a user-centered iterative design [43]. Ozkaynak and colleagues [52] noted that an important difference between patient and clinician design for CDS is integrating technology into the user’s work flow. Designing CDS around clinician work flow is much easier than for patient work flow as clinicians function within a health system, whereas work flow for patients spans multiple health care settings and includes home, routines of daily living and communication with family, clinicians and the health care system [52,53,54,55]. The CDS program that was evaluated in this study attempts to bridge the gap between care delivered in health care settings and self-care in the home. There is a need for applications that improve outcomes across settings and patient populations [56]. Future studies are needed to assess the impact of this CDS program on patient outcomes.

Through this process, we identified patient barriers to use of CDS and clinicians’ concerns about patients using CDS for self-management. From these barriers and concerns, we derived Design Objectives for CDS for symptom self-management, which included ensuring patient safety, communicating clinical concepts effectively, promoting communication with clinicians, supporting patient activation and facilitating navigation and use. These objectives may be useful to inform the design of CDS systems for symptom self-management of other conditions.

The barriers we identified for CDS for cancer symptom self-management had similarities and differences to barriers associated with self-management in patients with non-malignant conditions [57, 58]. Similar barriers included lack of knowledge, poor communication between patients and clinicians, and logistical issues in obtaining care. However, in contrast to the patients who did not have cancer, the cancer patients did not report barriers related to physical limitations, financial constraints, a need for social and emotional support, or challenges adhering to treatment [57, 58]. Other differences in concerns that we noted between patients with cancer and those with non-malignant conditions were that cancer patients and their caregivers expressed concerns related to ensuring patient safety and negative health beliefs about use of narcotic pain medications. Shumacher and colleagues [59, 60] examined pain management processes among cancer patients and their caregivers and found similar concerns related to ensuring patient safety, especially in the context of narcotic pain medications. Patients and their caregivers had little interaction with clinicians in the home setting and had to master complex tasks related to taking their medications and reported that they felt that they had to be the final safety check.

Limitations

The sample for this study was drawn from one institution using purposive sampling so it is not representative of all cancer patients. All of the patients and their caregivers in this study indicated that they used the Internet for seeking health information at least sometimes. This rate is higher than the 72% of Internet users found in the 2014 Pew Research survey [61]. Our rates for Internet usage may be higher as the majority of our sample had greater than a high school education. Factors associated with lower access of the Internet are older age, lower education and lower income [62]. Another limitation is that the majority of the sample was Caucasian and had higher levels of education. Further testing of this approach in a more diverse group of patients with cancer is warranted. Although the use of a simulated model provided a practical and economical approach to iterative development of the CDS tool, this approach did not allow for real time navigation through the algorithms for patients. This would be the next step in the development process.

Conclusions

The boundaries of health care are expanding and patients and their caregivers often have to self-manage complex cancer care at home. Thus, patient-centered decision support that meets this need is important and timely. Our system provides tailored information that informs patients when to call their clinicians, provides a script about what to tell tem about their symptom and specific suggestions about how the self manage their symptoms at home. Patients and their caregivers rated SAMI-Self-Care as highly acceptable and found the recommendations helpful. The patient-centered CDS design objectives we derived for cancer symptom self-management may be applicable for the self-management of other conditions.