Development and pilot testing of a decision aid for navigating breast cancer survivorship care

Abstract

Background

The predominant oncologist-led model in many countries is unsustainable to meet the needs of a growing cohort of breast cancer survivors (BCS). Despite available alternative models, adoption rates have been poor. To help BCS navigate survivorship care, we aimed to systematically develop a decision aid (DA) to guide their choice of follow-up care model and evaluate its acceptability and usability among BCS and health care providers (HCPs).

Methods

We recruited BCS aged ≥ 21 years who have completed primary treatment and understand English. BCS receiving palliative care or with cognitive impairment were excluded. HCPs who routinely discussed post-treatment care with BCS were purposively sampled based on disciplines. Each participant reviewed the DA during a semi-structured interview using the ‘think aloud’ approach and completed an acceptability questionnaire. Descriptive statistics and directed content analysis were used.

Results

We conducted three rounds of alpha testing with 15 BCS and 8 HCPs. All BCS found the final DA prototype easy to navigate with sufficient interactivity. The information imbalance favouring the shared care option perceived by 60% of BCS in early rounds was rectified. The length of DA was optimized to be ‘just right’. Key revisions made included (1) presenting care options side-by-side to improve perceived information balance, (2) creating dedicated sections explaining HCPs’ care roles to address gaps in health system contextual knowledge, and (3) employing a multicriteria decision analysis method for preference clarification exercise to reflect the user’s openness towards shared care. Most BCS (73%) found the DA useful for decision-making, and 93% were willing to discuss the DA with their HCPs. Most HCPs (88%) agreed that the DA was a reliable tool and would be easily integrated into routine care.

Conclusions

Our experience highlighted the need to provide contextual information on the health care system for decisions related to care delivery. Developers should address potential variability within the care model and clarify inherent biases, such as low confidence levels in primary care. Future work could expand on the developed DA’s informational structure to apply to other care models and leverage artificial intelligence to optimize information delivery.

Background

Globally, a rising incidence of breast cancer diagnoses with low mortality rates resulted in an increasing pool of breast cancer survivors (BCS) with long-term care needs [1]. Most countries adopted the oncologist-led model where breast cancer-related follow-up care is concentrated in specialist settings [2]. However, the extensive utilization of oncologist services in the post-treatment survivorship phase significantly strains the current capacity of cancer centres, rendering the oncologist-led model unsustainable in addressing survivors’ unmet needs [3]. In response, countries began trialling alternative care models with greater involvement of primary care health care providers (HCPs) [3,4,5,6,7]. Despite evidence suggesting comparable effectiveness to the oncologist-led model, adoption rates of alternative models were poor, prompting calls for strategies to guide model selection [3, 5]. Furthermore, it is increasingly important to empower BCS in this decision-making process to ensure that the chosen model aligns with each survivor’s preferences, maximizing the relevance and value of survivorship care [3, 8,9,10].

To help BCS navigate through possible breast cancer survivorship care models, decision aids (DAs) could be utilized [11]. A DA is a tool designed to provide neutral, balanced, and evidence-based information on the possible care options. Additionally, it elicits users’ preferences to exemplify the trade-offs between options to make an informed decision [12]. Complementing DA development, artificial intelligence (AI) tools are increasingly explored and embedded within DAs to enhance personalized information delivery and communication [13, 14]. While available evidence demonstrated favourable outcomes of DA usage in increasing knowledge, reducing decisional conflicts, and enhancing satisfaction for cancer-related decisions [15,16,17], there was a disproportionate focus on screening and treatment decisions. A recent systematic review focusing on health services or care modality decisions after primary cancer treatment identified a Dutch study that examined DA usage for breast cancer follow-up care intensity in hospitals [18]. The study showed promising results in improving shared decision-making and reducing cost [19]. Recognizing that cancer survivorship care follow-up options are context- and cultural-specific, direct extrapolation of this DA across countries would be suboptimal.

Singapore is a high-resource country in Southeast Asia where the oncologist-led model is the predominant follow-up modality for BCS [20]. Survivorship care development is at its infancy stage in Singapore, where primary care delivery was rated less favorably than in Western countries [21]. Thus, a complete discharge to a primary care-led model is likely unacceptable with poor adoption. A shared care model involving oncologists, family physicians, and community pharmacists is then piloted in Singapore with assurance of oncologists’ continued involvement. Complementing evaluation efforts in an ongoing trial (NCT04660188), this study aimed to develop a DA to guide BCS’ choice between oncologist-led and shared care models for cancer follow-up and evaluate its acceptability and usability among prospective users through extensive alpha testing. This development exercise would exemplify efforts to devise strategies for alternative model adoption, relieving the strain on acute care resources.

Methods

Study design and setting

As part of the DA development process [22], we utilized a mixed-methods design for the alpha testing phase conducted from October 2019 to April 2022. Adopting a user-centred design approach, the preliminary prototype developed was subjected to an iterative process of testing and revising [23, 24] at the National Cancer Centre Singapore, the largest public ambulatory cancer centre in Singapore. This study was approved by the SingHealth Institutional Review Board (CIRB 2019/2596).

Theoretical framework

We systematically developed the DA based on the recommendations by Coulter et al., adhering to the International Patient Decision Aid Standards framework (Fig. 1) [22, 25]. A steering group comprising medical oncologists, an oncology pharmacist, and health service researchers without any conflicts of interest supervised the development process. This group ensured that changes made to the DA during testing were appropriate and of clinical relevance.

Fig. 1

Systematic development process of decision aid for breast cancer survivorship follow-up care, adapted from Coulter et al.

The prototype design comprised three components (Fig. 1). First, we performed extensive literature review to develop the DA content. We we reviewed decisional needs from BCS’ and primary care HCPs’ perspectives based on two local qualitative studies [26, 27]. Catering to reported informational needs, we designed the DA to address the following content areas: principles of cancer survivorship, roles of participating HCPs, clinical evidence of care models, care coordination strategies, and cost considerations. Published clinical guidelines provided information on the core elements of survivorship care [28,29,30]. Official institutional webpages provided information on charging and patient resources [31]. The steering group reviewed the description of care models retrieved from the literature [4, 6, 7]. Second, we included a preference clarification exercise to consider BCS’ confidence in primary care HCPs to care for their cancer, an area of concern highlighted by BCS [26]. This exercise prompted users to rate the importance of a list of attributes for their care [32]. Lasty, the preliminary DA prototype (Additional file 1) was developed as an English tool in a digital format using in-built functionalities within the Microsoft PowerPoint and infographics. We then used this preliminary DA prototype in the alpha testing phase to optimize users’ experience and maximize the DA’s usability and acceptability.

Eligibility criteria and recruitment

Adults aged ≥ 21 years old, formally diagnosed with breast cancer, completed primary treatment (excluding endocrine therapy), and able to read and speak English were eligible. BCS receiving palliative care and/or with cognitive impairment that negatively affects their ability to navigate the DA and articulate their responses were excluded. With the aim of recruiting prospective DA users, eligible participants were identified by medical and surgical oncologists from clinics attended by predominantly BCS. HCPs aged ≥ 21 years old who were involved in discussing post-treatment care with BCS were eligible and contacted by the study team. We excluded HCPs who were steering group members and purposively sampled them to achieve diversity in disciplines. Informed consent was obtained from each participant.

Data collection

Adopting recommendations for usability testing to be held in small groups each round, we targeted five BCS and two to three HCPs per round [33]. Upon recruitment, BCS participants completed a demographic and clinical questionnaire on their age, race, education level, diagnosis date, cancer stage, and treatment history. HCP participants completed a demographic questionnaire on their profession and years of clinical experience.

Next, we conducted a qualitative interview with each participant as he/she reviewed the DA in person using tablets provided by the study team or via virtual Zoom meetings using the remote-control function. A ‘think aloud’ approach was adopted where the participants would verbalize their immediate thoughts while reviewing the DA [34,35,36]. The interviewer posed additional questions from the interviewer guide on the content comprehensibility, adequacy, and format (Additional file 2). Additionally, HCPs assessed the DA for accuracy. Each interview lasted approximately 20–40 min and was audio recorded. Lastly, participants completed an acceptability questionnaire adapted from the Ottawa Hospital Research Institute [37]. BCS rated the format, content comprehensibility, and DA’s utility in decision-making, while HCPs provided their perceptions of the DA and its compatibility with their current workflow.

Data analysis

Descriptive statistics were used to summarize the participant characteristics and the quantitative measures in the acceptability questionnaires. Frequencies and percentages were used for categorical data. Median and range were used for continuous non-normally distributed data. All analyses were performed using STATA version 17.

Interviews were transcribed verbatim and analysed using directed content analysis in Microsoft Excel [38]. In each round, two study team members first reviewed the transcripts to familiarize themselves with the data before coding independently according to key concepts outlined in the interviewer guide. The coders then met to resolve any discrepancies. A summary report of findings and proposed revisions to recurring concerns raised by participants was then discussed with the steering group before finalizing the changes for the next testing round. Due to the iterative nature of alpha testing, data analysis was interspersed with rounds of interviews until thematic saturation. Saturation occurred when the steering group deemed that a new round would not yield additional significant insights.

Results

Study participant characteristics

We recruited 15 BCS and 8 HCPs across three rounds of alpha testing. The age range of all BCS was 46–67 years, with the majority being Chinese and diagnosed 8 to 13 years ago. In each round, BCS were sampled from different education levels. All were non-metastatic cases that received surgery. For HCPs, a diverse range of professions was sampled, including medical, radiation, and surgical oncology, nursing, and pharmacy. All HCPs had ≥ 10 years of relevant clinical practice (Table 1).

Table 1 Study participant characteristics

Alpha testing results

We conducted three rounds of iterative testing, with quantitative acceptability measures summarized in Table 2, while the qualitative comments and modifications for each testing round summarized in Table 3.

Table 2 Acceptability of decision aid among breast cancer survivors

Table 3 Summary of qualitative comments on prototype for each testing round with key corresponding modifications

Navigation and interactivity

While all BCS in round 1 found the DA to be easy to navigate, we acted on qualitative comments of a lack of clear directives on the clickable components of each page by standardizing navigation buttons and adding an example of a clickable button (Additional file 3). However, this added feature confused BCS in round 2, where 2/5 BCS indicated difficulty in navigation. Replacing with a new, animated introductory section to orientate users to the standardized navigation bar in round 3, participants responded favourably where all BCS found it easy to navigate. Adopting participants’ suggestions, we fine-tuned the colour scheme to minimize confusion over clickable text. Following increased animation use, a higher proportion of BCS (4/5) perceived the DA to be interactive in round 3.

Amount and type of content

We consistently supplemented the DA content based on participants’ information requests across all rounds, including frequency of treatment side effects, cost listing, and psychosocial resources. Notably, requests for information on the usual care model, the value of HCPs, and their specific care roles in the shared care model recurred. We improved text conciseness and optimized the length of DA after 4/5 BCS in round 1 found it ‘too long’. Eventually, all BCS found the final prototype in round 3 ‘just right’.

Content clarity and presentation

In the first two rounds, participants persistently feedbacked on medical jargon used in describing treatment side effects and confusion over the naming conventions of HCPs (e.g., primary care doctor, community pharmacist). In response, we reviewed the prototype to paraphrase jargon into layman language and created a new section dedicated to describing each type of HCPs a BCS may encounter before elaborating on care options. Furthermore, participants in round 1 highlighted the concept of ‘late and long-term side effects’ as challenging to grasp from text presentation. We then replaced the text with an animated infographic which participants in round 2 found useful and further addressed concerns over animations being too fast by slowing them down for better comprehension. We also incorporated participants’ suggestions for using a cyclic pictorial to depict shared care instead of linear imagery that did not capture the shared communication channels across HCPs as a distinct feature of shared care. Lastly, we utilized a human body image to contextualize the various treatment side effects for improved relatability to BCS. Screenshots reflecting the revisions are found in Additional file 3.

Information balance

As the shared care model is a relatively new concept, the preliminary prototype provided a disproportionately higher amount of information to explain the new model. Most BCS (3/5) in round 1 found this unbalanced information to favour the shared care option. To reverse this impression, we separated the description of care options into two separate sections with similar headings at comparable lengths. Despite the revision, most BCS (3/5) in round 2 continued to perceive the information presented as unbalanced and favoured shared care. We then reviewed the literature for strategies to improve information balance and adopted a side-by-side presentation with a head-to-head comparison table [39, 40]. This change improved the perceived balance, where all BCS in round 3 perceived the DA as balanced and not favouring either option.

Preference clarification exercise

Overall, participants affirmed the relevance of consideration factors included in the exercise but clarified phrases such as ‘patient navigation’ and ‘favourable cost-savings’. On its usability, participants in round 1 reported hesitancy in using sliders across continuous scales and wanted a results summary page to reflect a clear stance on their preference. In response, we transformed all questions in the exercise into Likert scales with interim options mapped to a choice disposition (pro-usual or pro-shared) with an introductory example question to improve results interpretation. However, this sample example question confused participants in round 2 as they reported difficulty responding to questions due to a lack of concrete experience of shared care to project their confidence. Acknowledging the hypothetical scenario of the shared care option, we revamped the exercise to a tabular presentation of consideration factors where users would provide a binary response indicative of their slant towards either option. We rephrased the shared care option to ‘I am willing to try out shared care’ to guide users to clarify their readiness for the alternative care option. While no major comprehensibility and usability concerns emerged in round 3, one participant disregarded preference statements on shared care due to preconceived disposition. To remove the undue influence of such preconceptions, we eventually hosted the exercise as a questionnaire on an external website with a scoring mechanism. Each statement addressing a consideration factor will be scored as ‘pro-usual’, ‘neutral’ vs. ‘pro-shared’ and accounted for its perceived importance. The final score will be an arithmetic mean of the responses to all questions and linearly transformed to return a final score ranging from 0 to 100%, with 50% representing neutrality towards either option, < 50% and > 50% indicating preference towards usual and shared care, respectively (Additional file 3).

Perceived utility of DA for routine use

Most BCS (60–80%) in each round found the DA useful for decision-making and the preference clarification exercise helpful (Table 3). HCPs shared similar sentiments, with the majority agreeing that the DA is a reliable and suitable tool to help BCS make informed and preference-based choices (Fig. 2). No HCP perceived the use of DA as against their beliefs and would cause more harm than benefits.

Fig. 2

Acceptability of decision aid among health care professionals (N = 8) for a perceived utility of decision aid and b implementation of decision aid for routine use

All BCS were willing to discuss the DA with their HCPs and recommend it to other survivors in the final round (Table 3). For the preferred mode of distribution, the majority’s preference for digital in the first two rounds (80%) was reversed in the last (20%). Qualitative comments revealed that the paper format would be appropriate for BCS to review in clinic waiting areas (Additional file 4). No consensus was achieved on the suitable timing of DA introduction, with participants indicating preference as early as upon diagnosis to years after active treatment. For HCPs, the majority agreed that it would be easy to experiment with the DA and integrate it into routine care provision without major changes to the existing approach (Fig. 2).

Discussion

We described the systematic development of a digital DA for BCS considering cancer follow-up care models through three iterative rounds of alpha testing. The final prototype (Additional file 5) was of appropriate length, easy to navigate, balanced, interactive, and usable by prospective users, including BCS and HCPs. This study represents an endeavour to expand DAs usage to guide alternative care model uptake based on care preferences. The development process revealed unique challenges and opportunities for future work.

The decision on survivorship care models differed from the traditional scope of DA application in oncology for procedures such as screening and treatment. Besides the reversibility of the decision, care model selection is inherently complex as it requires BCS to consider their health care setting and context beyond conventional risk and benefits associated with well-defined procedures [3]. The implication of this complexity is two-fold. Foremost, the care model offered as an option should be developed with a well-defined structure, controlling for potential variability in health system factors such as HCPs’ experience and communication style. For instance, we emphasized that all primary HCPs involved in our piloted shared care model received formal training from tertiary HCPs and detailed the continued access to oncologists as a cornerstone of care coordination. The purpose is to ensure the accuracy and clarity of care options presentation to promote informed decision-making. Second, comprehension of concepts related to care delivery may be challenging and vary with pre-existing knowledge or familiarity. This challenge potentially explained participants’ consistent request for more information on usual care and the roles of different participating HCPs across testing rounds. Stirling et al. shared similar sentiments while developing a DA on respite services for patients with dementia in various care settings, as users reported reduced relevance of the DA based on care site accessibility [41]. Furthermore, the perceived value of shared care may be compromised by a generally poor understanding of the ‘care integration’ concept in Singapore, where a study revealed a disproportionate focus on cost and accessibility [42]. Thus, it was unsurprising that participants found phrases such as ‘care coordination’ and ‘navigation’ confusing while consistently acknowledging cost and convenience as important consideration factors. Recognizing this complexity, BCS with lower health literacy may require HCPs’ assistance to clarify concepts related to the care system.

Besides being an implementation barrier for alternative care models, low confidence in primary care HCPs among BCS also poses a recurring challenge in the design of preference clarification exercises [5, 26]. In the initial testing round, participants consistently emphasized that their low confidence in primary care to manage cancer-related issues was the major deterrent from shared care, limiting the utility of the exercise in influencing their choice disposition. In contrast, BCS perceived the highlighted strengths of primary care in health promotion and comorbidity management as relatively less important, possibly due to inadequate community health-promoting practices [43]. Correspondingly, the team focused on strategies to counter these inherent biases towards the primary care HCPs involved in shared care without disrupting information balance. Acknowledging BCS’ uncertainty, we reframed the exercise to explore BCS’ openness towards trying shared care, minimizing potential cognitive dissonance between the exercise results and choice predisposition. After experimenting with different preference clarification methods each round, the multicriteria decision analysis was employed in the final prototype, consistent with recommendations from the latest meta-analysis [44]. This method probes users to consider and weigh factors besides the sole confidence factor.

While the primary aim of this study was not to evaluate DA implementation, preliminary results revealed that both BCS and HCPs perceived the utility of the DA favourably, with no major challenges anticipated with integrating DA introduction into existing workflows. While agreeable to the usability of the DA, the preferred distribution mode and the timing of introduction were heterogeneous. While BCS generally preferred the digital format in earlier rounds, a more neutral stance in the last round could be due to the older age of the participants. The availability of print materials in a formal cancer centre setting would be compatible with the information-seeking behaviour of elderly Chinese women in Singapore [45]. However, a significant drawback of the print version would be the potential loss of interactivity and animations embedded in the current prototype. Besides providing a print version as an alternative to boost access to the older group, BCS’ interpersonal networks could be tapped to improve uptake of the digital format through recommendations by HCPs and fellow survivors [45]. Our results exemplified this strategy’s feasibility as most BCS respondents were willing to recommend the DA to others, potentially through support groups.

The main limitation of this study was the underrepresentation of low literacy groups, as all survivors minimally held a secondary school degree. Nevertheless, we also drew strength from our user-centred design approach [46], where the sample frame included prospective DA users. We purposefully sampled HCPs from disciplines that routinely encounter BCS, representing windows of opportunities for DA introduction and usage. Additionally, we managed to sample BCS with varied characteristics to capture diverse user experiences. These characteristics included an age range that coincided with the highest breast cancer incidence rates and a stage distribution that mirrored the Singapore Cancer Registry data [47]. This codesign engagement is crucial in avoiding the trend of suboptimal adoption rates reported by DA developers [48]. Furthermore, we plan to nest the developed DA formally within the shared care model to be used by prospective BCS in the upcoming scaling up phase. This formal inclusion as a packaged intervention could be an implementation strategy to promote sustained uptake. On generalizability, the DA developed provides a structural foundation for information adaptation to portray the primary care-led model as an alternative option. Notably, both scenarios necessitate a basic understanding of the health system. Lastly, future work should explore employing AI in DAs to enhance and personalize information delivery, language complexity, and quantity of information based on predefined characteristics such as age and familiarity with the primary care system.

Conclusion

We systematically developed an acceptable DA for BCS considering the usual or shared care model piloted in an ongoing trial. Our experience highlighted an additional need to provide contextual information on the health care system when addressing decisions related to care delivery. Importantly, prospective developers for similar decisions should actively address variability in health system factors (e.g., HCPs’ training) to describe care models accurately while consciously identifying and clarifying inherent biases, such as low confidence levels in primary care observed in our study. The DA is now ready for integration into the shared care model to be scaled up and field-tested. Future work could explore adapting the informational structure to care models with different degrees of primary care involvement and leveraging AI to optimize information delivery (Additional file 5).