Review methods
Our review focused on descriptions of the PtDA development process. We obtained these descriptions from three main sources: a) published papers describing 86 PtDAs that had been evaluated in randomised controlled trials and included in the latest update of the Cochrane Collaboration review of decision aids [3]; b) linked papers cited in the trial reports that provided expanded descriptions of how the PtDAs had been developed; and c) papers and web reports describing the development process used by a convenience sample of organisations that had developed several decision aids and had published papers or web reports describing their development process. The checklist we used to review these descriptions of the PtDA development process appears in Additional File 1, Appendix 1.
The PtDA development process has been described in a number of trial reports and associated articles. Many of the studies included in the Cochrane Collaboration review gave only cursory descriptions of how their PtDAs had been developed, but several linked papers provided useful supplementary detail [5–16] . Many PtDAs reported in the trials included in the Cochrane Collaboration review were ‘one-offs’ designed for research purposes rather than for wider distribution, and therefore did not generate insights into the implementation issues that might have been addressed during development. However, the experience of organisations involved in developing series of PtDAs is helpful for fleshing out the details of what is involved. Some groups that have developed multiple PtDAs have proposed guidelines for their development [17–20], or described insights generated by a particular approach [21]. The following is a brief overview of selected approaches to decision aid development.
Processes used by experienced PtDA developers
Ottawa Decision Support Framework
O’Connor is among the earliest authors to describe the development of a PtDA, and the Ottawa Decision Support Framework (ODSF) guided the development of at least 22 of the PtDAs included in the Cochrane Collaboration review [10, 18]. Based on expectancy value, decisional conflict, and social support theories, the framework includes three key elements: 1) assessment of determinants of decisions (both patients’ and providers’); 2) provision of decision support interventions to prepare the patient and provider to make and implement a decision; and 3) evaluation of the success of the interventions at improving the quality and outcomes of the decision process. Additional detail is provided to define determinants of decisions, such as socio-demographic and clinical characteristics; patients’ and providers’ perceptions of the decision and of what important others think about the decision; and resources (both personal and external) available to make the decision.
The authors noted that the goals of decision support are to address modifiable and suboptimal decision determinants, such as inadequate knowledge, unrealistic expectations, unwanted pressure, and inadequate support. They encouraged the use of tailored outcome probabilities, detailed descriptions of benefits and risks, and information on the opinions and perspectives of others (both clinicians and patients) on the decision. Using the example of a decision aid aimed at helping women decide about use of postmenopausal hormone therapy, O’Connor outlined an iterative development process involving the research team and panels of patients and experts, with the PtDA content based on clinical guidelines, and structured guidance in clarifying values and implementing a decision is provided by a personal worksheet [18].
The Ottawa Framework is particularly relevant to ‘preference-sensitive’ decisions, which involve careful deliberation and consideration of trade-offs among options. Perhaps because the PtDA developed by O’Connor was based on an existing high-quality clinical guideline, the framework provides little advice for how developers should review and synthesize the relevant clinical evidence. The framework also does not address how developers might deal with conflicts of interest, achieving consensus on the evidence, maintaining the PtDA content over time, or implementation outside research settings.
Cardiff University
Based on their experience of developing three web-based PtDAs over a seven-year period, Elwyn and colleagues proposed a development process for web-based decision support interventions [17]. This systematic ‘process map’ includes three main steps: 1) content specification, with an emphasis on ensuring that patients’ perspectives on the proposed options are sought and included in addition to synthesis of the scientific evidence; 2) design, including storyboarding, an iterative phase of trial and experimentation called “sandpit” testing, and usability testing; and 3) field testing with patients facing the decision and clinicians who are interacting with them. The process calls for extensive documentation, including: a protocol document that explains the decision and highlights the rationale for developing a PtDA; evidence synthesis based on systematic reviews or comprehensive literature searches; storyboard; and technical specification document to guide the website development.
The Cardiff projects were overseen by a project management group, which retained editorial control, and included involvement at all steps by key stakeholders including clinicians, patients, and policymakers. Unique challenges faced by developers of web-based tools are highlighted, including decisions regarding navigation (free versus mandated) and use of interactivity (audio, video, gaming, avatars, etc.) in ways that add value and enhance ease of use yet avoid over-engineering. The authors found little evidence to inform best practices in these areas.
The process outlined by Elwyn and colleagues is widely applicable across a range of situations for which decision support interventions may be developed (i.e., screening, treatment, etc.) and a variety of media (although some of the concepts included, such as storyboarding, are adapted from film production). However, as the authors acknowledge, the process is time-consuming and costly: three PtDAs developed using this process each took two to three years to develop and test. Insights gained from early efforts could be generalized to create templates to allow more efficient, less costly future development. The process outlined does not offer recommendations regarding conflict of interest, processes for achieving consensus on the evidence, maintenance of PtDA content over time, or implementation outside research settings.
Dutch Institute for Healthcare Improvement
Researchers at the Dutch Institute for Healthcare Improvement reported on the development over a 12-month period of 6 decision aids based on existing evidence-based clinical guidelines [19]. Citing the Ottawa Decision Support Framework and the IPDAS standards, the authors followed four key steps: 1) establishment of criteria and selection of topics; 2) assessment of patients’ information needs via literature review and focus groups; 3) drafting of the aid, including iterative review by a multidisciplinary working group of health professionals, guideline developers, decision-making experts, and patients, and with reference to existing aids on the topic; and 4) endorsement of the aid and establishment of ownership and responsibility for the maintenance and updating of both the supporting guideline and the decision aid itself. With regard to implementation, the authors call for concomitant development and coordinated release of clinical practice guidelines and accompanying PtDAs that support their application, and for identifying and acknowledging early in the guideline development process any so-called ‘grey zones’ of uncertainty regarding patient preferences.
The outlined process appears efficient and scalable when high-quality evidence-based practice guidelines are available. However, the authors acknowledge that additional research is needed to evaluate the effect of the aids in practice within the Dutch health care system. The evidence synthesis step used by other developers is addressed by use of national evidence-based guidelines; the authors note that these should also meet internationally accepted quality criteria, for example the AGREE guidance [22]. The resulting tools include a values-clarification method, but the process by which the method was selected and ‘populated’ with non-directive, standardized questions is not defined.
Mayo Clinic
Montori and colleagues described insights gleaned from the pragmatic process they followed to develop the Statin Choice decision aid for patients with diabetes, which was evaluated in an RCT included in the Cochrane Collaboration review [21]. In particular, the authors describe how observations of clinical interactions during office visits, and of early prototypes in use during patient-provider encounters, can inform the ultimate format, design, and content of the final PtDA. Similar to other developers, their experience reinforces the importance of flexibility, iteration, and involvement of patients and clinicians throughout the process. This article does not recommend a particular development process, but rather offers insights unique to the approach their research group chose. The direct observation methodology that Montori and colleagues describe may complement more traditional needs assessment approaches for informing developers about what patients and physicians need from a PtDA. Observing early prototypes in the setting they are being designed for can also be an important step in ensuring that the intervention will work as intended and have the desired effect on the decision making process. These insights also highlight the importance of flexibility during the early stages of design and development.
Informed Medical Decisions Foundation
Developers that produce PtDAs for both research purposes and for public distribution, such as the Informed Medical Decisions Foundation (IMDF), provide details of their development process on their organization’s website (http://www.informedmedicaldecisions.org). Ten of the RCTs included in the most recent update of the Cochrane review used PtDAs developed by this process. IMDF lists the following elements in their PtDA development process: 1) involvement of healthcare providers representing key clinical specialties, overseen by a clinician who divests him or herself of any potential financial conflicts of interest; 2) involvement of patients at several stages, including needs assessment via focus groups and literature reviews; and 3) review and evaluation of PtDA drafts by providers and patients prior to their release for general use. The approach outlines processes for evidence review and synthesis, disclosure of funding source and conflicts of interest, and periodic review and updates.