- Research article
- Open Access
- Open Peer Review
Patient decision aids: a content analysis based on a decision tree structure
BMC Medical Informatics and Decision Making volume 19, Article number: 137 (2019)
This paper presents the preliminary results of a decision-tree analysis of Patient Decision Aids (PDA). PDAs are online or offline tools used to structure health information, elicit relevant values and emphasize the decision as a process, in ways that help patients make more informed health decisions individually or with relevant others.
Twenty PDAs are randomly selected from the International Patient Decision Aids Standards (IPDAS) (https://decisionaid.ohri.ca/AZlist.html) approved list. An evaluation tool is built bottom-up and top-down and results are described in terms of communicating uncertainty, completeness of the decision tree, ambiguous or misleading phrasing, overall strategies suggested within personal stories.
Twelve of the analyzed PDAs had branches of the decision tree which were not discussed in the tool and 6 had logically ambiguous phrasing. Many tools included dichotomous options, when the option range was wider. Several options were clustered within the “Do not take/Do not do” option and thus the PDA failed to provide all comparisons necessary to make a decision. Some tools employ expressions that do not differentiate between lack of information and known negative effects. Other tools provide unequal amounts or non-comparable bits of information about the options.
These results indicate a very loose range of interpretations of what constitutes an option, a treatment, and a treatment option. It thus emphasizes a gap between theory and practice in the evaluation of PDAs. Future developments of PDA evaluation tools should keep track of missing decision tree branches, accurate communication of uncertainty, ambiguity, and lack of knowledge and consider using measures for evaluating the completeness of the option spectrum at an agreed period in time.
Patient Decision Aids (PDA) are decision support systems that help patients decide by providing information that is purposefully structured for decision-making. Behind any decision support system there is an abstract representation of the decision-making process, called a decision-tree. In this paper we argue that most PDAs have missing decision-tree branches, many have logically ambiguous statements or include only 2 of many options, thus re-framing the decision by providing an incomplete context.
What are PDAs?
PDAs are online or offline tools used to structure health information and elicit relevant values in ways that help patients make more informed health decisions individually or with relevant others [1, 2]. Their design is based on the idea that informed decisions improve when information is structured and the decision process is emphasized [3,4,5]. PDAs differ from Patient Information Resources (PIR) in that they not only inform patients, but also help them elicit their values and preferences and may include information about the decision-making process, such as a need to search for more information or to delay the decision . In doing so these tools may be tailored to accept user’s input (called interactive PDAs, such as the one available here: ) or the values and constraints of relevant others (family or two person PDAs, such as the one available for deciding on a place of care ). A list of approved PDAs can be found at .
PDAs are underpinned by Shared Decision-Making (SDM), a philosophy of doctor-patient communication and the new emerging gold-standard in healthcare . SDM is usually a three-fold intervention aimed at activating patients to ask questions, training health and medical professionals to inform and include patients’ values in treatment decisions [11,12,13,14,15,16,17,18] and using patient decision-support systems before, during and after doctor-patient consultations [16, 18,19,20,21,22,23]. SDM is underpinned by the ethical philosophy of patient autonomy and right to make an informed decision [20, 24,25,26,27,28].
Why have PDAs been used?
PDAs have been designed to improve and standardize the quality of doctor-patient communication in order to allow patients to make more informed health-related decisions [1, 29,30,31]. Three main problems of doctor-patient communication are meant to be addressed in any PDA: the knowledge gap between doctors and patients in order to facilitate patient activation (asking questions, understanding doctor’s answers, etc.) [32,33,34,35], individual differences in information delivery during consultations [20, 36], and the lack of information about the decision-making process behind the health condition or screening [6, 37]. Thus, PDAs are aimed at providing the following solutions: they locally bridge the knowledge gap between doctors and patients enough to make a common-sense discussion possible; they standardize the transmission of information in doctor-patient interaction; they offer a longitudinal process-based perspective, rather than a static resolution-based perspective on health-related patient decisions. PDAs are not artificial intelligence tools designed to make a decision for the patient. They structure the information, elicit patients’ values and reveal the steps of the decision process in a way which makes an informed decision possible.
When and how are PDAs used?
PDAs are used for health concerns which have multiple possible approaches, options or treatments, and are ethically required to include the option not to take any treatment (see 27–29 for a bioethical discussion about this). When a health concern has a single known solution, it is nevertheless possible to build a PDA which helps the patient decide whether to choose the option available or not and to understand the consequences of this decision. PDAs may be used before, during or after doctor’s consultations, depending on their purpose and design.
Why improve PDAs?
The study of PDAs is very important because they have the potential to bridge various gaps between doctors and patients. First, doctor’s consultations take very little time , which translates either into relatively little time for addressing patients’ questions or into leaving out information such as self-management and complementary or alternative medicines (CAM) [39, 40]. Second, patients increasingly try to bridge the doctor-patient knowledge gap by searching for health information online [41,42,43,44]. With low research skills [45,46,47] and low health literacy in approximately 50% of the US population  and 47% of the European population , searching for health information online can lead to mis-information. Therefore, developers and medical practitioners, healthcare providers and decision-making specialists evaluate PDAs based on the extent to which they provide appropriate information and the relevant structure for decision-making.
This paper, therefore, aims to contribute to the improvement of PDA content development and structure. In the following sections we review the main decision-making theories that inform content structure for PDAs, and we present the results of a content decision-tree analysis of 20 randomly chosen PDAs. A custom-made decision-tree evaluation tool is described which includes both bottom-up and top-down indicators. Results will reveal how many of the PDAs have missing decision-tree branches, hinder option comparison and have logically ambiguous phrasing. The results are then discussed with respect to possible explanations, solutions and future research.
In the next subsections we show that completeness and option comparability of the PDA decision-tree, as well as the logical ambiguity of PDA contents are not thoroughly addressed through the currently available PDA evaluation criteria. With this aim, we first review current PDA evaluation criteria and tools. Then, we show why analysing the content structure of PDAs from the proposed points of view is important.
Current PDA evaluation criteria and tools
PDAs are evaluated based on their development process and their impact. At development level, the International Patient Decision Aids Standards Collaboration (IPDAS) includes evaluation of the following domains: “systematic development process; providing information about options; presenting probabilities; clarifying and expressing values; using patient stories; guiding/coaching; disclosing conflicts of interest; providing internet access; balanced presentation of options; using plain language; basing information on up to date evidence; and establishing effectiveness” , p. 1. These variables are measured a priori.
The impact of PDAs has been assessed with respect to different constructs like health outcomes, quality of the decision-making process, quality of care and value-congruence of decision. These are measured a posteriori.
Health outcomes include “values clarity, decision certainty, decision regret, confidence, desire for participation in decision, question asking, actual participation in decision, communication quality (information provision/receipt, good processes of communication/care, satisfaction with communication/decision/care” , p. 3.
The quality of the decision-making process  is evaluated based on the extent to which patients have a clear formulation of the decision problem (measured by the Preparation for Decision Making Scale [32, 53], feel informed about the options, risks, benefits and consequences (measured by a subscale of the Decisional Conflict Scale ), feel they know their values, are actively involved in their care decision (measured by the Perceived Involvement in Care Scale  based on their preferences (measured by the Control Preferences Scale ).
Quality of choice is operationalized as “the extent to which patients are informed and receive treatments that reflect their goals and treatment preferences” . This is an objective assessment of patient’s knowledge of the options and outcomes and the concordance of the chosen option and what matters most to the patient . The last aspect of quality of choice is also called value congruence of the decision [1, 2].
Although the IPDAS requirements are minimal , their ambiguity has provided grounds for criticism . On the other hand, process-based evaluation criteria have been criticized for overlooking improvements in patients’ quality of life . In this paper we argue that, for the moment, PDAs also have problems accurately representing and presenting the decision tree underpinning their content structure. Moreover, we argue that the PDAs analysed represent incomplete (or clustered), partially comparable decision trees which also have logically ambiguous phrasings.
Why consider completeness, comparability and logical non-ambiguity to evaluate PDA content?
In general, content structure in PDAs is important for PDA developers because it directly influences the resolution of the decision-making process. Incompleteness, non-comparability and logically ambiguous phrasing are content structure characteristics which lead to biased decisions, by definition [57,58,59]. Incompleteness of the decision tree means that there are options which have not been presented or are somehow concealed . Non-comparability means the criteria used for evaluating options are not the same for each option [57, 59]. Logical ambiguity means that the phrasing does not allow the user to make a clear inference based on the information provided. While incompleteness and non-comparability are characteristics of the decision-tree underlying the content structure , logical non-ambiguity is a characteristic of the content itself. However straightforward this may seem, evaluation of contents based on these criteria is not trivial.
There are three main theories which explain the influence of content structure on the resolution: descriptive, normative and predictive theories of decision. The descriptive theories of decision may support a certain content structure based on how patients decide and what their decision-making needs are [6, 37, 60, 61]. Normative decision theories may support a certain content structure to avoid the risks associated with cognitive traps [61, 62] or with ethical concerns [51, 63, 64]. Predictive theories of decision relate current content structure to patients’ future decisions. These may argue that a certain content structure will make patients choose one option more often than others, irrespective of what would actually meet patients’ needs [65, 66]. Decision support systems are expected to attenuate the shortcomings emphasized by descriptive theories of decision-making behaviour and employ evidence-based normative theories in order to avoid predicted hazardous decisions.
Decision tree analysis is based on normative theories of decision and has long been used in medical decision-making [67,68,69,70], as well as in other disciplines . For PDA development and evaluation, a decision tree is an abstract structure which transforms information into knowledge by providing the following structure :
What is the decision?
What are the options (including the option to do nothing)?
What are the expected outcomes of each option?
What are the probabilities of each expected outcome?
Which options have unknown outcomes and why are they unknown (research has not been performed; research is under way; research results are contradictory, no research has been undertaken so far, etc.)?
The completeness of the option range;
The comparability of information across options.
The use of a decision tree analysis assumes that the decision-maker is autonomous [12, 63, 73, 74]. From this point of view, PDAs should address the needs of patients, surrogate decision-makers, as well as family members or relevant others considered important by the decision-maker.
Normatively, PDAs should avoid the following traps:
partitioning the options, since decisions are context dependent; for example, by listing only a part of the available options, instead of all the available options, certain options may seem more appealing than in the complete context.
presenting detailed outcomes of only one part of the options; for example, by providing expected frequencies for treatment outcomes, but not for the option of not taking any treatment.
presenting detailed outcomes for taking the treatment for short periods of time, when patients are usually expected to take the treatment for much longer periods of time. For example, presenting success rates and side-effects for taking contraceptives for one year, when most women are expected to take them for 5 to 10 years.
presenting unknown information as lack of effect; for example, by reporting that “studies have not shown the effects” or that “there are no studies showing that...”, when studies have not been performed on that particular treatment.
From a descriptive point of view, there are further criteria which are relevant for the quality of the decision. For example, PDA developers should avoid suggesting overall strategies that decision-makers use to avoid deliberation. These are usually described by the literature on heuristics and biases [62, 75, 76], small sample decisions [77, 78], emotional decisions [79,80,81] and custom based or social norms based decisions [82,83,84]. Such strategies fundamentally change the decision-tree and simplify it to suit cognitive limitations.
Based on these theories, it is possible to propose the following definition: an informed decision is a decision in which all branches of the logical decision tree are openly discussed. Moreover, if all branches of the logical decision tree are described in a complete and comparable manner together with the patient’s values and the steps of the decision-making process, then the decision is considered informed. When some branches have unknown information, the decision is considered informed if the lack of information is communicated clearly. If some branches are not known or are clustered in ways that do not allow full comparisons, or if it is unclear where there is incomplete knowledge, the decision is not informed. Also, the decision is considered informed, if the steps in the decision-making process needed in order to acquire more information are fully and openly discussed.
To sum up, based on both a decision theory driven definition of the informed decision and the ethical principle of patient autonomy, it is possible to suggest that the following criteria are used for PDA evaluation:
Openly signalling incomplete knowledge
Identifying missing decision tree branches
Controlling that criteria in the Pros and Cons section is comparable
Controlling that criteria in the Pros and Cons section are expressed in logically non-ambiguous statements
Expressing information content in logically non-ambiguous statements
Controlling overall strategy reports in Personal Stories
Presenting the set of Personal Stories in ways which do not balance the choice toward a certain option
Developing PDAs in ways which allow a Shared Decision to be made with relevant others, including the doctor
Developing PDAs in ways which allow a Shared Decision to be made if the intended users are Surrogate decision-makers
In this section, an evaluation tool is built bottom-up (based on grounded theory methodology [85, 86] which elicits variables driven by the particularities of the PDA content analysed) and top-down (based on relevant theories of decision-making identified in the Background section). We present the sample and operationalization of bottom-up and top-down indicators such as communicating uncertainty, completeness of the decision tree, ambiguous or misleading phrasing, and overall strategies suggested within personal stories.
The IPDAS currently holds a list of 34 PDA developers that provide free access to their IPDAS compliant tools . A total of 337 PDAs have been developed in this way ) at the time the analysis was made.
Twenty online tools have been randomly selected from the IPDAS accepted PDA list in order to construct the evaluation criteria (see the sample in Table 1). Only one tool from the sample could not be evaluated because it was not free for use and required international ordering.
From the sample, there were 17 interactive tools (i.e. they accept input information from patients, such as blood test results or other relevant information) and 2 which included sections on or openly discussed sharing the decision with other parties beside the doctor (support groups, psychologists, family, etc.). Also, 3 tools were only available in PDF format, 13 as Web Pages and 4 in several formats (Web and PDF).
The tools analysed can be divided into four types, based on the way they structure the decision-making process (see Decision aid type in Additional file 1):
eligibility, risk assessment, decision, documents 
knowledge, pros and cons 
knowledge, values, resolution 
question structure 
test, information, re-test, decision 
The structure of the decision-making process depends on the organization that developed the tool.
Data analysis and operationalization
Data analysis was performed in two steps. First, a core set of top-down criteria has been created to assess the decision-tree first-order components and second order characteristics. Secondly, grounded evaluation criteria were added to the evaluation grid based on the characteristics of each tool. At the end of this analysis, all tools would be evaluated against all the criteria (see Additional file 1 for the data analysis database). No additional criteria have emerged after the 11th tool analysed and a total of 37 criteria have been built in this way. The criteria constructed in this way are provided in Table 2 and Table 3. Based on the review presented in the Backgound section, they are clustered into three main categories: decision tree, decision process and content.
Decision tree criteria refer mostly to the elements of the tree and their second order characteristics, like option range completeness, comparability and incomplete or complete knowledge.
The Decision tree category includes three criteria: Incomplete knowledge acknowledged, Missing structure, and Pros and Cons logic.
Incomplete knowledge acknowledged is a criterion which indicates whether there is insufficient information about a tree node. It proposes that PDAs should emphasize cases in which information currently available in the decision tree is incomplete, that is either uncertain (expressed usually in probabilities of success, false-positive, false-negative probabilities, etc.), ambiguous (e.g. if work is currently being undertaken to find a suitable treatment option or if research results are too contradictory or non-comparable to each other to release probabilities of success, etc.) or unknown (e.g. if no study has ever been conducted). Good performance in this case would be to have PDAs which indicate not just the quality of the literature reviewed in the development process, but also clearly state cases in which nothing is known about this option (e.g. there has been no research undertaken on this topic). In this case, the lack of research should be clearly delimited from negative result cases (e.g. research has shown that …, but did not show this …) and from contradictory results (e.g. some research has shown this and some research has shown that, based on the same or comparable research design).
Missing structure is a criterion aimed at checking if there are any missing branches in the PDA decision tree, by comparing the decision tree which emerges from the PDA with the logical decision tree derived from it. The PDA decision tree can be revealed for any PDA by laying down the options, sub-options, the probabilities and so on. This abstract structure is laid out only for the options and the information provided in the PDA content and possibly the first order logically accurate inferences based on it. Based on this structure, the complete and comparable logical decision tree can be constructed. This is the standard of comparison for the PDA decision tree based on which the criterion Missing structure is evaluated.
Pros and cons logic is a criterion which is also based on the construction of the decision tree, but only the one which emerges from the pros and cons section in the PDA. As in the case of the Missing structure criterion, based on the pros and cons decision tree, a complete and comparable logical decision tree is constructed as the standard for comparison. Differences or imbalances in the information provided for the pros and cons decision tree and the logical one are described and counted in the Pros and cons logic criterion.
The Decision process section includes four criteria: Overall strategy in Personal Stories, Decision-aid type, Shared Decision, Surrogate awareness.
Overall strategy in Personal Stories describes the cases in which personal stories suggest that the person has employed a certain strategy over the entire decision tree. Strategies like “Try out everything” or “Choose only treatments with less side-effects” or “I had heard about this” implying the use of the accessibility bias and so on are likely to help people choose a strategy, and not necessarily to make a choice. Some strategies help elude the deliberation and value elicitation process, but the positive or negative nature of this situation cannot be judged irrespective of the resolution content. In some cases it may completely bypass information which could be relevant, in others it may be an indicator of how others have decided. There is no example of good practice in this case, since the influence of overall strategies depends mostly on their health-related consequences. This is why, in this analysis we are only interested in controlling this variable.
Biased Personal Stories refer to the use of another type of heuristic, which can be derived from the personal story section, and that is the frequency with which others have chosen a certain treatment. If the personal story section includes a total of four stories and three of them discuss the same treatment option, a decision-maker may deduce that this is the most frequently chosen option and be influenced by it or, on the contrary, deduce that this is what the designer or funding body of the PDA would prefer and choose the one less favoured. Good practice in the use of personal stories should include equal frequencies of each option .
Shared Decision and Surrogate Awareness are two criteria which verify whether the values and choices of other relevant people are taken into consideration, either directly (Shared Decision) by providing separate space for their preferences, options, values or choices, or indirectly, by suggesting that their preferences, etc. be taken into consideration. In this case, good practice varies from problem to problem (Surrogate Awareness).
The Content category includes a single criterion: Logically ambiguous phrasing. This criterion describes and counts the cases in which a phrase or sequence of sentences does not allow a clear-cut judgment to be made or opens up several logical possibilities which hinder the accuracy of the inference made based on this phrasing.
The bottom-up criteria can also be clustered into the same categories as before, pointing to the decision tree, process and content (Table 3). From the Decision tree category, the criterion which requires further explanation is Clustered Options. This criterion counts the cases in which several options are presented as a single option, as visible in the PDA section called “What are my options?” or in the Pros and Cons Section. As discussed earlier, such clustering leads to the impossibility of consistently comparing all options and it may thus bias the decision .
All indicators have dichotomous answers (yes/no), except the following: “Probable/certain”, “Story 1-4 Reported Outcomes”,” Story 1–4 Reported Side-effects”, “Overall Strategy in Personal Stories”, “Story 1-4 Reported Constraints”, “Decision-aid type”, “Phrasing problems“, “Missing structure 1 and 2”.
The data analysis revealed the following:
12/20 PDAs have missing decision tree branches;
8/20 PDAs cluster several options into a single one;
12/20 PDAs have logically ambiguous phrasing;
4/20 PDAs have missing or unclear information in the pros and cons section;
1/20 presented frequency measures with significance estimates;
1/20 PDA has a clearly biased personal story section.
In the following section the main results will be presented with respect to each relevant indicator. The numbers presented between block parentheses, e.g. [n], represents the number of the PDA given in Table 1. All results presented are based on the database in the Additional file 1.
Missing decision-tree branches
Including dichotomous options, when the option range is wider
For example, in tool , the options are presented as dichotomous, but after studying the information documents (presented only when the person rejects the medication) it is clear that there are more options clustered in the Do-not-take-medication option.
Including only certain options, but omitting others
For example, in tool  there is no option for taking only metformin (medication). The results presented include: 1) major lifestyle changes; 2) metformin and lifestyle changes; 3) placebo and lifestyle changes. It is unclear if there is no evidence for taking only medication with no lifestyle changes (although this may be a logical and possibly convenient option) or if the studies reveal contradictory results. In tool  the option called “watchful waiting” is also not described, despite being mentioned as different from “active surveillance”. Also, in tool  the option of assisted living (Gawande 2015) is not included, despite the fact that this service is also available in the country in which the tool was designed for.
Missing options in the final decision
No presentation of side-effects replaced with suggestion to discuss with the doctor
One of the analysed tools did not discuss the side effects of radiotherapy, mastectomy or lumpectomy . In the case of radiology it states that side effects should be discussed with the doctor. Side effects are a very important attribute of patient decision-making and very much relevant for this treatment decision.
Providing information about false-negative or false-positive results, but not both
For example, in PDA , a discussion about false-negative results was included, but there was no discussion about false-positive results.
Comparability of options
Failing to consider treatment time
In the tool for Dupuytren’s Disease  there is no discussion on the risks of collagen injection on the long term. If lack of knowledge on this aspect is the reason for this, then this should be emphasized.
Omitting decisions based on the cause of symptoms
Perhaps the most intuitive example is given by tool , where the treatment depends on the cause of the “sore throat” symptom. Consequently, a useful decision-making structure would provide advice on how to see what caused the sore throat and then recommend treatment there off. There is no mention that it is not possible to know what caused the sore throat or that it is too expensive, or any line of thinking which generated the options. Only 2 PDAs discuss the cause of symptoms, while 15 of the 20 PDAs analysed do not (Table 5).
Not giving the same details about the option “Do not take any treatment”
For example, the line “Why should you avoid type 2 diabetes” presents information in a way that is not comparable to the treatment options. Information about how many people get eye problems, nerve and kidney problems and after how long do these or does death occur would provide a more balanced approach to this ethical possibility.
The comparability problems are visible both from the missing structure variables summarised above and detailed in the Additional file 1, as well as by looking at the clustered options variable. In general, 8 out of the 20 PDAs analysed have either clearly clustered options or a possibly clustered option (Table 6).
Logically ambiguous phrasing
Mis-representing values, preferences and feelings
For instance, tools designed by Healthwise include a decision stage called “Your feelings” which usually is designed to elicit preferences. However, the questions in this stage seem not to differentiate between preferences, values and feelings. For example, tool  has a value elicitation section in which the questions are not only difficult to answer, but also have inappropriate scale measurements.
"I want chemo or lymph node surgery for the best chance of cure at the start (Not important … Somewhat important … very important).
"I might not need more treatment, so I want surveillance (same scale as previous);
"I’d rather have side effects from treatment than have surveillance (same scale);
"I can make sure I go to checkups and tests during surveillance (same scale);
“I don’t mind banking my sperm to have treatment” (same scale).
Phrasing the questions
For example, what should a patient answer to the first question if lymph node surgery is an option, but chemo is not? What should a patient answer to the second question, if surveillance is desired despite the fact that treatment might be needed? Another possibility is to rephrase the questions to elicit preferences of each option discussed.
A better scale would have been “Strongly Agree... Strongly Disagree”.
In general, value elicitation requires a different approach. For example discussing the degree to which each option leads to loss of personal freedom or dependence on others or the degree to which each option leads to increased personal care and less care for significant others (dependent or not) and so on. If taking a pill or doing exercises for the rest of your life is seen as “being dependent”, while a short term intervention is seen as “giving freedom” or the other way around, this may interfere with the final resolution. The current form does not allow value and preference comparisons.
Inaccurate reporting of probability
For example, in tool  the frequencies are explained in an inaccurate manner: “Blocks of 100 faces show a ‘best estimate’ of what happens to 100 women …”. However, blocks of 100 faces shows what may happen to 100 women, since not every 100 women will have the same results, but 100 randomly chosen women. Another example is the phrase “There is no way of knowing in advance if you will be one of those affected” is biased because it conceals that there is no way of knowing that you will be one of the not affected. The correct version would be, for example: “There is no way of knowing in advance if you will be one of those affected or not (who break a hip or not)”.
Incomplete presentation of study results
Presenting what research does not show is just as biased as showing only what it does show. In tool  the following phrase seems difficult to use: “Some men try dietary supplements for BPH, such as saw palmetto or beta-sitosterol. But scientific studies don’t show that saw palmetto helps with urinary problems or that beta-sitosterol is safe or helps over the long term”. This brings on the question of: what do these studies show? Do they show that these supplements are not better than placebo? Do some studies show significant effects, while other studies show insignificant effects? The reader cannot draw any practical conclusions based on this presentation of results.
In general, incomplete knowledge acknowledgment has been observed in 2 out of 20 PDAs, while most of the other PDAs analysed do not acknowledge incomplete knowledge relevant to the decision (Table 7). Further medical literature reviews on all areas of the PDAs studied is required in order to establish whether this situation is due to the fact that there is incomplete knowledge over the topic or to the fact that this was not a requirement of PDA standards. However, 9 out of the 16 PDAs which did not include any incomplete knowledge acknowledgment also have missing decision tree branches (Missing structure 1 OR Missing structure 2 * Incomplete knowledge acknowledged). The rest of 7 PDAs do not have missing branches. Also, from these 16, 5 have missing Pros and Cons logic and for the rest of 11, the Pros and Cons logic is not applicable (Pros and Cons logic * Incomplete knowledge acknowledged). Only 3 of these 16 PDAs without incomplete knowledge acknowledgment have related PDAs on the same topic and 8 do not have them (Related PDAs * Incomplete knowledge acknowledged).
Overall, logically ambiguous phrasing has been visible in 12 out of 20 PDAs analysed (Table 8).
In the Personal Stories Section, 4 PDAs had reports of overall strategies suggested within the personal stories. Only 1 was clearly biased toward one of the treatment options in the sense that the majority of personal stories  reported the choice of one of the options (see Table 9).
Most of the PDAs analysed did not discuss the possibility of having a shared decision, and only 2 of them were designed to allow for such a decision. From the 5 PDAs which advised the user to have a shared decision with the patient, 2 suggested it is shared with the doctor and 1 suggested it is shared with the doctor and others and 2 with the patient (Table 10).
Based on a random selection of IPDAS approved PDAs, we showed that many current tools have missing branches of the decision tree, discuss only two of many options or partition the option range such that comparability is impaired and had logically ambiguous phrasing.
The main limitation of this study is the small sample on which the conclusions are based. However, the random character of the sampling method adds strength to the conclusions. Either way, the analysis can be replicated and all variables use objective measures.
Based on this analysis, there are important questions to address for future research:
Do dichotomous options lead to informed decisions?
Many of the PDAs analysed present only two options. Some of them analyse three or four options, but this is rare. The most troubling cases are those in which only one option is basically presented, while all the other possible options are clustered in the “Do not” option. If other options are envisaged, they are inserted in a different tool, and this does not ensure comparability of all cases.
Do we need to re-think the value elicitation sections?
The decision-making literature makes a very clear distinction between values, preferences and feelings. While preferences express the order of the options, values are over-arching principles which can be applied differently to specific situations (such as moral values ). For example, freedom is a value opposed to dependency. It can be applied in many forms, from self-management, to patient participation in treatment decisions, to choosing treatments that avoid dependency on others and so on. In other circumstances, it is not possible for a patient to understand how a general value can be applied to a particular medical situation. In such cases, a decision tool should elicit the core value, help the patient realize whether further information is needed and direct the patient towards a discussion with the medical practitioner.
Does emphasizing what we don’t know lead to informed decisions?
Information about what information is missing is never included in the PDAs analysed. Missing information includes discussions about the cause of symptoms in the cases in which the PDA addresses treatment for symptoms (sore throat, low back pain, etc.), but also about lack of research or inconclusive results.
Two examples of good practice emerge from the analysed sample. Despite the fact that these tools have their own problems, they provide some interesting solutions for two of the problems identified earlier.
Tool  is a highly interactive PDA that provides risk assessments based on the patients’ blood test levels. Much information is available in pdf format. Several medicines are described in separate files, nutrition and exercise information is available. This is the only tool which presents so many options. However, because they are not presented on the same structure, comparing the treatments and their expected outcomes may prove difficult.
Tool  is an example of good practice in reporting uncertainty: “Another thing to understand is that the evidence can’t predict what’s going to happen in your case. When evidence tells us that 2 out of 100 people who have a certain test or treatment may have a certain result and that 98 out of 100 may not, there’s no way to know if you will be one of the 2 or one of the 98.”
Impact of the findings in practice
In light of these findings, there are grounds to consider a logical decision tree analysis for developing all PDAs. This recommendation is valid also for groups who perform literature reviews for PDA content development. It seems crucial that the decision tree emerging from the PDA is constantly verified against the logical decision tree, in order to identify the missing branches, clustered options, ensure completeness (at the time of the design) and comparability of options or elicit nodes where current level of knowledge is incomplete. More than this, the relationship between PDA content development and the decision tree should be one of mutual construction.
There is however a danger that even if the completeness of the decision tree is checked, some branches will remain concealed in two ways. First, an internally consistent (a perfectly comparable and complete decision tree with revealed areas of incomplete knowledge) but externally inconsistent PDA may create the illusion that the search for new options is not needed. Without a critical eye these criteria will not guard against internally consistent, but externally incomplete PDAs. Second, these criteria are not helpful if the end-goal of the decision is not also openly revealed. For example, to avoid this danger, PDAs could openly state whether the end goal of the PDA was to increase quality of life, cure the illness and/or to make a value-driven decision, all of them or something else.
In view of the results presented, it is possible to speculate on possible explanations for the missing decision-tree branches and the dichotomous options: (1) the goals of the decision are not clear or possibly confused with the decision-making problem; (2) there may still be no clear definition of what an option, a treatment option and a treatment are [108, 109]; (3) there may be an imbalance of power within different branches of medicine or between allopathic medicine and non-allopathic medicine. These could become the subject of further studies and will be briefly explained below.
First, options (treatment options, in this case) are defined with respect to certain (health) goals. Different goals lead to different treatment options . Sometimes the goal is to cure an illness, other times the goal is to alleviate symptoms, other times the goal is to increase the quality of life and other times the goal is to make a value-congruent decision even if it leads to lower quality of life or death [63, 111, 112] or all of the above. Further research could investigate the extent to which the goals of the treatments, as opposed to the decision-problem are clearly stated in the PDAs.
Second, a potential treatment, as opposed to a validated treatment option, is generally accepted as a cure if it has been tested. However, it is not yet clear what kinds of potential treatments become the subject of testing. In the absence of a full list of potential treatments being tested, waiting to be tested and having been tested worldwide, it is possible that what constitutes a treatment depends on the way potential treatments are (accidentally or purposefully) selected to become the subject of testing and not just on the likelihood that they will cure the illness or increase quality of life, etc. It is possible that what makes a treatment is as much a representation of evidence as of its likelihood of belonging to the power or the mainstream group . Thus, being a more likely representative of the power or mainstream group leads to a higher probability of being selected for testing. Irrespective of the way in which treatments get to become the subject of research testing or not (availability of funding, notoriety, etc.), some treatments will become a treatment option or not simply because there is research done about them or not, and not because it is really the best treatment available. While evidence-based medicine is beyond any doubt the desirable standard, reflecting on how treatments become valid options or not is also very important. One type of power imbalance is visible in the analysed PDA discourse where treatments belonging to allopathic medicine are individualized, while treatments belonging to CAM are not differentiated. For example, life-style changes are discussed as if all such changes are equal. Similarly, physical exercising is discussed as if all types of exercises are equal. Consequently, the gap between theory and practice may be further studied not only in terms of content, such as the one provided in this paper, but in terms of the power relations emerging from the content and of the stated goals. Further research may address these issues directly and propose solutions for them, particularly in the Shared Decision-Making conceptual framework to which PDAs belong.
Third, assuming there is no power imbalance and the goals are clear and differentiated from the decision problem, a decision-making option should represent a single treatment option with respect to the decision problem and the goal, not more. A treatment which solves side-effects of another treatment should be part of the option, but a treatment which is likely to have independent effects should not be clustered with others in ways which make it difficult to compare with the others. Further research may devise clear guidelines to help practitioners and PDA developers to differentiate between these practically relevant theoretical distinctions.
Availability of data and materials
The dataset supporting the conclusions of this article is included in the Additional file 1 of this article.
Complementary and Alternative Medicine
Grading of Recommendations Assessment, Development and Evaluation
International Patient Decision Aids Standards
Patient Decision Aid
Patient Decision Aids
Portable Document Format
Patient Information Resources
Stacey D, Legare F, Lewis K, Barry M, Bennet C, Eden K, et al. Decision aids for people facing health treatment or screening decisions (review). Cochrane Database Syst Rev. 2017;4:CD001431.
Stacey D, Legare F, Col N, Bennet C, Barry M, Eden K, et al. Decision aids for people facing health treatment or screening decisions (review). Cochrane Database Syst Rev. 2014;Art No C(1):1–331 Available from: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD001431.pub4/full.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;17.
Legare F, Ratte S, Stacey D, Kryworuchko J, Gravel K, Graham ID, et al. Interventions for improving the adoption of shared decision making by healthcare professionals. Cochrane Database Syst Rev. 2010;5.
Stacey D, Bennett CL, Barry MJ, Col NF, Eden KB, Holmes-Rovner M, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011;10.
Coulter A, Ellins J, Swain D, Clarke A, Heron R, Rasul F, et al. Assessing the quality of information to support people in making decisions about their health and healthcare. Oxford: Picker Institute Europe; 2006. p. 70.
Keevil J, HealthDecision. Cardiovascular risk decision support tool. Patient Dec Aid. 2015; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1804. Cited 5 Feb 2017.
Ottawa Hospital. When you need extra care, should you receive it at home or in a facility? Patient Dec Aid. 2010; Available from: https://decisionaid.ohri.ca/docs/das/Place_of_Care.pdf. Cited 2017 Feb 5.
Ottawa Hospital Research Institute. A to Z list of patient decision aids. Available from: https://decisionaid.ohri.ca/AZlist.html. Cited 4 Feb 2019
European Health Parliament Commitee on Patient Empowerment and Centredness. In: Bonsignore C, Brolis E, Ionescu A, Karusinova V, Mitkova Z, Raps F, et al., editors. Patient empowerment and centredness; 2015. Available from: https://www.healthparliament.eu/wp-content/uploads/2017/09/EHP-papers_Patients-empowerment.pdf Cited 14 Aug 2017.
Hibbard JH. Engaging health care consumers to improve the quality of care. Med Care. 2003;41(1):I61–70 Available from: http://www.jstor.org/stable/3767729.
Kukla R. How do patients know? Hast Cent Rep. 2007;37(5):27–35.
Boivin A, Green J, van der Meulen J, Legae F, Nolle E. Why consider patients’ preferences? A discourse analysis of clinical practice guideline developers. Med Care. 2009;47(8):908–15.
Zhang Y, Su H, Shang L, Li D, Wang R, Zhang R, et al. Preferences and perceived involvement in treatment decision making among Chinese patients with chronic hepatitis. Med Decis Mak. 2011;31(2):245–53 Available from: http://mdm.sagepub.com/cgi/content/abstract/31/2/245.
Xie B, Wang M, Feldman R, Zhou L. Exploring older and younger adults’ preferences for health information and participation in decision making using the health information wants questionnaire (HIWQ). Health Expect. 2012;17:795–808.
Legare F, Politi M, Drolet R, Desroches S, Stacey D, Bekker H. Training health professionals in shared decision making: an international environmental scan. Patient Educ Couns. 2012;88. https://doi.org/10.1016/j.pec.2012.01.002.
Légaré F, Thompson-Leduc P. Twelve myths about shared decision making. Patient Educ Couns. 2014;96(3):281–6 Available from: http://linkinghub.elsevier.com/retrieve/pii/S0738399114002699. Cited 2016 Oct 22.
Diouf TN, Menear M, Robitaille H, Painchaud Guerard G, Legare F. Training health professionals in shared decision-making: update of an international environmental scan. Patient Educ Couns. 2016;99:1753–8.
Elwyn G, Edwards A, Wensing M, Hood K, Atwell C, Grol R. Shared decision making: developing the OPTION scale for measuring patient involvement. Qual Saf Health Care. 2003;12. https://doi.org/10.1136/qhc.12.2.93.
Elwyn G, Frosch D, Thomson R, Joseph-Williams N, Lloyd A, Kinnersley P, et al. Shared decision making: a model for clinical practice. J Gen Intern Med. 2012;27(10):1361–7 Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3445676/.
LeBlanc A, Ruud KL, Branda ME, Tiedje K, Boehmer KR, Pencille LJ, et al. The impact of decision aids to enhance shared decision making for diabetes (the DAD study): protocol of a cluster randomized trial. BMC Health Serv Res. 2012;12(1):130. https://doi.org/10.1186/1472-6963-12-130.
Kasper JF, Mulley AG, Wennberg JE. Developing shared decision-making programs to improve the quality of health care. QRB Qual Rev Bull. 1992;18(6).
Alden DL, Friend J, Chun MBJ. Shared decision making and patient decision aids: knowledge, attitudes, and practices among Hawai‘i physicians. Hawai’i J Med Public Health. 2013;72(11):396–400 Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3831568/.
Rubin M. The Collaborative autonomy model of medical decision-making. Neurocrit Care. 2014;20(2):311 Available from: https://link.springer.com/article/10.1007%2Fs12028-013-9922-2.
Emanuel EJ, Wendler D, Grady C. An ethical framework for biomedical research. In: Emanuel EJ, Grady C, Crouch RA, Lie RK, Miller FG, Wendler D, editors. The Oxford textbook of clinical research ethics. New York: Oxford University Press; 2008. p. 123–35.
Emanuel EJ, Emanuel LL. Four models of the physician-patient relationship. J Am Med Assoc. 1992;267(16):2221–6 Available from: http://www.antoniocasella.eu/salute/Emanuel_1992.pdf. Cited 22 Oct 2016.
Gulbrandsen P, Clayman ML, Beach MC, Han PK, Boss EF, Ofstad EH, et al. Shared decision-making as an existential journey: aiming for restored autonomous capacity. Patient Educ Couns. 2016;99(9):1505–10 Available from: www.sciencedirect.com/science/article/pii/S0738399116303093.
Elwyn G, Edwards A, Kinnersley P, Grol R. Shared decision making and the concept of equipoise: the competences of involving patients in healthcare choices. Br J Gen Pract. 2000;50:892–7.
Elwyn G, O’Connor A, Stacey D, Volk R, Edwards A, Coulter A, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333(7565):417 Available from: http://www.ncbi.nlm.nih.gov/pubmed/16908462. Cited 14 Oct 2018.
Elwyn G, O’Connor AM, Bennett C, Newcombe RG, Politi M, Durand M-A, et al. Assessing the Quality of Decision Support Technologies Using the International Patient Decision Aid Standards instrument (IPDASi). PLoS One. 2009;4(3):e4705. https://doi.org/10.1371/journal.pone.0004705 Cited 27 Oct 2016. Lee K, editor.
Munro S, Stacey D, Lewis KB, Bansback N. Choosing treatment and screening options congruent with values: do decision aids help? Sub-analysis of a systematic review. Patient Educ Couns. 2016;99(4):491–500 Available from: http://www.sciencedirect.com/science/article/pii/S0738399115301075. Cited 29 May 2017.
Hoffman AS, Llewellyn-Thomas HA, Tosteson ANA, O’Connor AM, Volk RJ, Tomek IM, et al. Launching a virtual decision lab: development and field-testing of a web-based patient decision support research platform. BMC Med Inform Decis Mak. 2014;14(1):1–27. https://doi.org/10.1186/s12911-014-0112-8.
Asadi-pooya AA, Emami M. Perception and use of complementary and alternative medicine among children and adults with epilepsy: the importance of the decision makers. Acta Med Iran. 2014;52(2):153–7 Available from: http://acta.tums.ac.ir/index.php/acta/article/view/4839.
Banks HS, Tsegay G, Wubie M, Tamiru A, Davey G, Cooper M. Using qualitative methods to explore lay explanatory models, health-seeking Behaviours and self-care practices of Podoconiosis patients in north-West Ethiopia. PLoS Negl Trop Dis. 2016;10(8):1–11.
McHorney CA, Bricker DE. A qualitative study of patients’ and physicians’ views about practice-based functional health assessment. Med Care. 2002;40(11):1113–25 Available from: http://www.jstor.org/stable/3767825.
Zeuner R, Frosch DL, Kuyemchak MD, Politi MC. Physicians’ perceptions of shared decision-makingbehaviours: a qualitative study demonstrating thecontinued chasm between aspirations and clinicalpractice. Health Expect. 2014;18:2465–76.
Pieper D, Jülich F, Antoine S-L, Bächle C, Chernyak N, Genz J, et al. Studies analysing the need for health-related information in Germany - a systematic review. BMC Health Serv Res. 2015;15(1):1–18. https://doi.org/10.1186/s12913-015-1076-9.
Ildikó GA, Ábrám Z, Enikő A-L, Kinga T. Particularities of doctor-patient communication, assessed in 6 Romanian ambulatory practices. Acta Medica Marisiensis. 2012;58(3):166–8 Available from: http://actamedicamarisiensis.ro/particularities-of-doctor-patient-communication-assessed-in-6-romanian-ambulatory-practices/.
Kiesler DJ, Auerbach SM. Optimal matches of patient preferences for information, decision-making and interpersonal behaviour: evidence, models and interventions. Patient Educ Couns. 2006;61. https://doi.org/10.1016/j.pec.2005.08.002.
European Commission. Eurobarometer qualitative study: patient involvement. 2012. Available from: http://ec.europa.eu/public_opinion/archives/quali/ql_5937_patient_en.pdf
Fox S. Online Health Search 2006. Washington, DC; 2006. Available from: https://www.pewinternet.org/2006/10/29/online-health-search-2006/
Fox S, Duggan M. Health online 2013. Washington, DC; 2013. Available from: http://www.pewinternet.org/2013/01/15/health-online-2013/
Ayers SL, Kronenfeld JJ. Chronic illness and health-seeking information on the internet. Heal. 2007;11(3):327–47 Available from: http://hea.sagepub.com/cgi/content/abstract/11/3/327.
Chen C-C, Yamada T, Smith J. An evaluation of healthcare information on the internet: the case of colorectal cancer prevention. Int J Environ Res Public Health. 2014;11(1):1058–75 Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3924492/.
van Deursen AJAM, van Dijk JAGM, Peters O. Rethinking internet skills: the contribution of gender, age, education, internet experience, and hours online to medium- and content-related internet skills. Poetics. 2011;39(2):125–44 Available from: www.sciencedirect.com/science/article/pii/S0304422X11000106.
McTavish J, Harris R, Wathen N. Searching for health: the topography of the first page. Ethics Inf Technol. 2011;13:227–40.
Lawrence S, Gilles CL. Accessibility of information on the web. Nature. 1999;400:107–9.
Bohlman-Nielsen L, Panzer AM, Kindig DA. Health literacy: a prescription to end confusion: The National Academies Press; 2004. Available from: https://www.ncbi.nlm.nih.gov/books/NBK216032/.
Sorensen K, Pelikan MJ, Rothlin F, Ganahl K, Slonska Z, Doyle G, et al. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health Adv Access. 2015;25(6):1053–8 Available from: http://www.ncbi.nlm.nih.gov/pubmed/25843827.
Joseph-Williams N, Newcombe R, Politi M, Durand M-A, Sivell S, Stacey D, et al. Toward minimum standards for certifying patient decision aids. Med Decis Mak. 2014;34(6):699–710 Available from: http://journals.sagepub.com/doi/10.1177/0272989X13501721. Cited 14 Oct 2018.
McDonald H, Charles C, Gafni A. Assessing the conceptual clarity and evidence base of quality criteria/standards developed for evaluating decision aids. Health Expect. 2011;17:232–43.
Sepucha KR, Borkhoff CM, Lally J, Levin CA, Matlock DD, Ng CJ, et al. Establishing the effectiveness of patient decision aids: key constructs and measurement instruments. BMC Med Inform Decis Mak. 2013;13(2):1–11. https://doi.org/10.1186/1472-6947-13-S2-S12.
Graham I, O’Connor A. User Manual -- Preparation for Decision Making Scale [http://decisionaid.ohri.ca/docs/develop/User_Manuals/UM_PrepDM.pdf]. Accessed 30 Mar 2015.
Légaré F, LeBlanc A, Robitaille H, Turcotte S, Entscheidungskonflikten V, France Légaré K. The decisional conflict scale: moving from the individual to the dyad level. Z Evid Fortbild Qual Gesundh Wes. 2012;106:247–52 Available from: http://zefq-journal.com/article/S1865-9217(12)00040-2/pdf. Cited 7 Sep 2017.
Ballesteros J, Moral E, Brieva L, Ruiz-Beato E, Prefasi D, Maurino J. Psychometric properties of the SDM-Q-9 questionnaire for shared decision-making in multiple sclerosis: item response theory modelling and confirmatory factor analysis. Health Qual Life Outcomes. 2017;15(1):79 Available from: http://www.ncbi.nlm.nih.gov/pubmed/28431587. Cited 2017 Aug 9.
McCaffery K, Irwig L, Bossuyt P. Patient decision aid to support clinical decision making: evaluating the decision or the outcomes of the decision. Med Decis Mak. 2007;27:619–25.
French S. Decision theory: an introduction to the mathematics of rationality. Ellis Horwwod Ltd: Chichester; 1986.
Bridges DS, Mehta GB. Chapter 1. In: Representations of preference orderings. Berlin: Springer; 1995.
Roy B. Multicriteria methodology for decision aiding. Nonconvex optimization and its applications. Dordrecht: Kluwer Academic Publishers; 1996.
O’Connor AM, Drake ER, Wells GA, Tugwell P, Laupacis A, Elmslie T. A survey of the decision-making needs of Canadians faced with complex health decisions. Health Expect. 2003;6(2):97–109. https://doi.org/10.1046/j.1369-6513.2003.00215.x.
Gaissmaier W, Gigerenzer G. When misinformed patients try to make informed health decisions. In: Better Doctors, Better Patients, Better Decisions. Cambridge: MIT Press; 2011. p. 29–43.
Gigerenzer G, Muir Gray JA. Better doctors, better patients, better decisions. Cambridge: MIT Press; 2011.
Dworkin G. The nature of autonomy. In: The theory and practice of autonomy. Cambridge, New York: Cambridge University Press; 1988. p. 3–20.
Mackenzie C. Autonomy. In: Arras J, Kukla R, Fenton E, editors. The Routledge companion to bioethics: Routledge; 2015. p. 277–90. Available from: http://www.tandfebooks.com/doi/view/10.4324/9780203804971.
Perneger TV, Agoritsas T. Doctors and patients’ susceptibility to framing Bias: a randomized trial. J Gen Intern Med. 2011;26(12):1411–7. https://doi.org/10.1007/s11606-011-1810-x.
Bekker HL. The loss of reason in patient decision aid research: do checklists damage the quality of informed choice interventions? Patient Educ Couns. 2010;78(3):357–64 Available from: http://www.sciencedirect.com/science/article/pii/S073839911000008X.
Wellman MP, Eckman MH, Fleming C, Marshall SL, Sonnenberg FA, Pauker SG. Automated critiquing of medical decision trees. Med Decis Mak. 1989;9:272–84.
Bayoumi AM, Redelmeier DA. Decision analysis with cumulative Prospect theory. Med Decis Mak. 2000;20:404–12.
Sendi PP, Clemen RT. Sensitivity analysis on a chance node with more than two branches. Med Decis Mak. 1999;19:499–502.
Lamont A, Lyons MD, Jaki T, Stuart E, Feaster DJ, Tharmaratnman K, et al. Identification of predicted individual treatment effects in randomized clinical trials. Stat Methods Med Res. 2018;27(1):142–57.
Gladwin C. Ethnographic decision tree modeling. In: Ethnographic decision tree modeling. Qualitativ. Thousand Oaks: SAGE Publications Inc; 1989. p. 9–67.
Chernoff H, Moses LE. Utility and descriptive statistics. In: Elementary decision theory. New York: Dover Publishing, Inc; 1959. p. 79–118.
Dworkin G. Autonomy and informed consent. In: The theory and practice of autonomy. Cambridge: Cambridge University Press; 1988. p. 100–20.
Faden RR, Beauchamp TL, King NMP. The concept of autonomy. In: A history and theory of informed consent. New York: Oxford University Press; 1986. p. 235–73.
Gigerenzer G, Brighton H. Homo Heuristicus: why biased minds make better inferences. Top Cogn Sci. 2009;1:107–43.
Kahneman D, Slovic P, Tversky A. Judgment under uncertainty: heuristics and biases. New York: Cambridge University Press; 1982.
Hertwig R, Pleskac T. Decisions from experience: why small samples? Cognition. 2010;115:225–37.
Hertwig R. The psychology and rationality of decisions from experience. Synthese. 2012;187:269–92.
Botti S, Orfali K, Iyengar SS. Tragic choices: autonomy and emotional responses to medical decisions. J Consum Res. 2009;36(3):337–52 Available from: http://www.jstor.org/stable/10.1086/598969.
Andrade EB, Ariely D. The enduring impact of transient emotions on decision making. Organ Behav Hum Decis Process. 2009;109(1):1–8 Available from: http://www.sciencedirect.com/science/article/pii/S0749597809000211.
Druckman D, Olekalns M. Emotions in negotiation. Gr Decis Negot. 2007;17(1):1–11 Available from: http://link.springer.com/10.1007/s10726-007-9091-9. Cited 2014 Feb 3.
Wolfinger RE. Nondecisions and the study of local politics. Am Polit Sci Rev. 1971;65(5):1063–80.
Rauhut H, Winter F. A sociological perspective on measuring social norms by means of strategy method experiments. Soc Sci Res. 2010;39(6):1181–94 Available from: http://www.sciencedirect.com/science/article/pii/S0049089X10001250.
Hechter M, Opp K-D. Social norms. New York: Russel Sage Foundation; 2001.
Strauss A, Corbin J. Basics of qualitative research techniques for developing grounded theorq. 2nd ed. Thousand Oaks: SAGE Publications, Inc; 1998.
Charmaz K. Grounded theory in global perspective: reviews by international researchers. Qual Inq. 2014;20(9):1074–84.
Ottawa Hospital Research Institute. List of Patient Decision Aids Developers. 2019 Available from: https://decisionaid.ohri.ca/developers.html. Cited 2019 Feb 4
Mayo Clinic. Alzheimer’s: consider options for long-term care. Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1362. Cited 10 Feb 2018.
Healthwise. Low Back Pain: should I try Epidural Steroid Shots? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1420. Cited 5 Feb 2017.
Healthwise. Breast cancer risk: should i have a BRCA gene test? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1231. Cited 5 Feb 2017.
Agency for Healthcare Research and Quality. Reducing the risk of Cancer with medicine: a guide for women. Patient Decis Aid. 2010; Available from: https://www.ncbi.nlm.nih.gov/books/NBK51220/pdf/Bookshelf_NBK51220.pdf. Cited 5 Feb 2017.
National Breast and Ovarian Cancer Centre, University of Sydney. Understanding Ductal Carcinoma In situ (DCIS) and deciding about treatment. Patient Decis Aid. 2010; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1790. Cited 5 Feb 2017.
Healthwise. Dupuytren’s Disease: Should I Have Hand Surgery? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1052. Cited 5 Feb 2017.
Healthwise. Tennis elbow: Should I have surgery? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1082. Cited 5 Feb 2016.
Healthwise. Flu: Should I Take Antiviral Medicine? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1306. Cited 5 Feb 2017.
Option Grid Collaborative. Knee osteoarthritis: treatment options. Patient Decis Aid. 2011; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1541. Cited 5 Feb 2017.
Cochrane Musculoskeletal Group. Should I take risedronate (Actonel) for osteoporosis? Patient Decis Aid. 2011; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1410. Cited 5 Feb 2017.
Healthwise. Panic disorder: Should I take medicine? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1060. Cited 2017 Feb 5.
Healthwise. Prediabetes: Which Treatment Should I Use to Prevent Type 2 Diabetes? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1654. Cited 5 Feb 2017.
Healthwise. Prostate cancer: Should I choose active surveillance? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1563. Cited 5 Feb 2017.
Option Grid Collaborative. Prostate specific antigen (PSA) test: yes or no? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1682. Cited 5 Feb 2017.
Healthwise. Enlarged Prostate: Should I take Medicine? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1126. Cited 5 Feb 2017.
Option Grid Collaborative. Sore throat: antibiotics or not? Patient Decis Aid. 2014; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1556. Cited 5 Feb 2017.
Healthwise. Bed-wetting: should I do something about my child’s bed-wetting? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1013. Cited 2017 Feb 5.
Healthwise. Testicular Cancer: Which Treatment Should I Have for Stage I Nonseminoma Testicular Cancer After My Surgery? Patient Decis Aid. 2016; Available from: https://decisionaid.ohri.ca/Azsumm.php?ID=1018. Cited 5 Feb 2017.
Bekker HL, Winterbottom A, Fowler J, Volk RJ, Feldman-Stewart D, Butow P, et al. Using personal stories. In: Volk R, Llewellyn-Thomas HA, editors. 2012 update of the international patient decision aids standards (IPDAS) collaboration’s background document chapter E: International patient decision Airds Stadards collaboration; 2012. p. 26. Available from: http://ipdas.ohri.ca/IPDAS-Chapter-E.pdf.
Haidt J. The righteous mind. Why good people are divided by politics and religion. New York: Vintage Books; 2013.
Gillespie R. Health behaviour and the individual. In: Moon G, Gillespie R, editors. Society and health an introduction to social science for health professionals. London: Routledge; 1999. p. 99–110.
Mumford E. Medical sociology. Patients providers and policies. New York: Random House Inc; 1983.
Lindenberg S, Steg L. Normative, gain and hedonic goal frames guiding environmental behaviour. J Soc Issues. 2007;63(1):117–37.
Broke DW. Medical decisions at the end of life. In: Kuhse H, Singer P, editors. A companion to bioethics. 2nd ed. Malden: Blackwell Publishing; 2001. p. 231–41.
Dworking G. Autonomy and informed consent. In: The theory and practice of autonomy. Cambridge, New York: Cambridge University Press; 1988. p. 100–20.
I would like to give particular thanks to Prof. Rafael Wittek and Prof. Paul Milgrom for valuable discussions and suggestions on previous versions of this paper.
This research received no funding.
Ethics approval and consent to participate
Consent for publication
The author declares that she has no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.