Volume 13 Supplement 2
Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers
© Trevena et al; licensee BioMed Central Ltd. 2013
Published: 29 November 2013
Making evidence-based decisions often requires comparison of two or more options. Research-based evidence may exist which quantifies how likely the outcomes are for each option. Understanding these numeric estimates improves patients’ risk perception and leads to better informed decision making. This paper summarises current “best practices” in communication of evidence-based numeric outcomes for developers of patient decision aids (PtDAs) and other health communication tools.
An expert consensus group of fourteen researchers from North America, Europe, and Australasia identified eleven main issues in risk communication. Two experts for each issue wrote a “state of the art” summary of best evidence, drawing on the PtDA, health, psychological, and broader scientific literature. In addition, commonly used terms were defined and a set of guiding principles and key messages derived from the results.
The eleven key components of risk communication were: 1) Presenting the chance an event will occur; 2) Presenting changes in numeric outcomes; 3) Outcome estimates for test and screening decisions; 4) Numeric estimates in context and with evaluative labels; 5) Conveying uncertainty; 6) Visual formats; 7) Tailoring estimates; 8) Formats for understanding outcomes over time; 9) Narrative methods for conveying the chance of an event; 10) Important skills for understanding numerical estimates; and 11) Interactive web-based formats. Guiding principles from the evidence summaries advise that risk communication formats should reflect the task required of the user, should always define a relevant reference class (i.e., denominator) over time, should aim to use a consistent format throughout documents, should avoid “1 in x” formats and variable denominators, consider the magnitude of numbers used and the possibility of format bias, and should take into account the numeracy and graph literacy of the audience.
A substantial and rapidly expanding evidence base exists for risk communication. Developers of tools to facilitate evidence-based decision making should apply these principles to improve the quality of risk communication in practice.
Health decisions often require patients and clinicians to compare and choose among two or more options. The chosen path may bring with it a number of benefits and harms for the individual patient. Informing these choices with the best available evidence from scientific research is desirable and, where available, outcomes should be provided that have been quantified through research [1, 2]. For both written and verbal information, patients have a more accurate understanding of risk if probabilistic information is presented as numbers rather than words, even though some may prefer receiving words .
Patient decision aids (PtDAs) can be an effective strategy for integrating research evidence with patient values and other factors to facilitate greater patient involvement, improve decision quality, and increase knowledge about decision options . Sixteen out of the 86 trials in the Cochrane Collaboration review of PtDAs for treatment and screening decisions measured the effects of including numeric estimates on patients’ understanding . Presenting numeric estimates within a PtDA significantly improved the accuracy of risk comprehension (RR 1.7, 95% CI 1.5 to 2.1), compared with not receiving numeric estimates, with larger effect size when estimates were presented as numbers, versus describing them in words.
A theoretical rationale for evaluating patient decision aids on this quality standard
The argument outlined above serves, in effect, as a broad justification for considering “presenting quantitative information about decision outcomes in PtDAs” as an important quality standard when evaluating a PtDA. However, although the inclusion of numeric risk estimates in PtDAs appears to be an effective strategy for promoting informed decision making, many important questions about risk communication remain. Exactly how should numeric risk estimates be represented in order to maximize patient understanding? What are the essential elements of effective risk communication, and does empirical evidence support particular methods or “best practices” for representing and communicating numeric risk estimates using PtDAs?
While we were updating the International Patient Decision Aid Standards (IPDAS) Collaboration’s original Background Document, our working group developed a wide-ranging review of current knowledge in the field of risk communication, including relevant evidence from the broader science and social science literature. The purpose of this paper is to summarize our insights into the “state of the science” about the presentation of quantitative information about decision outcomes, and, in doing so, to focus on communication issues that are likely not only to have practical application in the development of PtDAs but also to inform the further development of the quality standards.
Since the literature on risk communication is so vast, this evidence summary was developed by expert consensus. Our focus was to provide clear guidance for PtDA developers regarding design issues for which substantial research evidence exists and to identify those design problems remaining to be resolved. Thus, since the purpose of this review was to provide guidance about risk communication generally, not to answer a focussed question, a systematic review was not feasible or appropriate.
The expert international working group was formed by inviting key authors in the field. The fourteen individuals who agreed to participate were drawn from North America, Europe and Australasia. Several had participated in writing the first version of this material in the IPDAS Collaboration’s 2005 Background Document.. As a first step, members of the group identified major issues in communicating quantitative information for PtDA development. This was achieved through an iterative and interactive online discussion process and drew heavily upon the combined expertise of the authors.
Each member of the working group assisted in drafting at least two of the issue-focused sections and worked closely with at least one other author in this task. Each team drew upon their collective expertise to define current best practices for communicating probabilities in PtDAs and to provide illustrative research findings in support of their recommendations.
Terms and definitions in risk communication
Simple frequency format
Expresses the event rate as an integer with an appropriate denominator (e.g. x in 100)
Simple percentage format
Expresses the event rate as a percentage (e.g. x%)
Natural frequency format
The term ‘natural frequencies’ was proposed for estimating the probability arising from a joint occurrence of events (e.g. the probability of having breast cancer given an abnormal mammography result). Natural frequencies preserve the base rate of the outcome (e.g. breast cancer) and report the ‘actual’ or ‘natural’ number of people having a particular outcome (e.g. having a positive test result). An example would be “Out of every 10,000 people, 30 have colorectal cancer. Of these, 15 will have a positive haemoccult test. Out of the remaining 9970 people without colorectal cancer, 300 will still test positive. How many of those who test positive actually have colorectal cancer? Answer: 15 out of 315”
An alternative representation of this information is the conditional probability format. For example: “The probability of having colorectal cancer is .003%. Of people who have the cancer, 50% get a positive test result. Of people who do not have cancer, 3% will nevertheless test positive. What is the probability that a person who tests positive has colorectal cancer? Answer: 4.8%”.
Infers the post-probability of outcome from the prior probability and a likelihood function.
Tailored health communication
Refers to providing information to a person based on characteristics that are unique to that person. It is assumed that tailored messages are perceived as more relevant to an individual and are therefore better processed and understood. Tailoring information using an individual’s specific risk factors might likewise increase people’s involvement with the information and lead to a better understanding.
It is concerned with the randomness or indeterminacy of future events.
On the other hand, this is the lack of knowledge needed to predict future outcomes, also known as “ambiguity” and is concerned with the lack of reliability, credibility, or adequacy of risk information. A primary example is imprecision in risk estimates which are typically expressed by confidence intervals.
Pictographs (sometimes called icon arrays)
They are visual graphic display formats which aim to represent the size of both the numerator and denominator in the one diagram. In other words, they show the part-whole relationship. Examples include systematic ovals, 100 face or human figure diagrams and displays where event icons are scattered rather than grouped.
It is the ability to understand and apply mathematical concepts.
Stories, also called testimonials, about individuals’ experiences or health outcomes, usually told from a first-person perspective.
We identified eleven major risk communication issues involved in the presentation of probabilities and related risk information: 1) Presenting the chance an event will occur; 2) Presenting changes in numeric outcomes; 3) Outcome estimates for test and screening decisions; 4) Numeric estimates in context and with evaluative labels; 5) Conveying uncertainty; 6) Visual formats; 7) Tailoring estimates; 8) Formats for understanding outcomes over time; 9) Narrative methods for conveying the chance of an event; 10) Important skills for understanding numerical estimates; and 11) Interactive web-based formats.
Presenting the chance an event will occur
Suitable formats for presenting numeric chances depend on the nature of the task . When the task is to present the chance of a single event, simple frequency formats that include a number and time interval (such as “Every year 10 in 100 people with pre-diabetes develop diabetes”), or simple percentage formats (such as “Every year 10% of those with pre-diabetes will develop diabetes”) are more transparent than formats such as “The chance of developing diabetes is 10%”. The last statement is problematic because it does not specify the “denominator” (i.e., the reference class, as in, for example, “10% of all pre-diabetics in one year”). Without a clear description of to whom this estimate refers, people might impose their own erroneous interpretations such as “I only have a 10% chance of developing diabetes in my lifetime” . Similarly, when patients who take fluoxetine for mild depression hear from their doctor that there is a “30-50% chance of developing a sexual problem such as impotence or loss of sexual interest,” some may think this means they will have problems in 30% of their own sexual encounters. The “denominator” or reference class used by the doctor is “patients on fluoxetine”, but the denominator used by the patient is “their own sexual encounters” .
There is also some evidence that risks presented in simple frequencies are perceived as higher than when they are presented in their equivalent percentage value, especially in patients with lower numeracy  and (possibly) when smaller percentages are presented . Given this potential format bias, one should be careful when comparing results of studies that have used different formats (percentages or simple frequencies). Formats should aim to be consistent throughout a PtDA (see below). Providing simple frequency AND percentage appears to add no advantage , and there is strong evidence that “1 in x” formats with variable denominators are more difficult to understand and elevate risk perceptions[10, 11]. They should be avoided for all tasks.
1. In summary, it is most important when presenting the chance of a single event to clearly define the denominator or reference class over time. Percent or simple frequency formats can be used for presenting the chance of a single event. However, in deciding which one to use, consider what other information needs to be presented and what the purpose of the PtDA is, overall, so there is format consistency throughout. Visual formats may also help to reduce bias (see Formats for understanding outcomes over time, below).
When the task is to compare the chance of occurrence of two or more independent events (e.g., the chance of symptom relief with drug A compared with placebo), formats that express the chance of an event using one number, such as percentages, work better than simple frequencies involving more than one number, such as 1 in 100 . If using simple frequencies such as 1 in 100, one should use the same denominator (e.g., 1 in 100 versus. 2 in 100) as these are easier to compare than frequencies using different denominators (e.g., 1 in 100 versus. 1 in 50) [8, 10]. Consistent denominators should always be used. When choosing the size of the denominator, smaller numbers (e.g., 100) are easier to understand and remember than larger numbers (e.g., 10,000) . There has been discussion about whether people find percentages less than one (e.g., 0.1%) more difficult to understand than the equivalent simple frequency (e.g., 1 in 1000) [9, 10]. However, this problem may reflect difficulty manipulating decimal points (e.g., asking someone to represent 1 in 1000 as a percentage) rather than a comprehension problem .
In summary, percentages (e.g., x %) may have an advantage over a simple frequency format (e.g., x in 100) for comparing the chance of occurrence of two or more independent events. As mentioned before, it remains important to clearly define the denominator or ‘reference class’ and to aim for a consistent format throughout the PtDA taking into account the information and tasks required.
However, other formats are more suitable for tasks that involve presenting changes in numeric outcomes (see Presenting changes in numeric outcomes, below) and conveying the frequency of joint occurrences of two or more dependent events, such as the conditional probability that a person with a positive test result has the disease (e.g., see Outcome estimates for test and screening decisions, below).
Presenting changes in numeric outcomes
Most efforts to communicate changes due to interventions (i.e., treatment effects) or across time (e.g., improvements of health) use either side-by-side total risk presentations or difference presentations. Difference presentations depict the change in risk and can influence risk perceptions through framing effects. Research has shown that relative risk presentations (e.g., “30% lower risk”) tend to magnify risk perceptions and decrease understanding, compared to absolute risk presentations (e.g., “the risk is lower by 5 percentage points”)[13, 14]. Number needed to treat (NNT) is sometimes used, but several studies suggest that this format is poorly understood by patients and may increase the perceived effect of treatment .
A variant is to present incremental risk (absolute risk increase) after a “baseline” total risk level has been shown. This approach emphasizes the size of the change relative to the size of the total risk, and was shown to lower risk perceptions . Such language (e.g., “5 more women get…”) was incorporated into the Schwartz et al. drug facts box , and used in decision tools such as Adjuvant! A PtDA trial suggested that incremental risk language works best when accompanied by visual displays . In particular, when the baseline risks are small, relative risk reductions are perceived to be larger than absolute risk reductions [13, 19, 20].
The framing of outcomes in terms of losses or gains has been shown to affect people’s choices . Framing outcomes in terms of potential gains (e.g., the chances of survival) often generates risk-averse choices, whereas framing outcomes in terms of potential losses (e.g., the chances of death) often generates risk-seeking choices. In clinical situations, the effects of the framing of outcomes as losses or gains tend to vary across situations ; the variable effect of different frames of risks or rates is due to emphasizing different aspects of the information.
In summary, when presenting changes in rates, absolute risks should be given either in percentages or simple frequencies, and if possible along with the absolute risk increase (or decrease). If frequencies are used, the denominators should be equal.
Outcome estimates for test and screening decisions
A number of studies and a recent Cochrane Collaboration review [13, 23] have shown that natural frequencies are better than conditional probabilities where events are connected. It is unclear whether people use Bayesian reasoning when making screening decisions, but natural frequency formats are still proposed as the best way to help people understand these kinds of estimates [24, 25]. So, if a PtDA requires people to calculate the probabilities associated with jointly occurring events, then a natural frequency format would be preferable to conditional probabilities.
However, screening can also be viewed as an “intervention” that has an effect (e.g., reducing death from colorectal cancer). Rates of death from a particular cancer with and without screening are actually the chances of independent events. As noted earlier, there may be some advantage to presenting such information in a percentage format, but cancer incidence and mortality rates are usually low in the general population and possible format biases due to small numbers need to be considered (see Presenting the chance an event will occur, above). Similarly, the chance of having a disease if your test result is positive can be thought of as the “post-test probability”, and some would suggest this could be calculated on behalf of the patient and presented in a percentage (1%) or simple frequency format (e.g., 1 in 100). Thus, we recommend that PtDA developers consider both the nature of the task required and the other information that needs to be conveyed in the same document. It is important to clarify what the reference class is (e.g., women aged 50 who are having biennial mammography over 10 years) and to keep the denominator constant. Once again, “1 in x” formats should be avoided as they consistently perform worse.
The current IPDAS criteria recommend that screening PtDAs include estimates of: 1) disease with and without screening; 2) false positives; and 3) false negatives. The updated Cochrane Collaboration review of PtDAs includes 34 trials about screening and test decisions . Five of these trials measured the accuracy of risk perception [27–31]. Four of these reported significantly improved risk perception [27–30] regardless of whether accurate risk comprehension was measured as numbers [27, 28] or as gist-based risk comprehension in words [29, 30]. Four of the trials also included quantitative estimates in accordance with the IPDAS criteria recommendations. Three of the PtDAs were available and all used different formats for numerical outcomes. None provided a head-to-head comparison of formats. Given the lack of head-to-head format comparison in these trials, we recommend applying the principles outlined in this chapter which are based (where possible) on comparative research.
Our review confirms that, in screening PtDAs, the application of IPDAS criteria about the presentation of quantitative estimates of screening outcomes improves the accuracy of risk perceptions.
Numeric estimates in context and with evaluative labels
To help users get perspective on the risk of disease, PtDA developers should consider including contextual information when feasible. Context is particularly important for PtDAs about disease prevention or cancer screening, in which the benefit is a reduction in disease specific mortality. One way to provide context is to provide the chance of death over the next 10 years from the disease under consideration (where possible according to age, smoking status, and other reliable risk factor information), as well as the chance of dying from other major causes and from all causes combined .
Directly interpreting the meaning of numeric information (e.g., telling patients how good or bad a 9% risk is) can also have a substantial influence on how patients use that information. This is because patients often do not understand the meaning of unfamiliar numbers without additional help, and, without meaning, information tends not to be used in subsequent decision making. In one series of studies, providing evaluative labels for numeric quality-of-care information (e.g., telling decision makers that the numbers represented “poor” or “excellent” quality of care) resulted in greater use of this information in judgments and less reliance on an irrelevant emotional state among the less numerate . In another study, evaluative labels for test results (that a patient’s test was “positive” or “abnormal”) induced larger changes to risk perceptions and behavioural intentions than did numeric results alone . The appropriateness of these changes, however, can be unclear in health contexts, and evaluative labels should be applied carefully.
Numeric risk estimates ultimately represent evidence-based, mathematical expressions of uncertainty about the future. The uncertainty expressed by risk estimates can be divided into two principal types: 1) “aleatory” or first-order uncertainty that reflects the randomness or indeterminacy of future events, and 2) “epistemic” or second-order uncertainty, otherwise known as “ambiguity,” that reflects limitations in the reliability, credibility, or adequacy of risk information. An understanding of each of these uncertainties is arguably essential for informed decision making. However, the optimal methods and outcomes of communicating these uncertainties to patients are only beginning to be understood .
The communication of aleatory uncertainty has been examined in a small number of studies for both textual and novel visual methods of representing randomness in decision support interventions (e.g., icon arrays displaying affected individuals in a scattered rather than clustered manner) [36–40]. Available evidence suggests that these methods have no significant effect on risk perceptions, although evidence is lacking regarding their effects on patients’ understanding of uncertainty. In one study, however, the communication of randomness was associated with greater subjective uncertainty about estimated risk . The communication of epistemic uncertainty has been examined in a small number of studies using confidence intervals to communicate ambiguity in probability estimates. These studies have shown that communicating ambiguity has little effect on risk perceptions, although it increases patient worry [41, 42], and these effects appear to be moderated by representational method (visual vs. textual) and individual differences (e.g., dispositional optimism) [39, 41]. Evidence is limited and mixed regarding the extent to which confidence intervals are understood by patients [43, 44] and how they influence perceptions of the credibility of probability estimates [38, 45]. Furthermore, the effects of communicating both epistemic and aleatory uncertainty on real medical decisions have not yet been evaluated.
The communication of ambiguity has been evaluated more fully outside health care. Numerous studies in behavioural decision research have shown that ambiguity leads to avoidance of decision making and pessimistic risk perceptions and affective responses (worry, distress) related to choice outcomes—a phenomenon known as “ambiguity aversion” [46–49]. However, most studies have examined hypothetical rather than real decisions.
In summary, evidence on the optimal methods and outcomes of conveying uncertainty is limited but growing. Novel representational methods have been developed to communicate both randomness (aleatory uncertainty) and ambiguity (epistemic uncertainty); these methods may be useful to incorporate in PtDAs along with estimates of risk magnitude. However, the communication of uncertainty can be psychologically aversive. More research is needed to determine both the optimal representational methods and effects of communicating uncertainty on patient perceptions, understanding, and decision making.
Presenting event rates with visual aids such as pictographs (also called icon arrays), bar charts, or flow diagrams may aid accurate understanding of probabilities. Visual displays can help reduce several biases, such as denominator neglect , framing effects [51, 52], and the undue influence of anecdotes . They also can aid the comprehension of more complicated concepts such as incremental risk . Graphs that clarify sub-set relationships (e.g., Venn diagrams, Euler circles) can lead to better judgements, for instance in Bayesian reasoning tasks [54, 55]. Others believe graphs help, but for different reasons . However, there has been some evidence that graphs can affect peoples’ tendencies to overestimate low probabilities and underestimate high probabilities – the magnifier effect . Others have shown the opposite effect (i.e., less overestimation) on low probabilities and no effect on high .
Although the use of visual displays is often recommended as an aid to interpretation for numerical data [59, 60], one important caveat is that people vary in their ability to extract data and meaning from visual displays. Galesic & Garcia-Retamero developed a graph literacy scale that predicts who actually profits from visual displays [61, 62]. For example, visual displays are helpful for understanding statistical information about health for people with low numeracy [63, 64] yet people who lack graph literacy may be better off with just numbers .
Graphs have sometimes been shown to be better able to convey the essential aspects of the information (i.e., “gross-level information”) , bottom line meaning, or gist , whereas numerical representations may better convey more precise aspects of the information (i.e., “detailed-level information” or verbatim) [66, 68]. Thus, a potential weakness of visual displays is that people may focus more on the pattern of data rather than the precise values, unless that is the main objective. Furthermore, some graphs are better suited for certain tasks (e.g., line graphs for trends over time, bar graphs for comparison across groups) [69, 70].
Graph type and formatting have an effect on comprehension and behaviour. For instance, one study showed that the formats that are perceived most accurately and easily by patients are vertical bars, horizontal bars, and pictographs. Preferred graphs do not necessarily lead to better performance than non-preferred graphs. Furthermore, pie charts and pictographs with randomly-distributed ovals lead to slower and less accurate estimates . Another study showed that different visual formats supported gist versus verbatim knowledge . Enhancing accuracy in estimates can be aided by displaying only the most crucial elements [71, 72], as well as by using icon arrays (blocks or stick figures) that are arranged as groups in a block rather than being scattered randomly -- the latter of which is useful to convey the concept that events (e.g., who is afflicted by disease) occur at random .
Finally, it has been shown that visual aids are most effective for comprehension when the entire population at risk is shown rather than only depicting sick people, for instance . In addition, for conveying small probability events (e.g., less than 1%), graphical displays (e.g., bar charts) that show only the number of people affected (i.e., foreground information) leads to greater risk aversion (e.g., greater willingness to pay for an improved product) than graphic displays that show part-whole relationship by including the total population or those not affected) (i.e., background) [74, 75]. It is still unclear whether icon-arrays (which show the part-whole relationship by including the full denominator) result in more accurate risk perception than icon-based displays (which do not have the full denominator).
A recent study in adults with lower education and literacy  found that numerator size was an important factor when presenting the changes in numeric outcomes for events out of 1000. Where the outcome is <100/1000, pictographs were better understood and processed more quickly than bar charts, particularly if the difference between event rates was small. These 1000 denominator pictographs stacked the icons along the long axis beside a referent scale to support easier reading. However, for more common outcomes ( >100/1000), bar charts were better, possibly because the icon arrangement was more complicated. In addition, the role of shading in processing the part-to-whole relationship of icon arrays is still not well understood.
In summary, visual displays can be a powerful tool to convey health-related statistical information, especially for people with higher graphical literacy and among those who have problems with understanding and applying numbers. However, some caution is warranted as visual displays may not be intuitively understood by everyone. They can be used to represent statistical information transparently, but they can also be misused and misrepresent statistical information . Overall, all visual aids should be pilot tested for understanding (not simply preferences), and developers should take care to avoid using misleading images (such as graphs with misleading scales) or using different scales within the same PtDA. Finally, the field still needs a more systematic theoretical understanding of why, when, and for whom visual displays are effective . Such theories could help to translate the growing research in graph cognition and design [78, 79] into practical advice for risk communicators.
To date, the effects of tailoring health risk information on improving health decision making appear mixed. Limitations in research quality and heterogeneity in outcome measures make drawing firm conclusions about effective strategies difficult. A meta-analytic review showed that tailored print messages about health have been effective in stimulating health behaviour change, but the effect size is small and depends on the variable that is used for tailoring . The effect of tailoring was modified by the type, visual layout and length of the printed material, type of behaviour (more effective for preventive behaviours) and by demographic factors [80, 81]. Tailored print messages have been shown to increase uptake of mammography screening  and pap testing [80, 81]. A review by Albada et al. showed that information tailored to an individual’s risk factors increased risk perception and resulted in better knowledge compared to generic information .
Results also are mixed with respect to the effect of tailored health messages on behaviour—for example, on cancer screening. Tailoring by behavioural constructs seems to be effective, while there was limited evidence of the effectiveness of information tailored by risk factors only, in particular for cancer screening. Bodurtha et al. also found that a ‘brief (tailored) intervention’ regarding mammography adherence did not change behaviour . No significant differences existed in mammography intentions, actual uptake, clinical breast examination, or self-examination between intervention and control study arms. However, among those who were most worried, mammography rates in the intervention group were higher. Thus, individual characteristics, such as worry about breast cancer and educational status, may modify the effects of tailored health messages.
Because most studies on tailoring health risk information were done for cancer screening, not much is known about the effect for other decisions. More insight is needed into why personalized risk messages might be better understood and whether they are relevant for other kinds of health decisions.
Formats for understanding outcomes over time
Choices of how to display long term outcomes to improve understanding of risk are challenged by the difficulty in obtaining accurate relevant long-term outcome estimates of benefit and risk . Randomized controlled trials and systematic reviews usually represent a few years of follow up at most. Yet to make an informed decision, patients and physicians are often interested in longer term outcomes. Observational studies can provide longer-term data but are prone to selection bias and confounding. An additional bias is the tendency for trials to aggregate short and long term mortality which leads to inaccurate estimates if hazard ratios are not constant over time . These methodological problems are beginning to be addressed by newer risk modeling approaches [86–89].
When data are available, formats used to improve patient understanding of outcomes over time include: (a) the chance of a specific outcome at a single point in the future; (b) chance of an outcome at multiple points in the future; (c) mortality or survival graphs showing risks over time; (d) cumulative future or lifetime chance of an outcome; and (e) rate of occurrence of an outcome that is likely constant over time.
Showing the chance of a specific outcome at a single point in the future has the advantage of simplicity of presentation and calculation from available randomized trials or cohort studies. Examples of this approach are the 10 year risk of cardiovascular disease used in estimates of risk and benefit of cholesterol medications  and the risk in 3-5 years of precancerous changes on pap smear or genital warts related to HPV vaccine . This method has also been used with multiple points in the future. Examples include presenting the risk of having to have repeat bypass surgery at 5 years and 10 years after the initial procedure , and expected deaths after lung transplantation for cystic fibroses shown at 1 month, 1 year, 3 years, 5 years, and 10 years .
Survival and mortality graphs are commonly used in presenting research studies and have been used to relay information to patients. However, patients’ interpretation of these graphs may be susceptible to various biases. When web-users were shown survival graphs for a hypothetical disease and treatment, they based their perceptions of treatment effectiveness on visual differences in these graphs . When a longer duration of data was shown, people perceived larger differences in risk even when the magnitude of risk reduction was identical. Mortality graphs may be more temporally consistent , but less well understood by patients . Given these findings and current limitations in evidence, a balanced approach using both survival and mortality graphs may be prudent until more information is available . A study presenting treatment options for esophageal cancer showed most patients understood graphical representations of even complex multidimensional patient-reported outcomes .
Another common format for representing outcomes over time are estimates of the cumulative chance of an event during a whole lifetime, although these can be difficult for people to understand . Nevertheless, this method is commonly used—e.g., in describing cancer risk in patients with BRCA gene mutations . People are also often shown the cumulative chance of an event over a certain period of time into the future (e.g., 10 years); for example osteoporosis treatment  and hormone replacement therapy in menopause . Cumulative risk over time is also used in PtDAs without an explicit endpoint when describing probabilities of outcomes after a specific event or intervention. Examples include comparisons of outcomes of Achilles tendon rupture with and without surgery , and of cardiac resynchronization therapy in heart failure . Rates are also used in conditions likely to have a relatively constant risk over time. An example is birth control and the annual risk of pregnancy with a specific method .
Although PtDAs providing quantitative risk information have been shown to increase accuracy of risk perceptions  and to promote knowledge and agreement between values and choices , there are no trials examining different formats for representing the risk of outcomes over time.
Narrative methods for conveying the chance of an event
Individual patient narratives (i.e., stories of what one person experienced or did, which are sometimes referred to as “patient testimonials”) are sometimes provided in PtDAs as a complement to statistical information about risks. These stories can provide rich detail about what it is like to experience a health condition or a particular outcome of treatment. However, the provision of narratives has been shown to influence perceived vaccination risk and intentions . In one study, narratives decreased the perceived chance of adverse events but increased the perceived severity of adverse events. Narratives also influenced vaccination intention even after controlling for the perception of vaccine riskiness. In the same study, the nature of the presented information (emotionality, richness) was also varied to assess the impact on risk perception and showed that the highly emotional narratives had a greater impact on perceived risk although the richness of the narratives did not.
Other studies have shown that patient testimonials influence treatment choices. In one study, participants receiving a disproportionate number of negative testimonials for surgery were less likely to choose surgery compared to participants receiving equally positive and negative examples for surgery . In Ubel’s second study, participants receiving no testimonials were most likely to choose bypass surgery (58%), compared to those receiving a proportionate number of testimonials (37%) and those receiving a disproportionate number of testimonials (34%). In this case, the testimonials significantly reduced the choice of the most effective but invasive and risky intervention.
Another study tested whether the use of a quiz or pictograph lessened an individual’s reliance on anecdotal evidence for angina treatment (bypass surgery or balloon angioplasty) . They found that, when statistical information was reinforced with pictographs and quizzes, anecdotes had no significant effect on treatment decisions. The same authors also found pictographs were the active ingredient that lessened the effect of anecdotes. This finding would argue for avoiding narratives without statistical information.
In summary, using narratives to present benefit and risk information may increase perceptions of risk severity, decrease the ability to accurately recall risk probabilities, and influence treatment choice. The relative number of narratives used can also influence decision making. However, narratives vary in their purpose, content, and emotional balance, and some types of narratives may be more or less likely to bias risk perceptions than others. For example, in principle, narratives that report outcomes should influence likelihood perceptions more than those which describe experiences or decision processes . Because of the potential that narratives might have unintended effects on risk communications, we suggest (a) that narratives should be used with caution until research better clarifies their effects, both positive and negative, and (b) that developers be more cautious about using narratives when attempting to present unbiased information for informed decision making than when attempting to be persuasive and promote behaviour change. If narratives are used to present benefit and risk information, they should be accompanied by statistical information in pictograph form. Graphical representations of risk may reduce the effect of narratives. However, it seems likely that information included in narratives is likely to influence the ways individuals either search for and/or process information, which may make them either useful or counterproductive when included in interventions designed to facilitate good decision making.
Important skills for understanding numerical estimates
Numeracy is the ability to understand and apply mathematical concepts. It can have considerable effect on the use and interpretation of numerical estimates. Higher numeracy can facilitate computations, the interpretation of numbers, information seeking, depth of processing, memory for numerical information, and trust in numerical formats, leading to improved risk comparisons, risk estimates, and value elicitations [33, 109–113]. Lower numeracy is associated with overestimation of risk probabilities [114, 115], higher susceptibility to factors other than numerical data (e.g., framing, mood states, labels used to interpret quantitative results and feedback from others) [33, 116], insensitivity to variations in risk magnitude (especially when multiple numbers are shown at once) , and greater denominator neglect [64, 118]. In particular, people with lower numeracy are less likely to derive affective meaning from numbers and are more influenced by affective considerations from non-numerical aspects of the task .
Levels of numeracy in the general population are relatively low. For example, a national survey showed that one-quarter of the general US population cannot say whether 1 in 10, 1 in 100, or 1 in 1000 represents the largest risk of getting a disease; 30% cannot transform 20 in 100 to a percentage; 40% cannot say what is 1% out of 1000; and about 75% cannot transform 1 in 1000 to a percentage . Greater numeracy is associated with higher education, younger age, and male gender [109, 110], although none of these characteristics guarantee high numeracy. For example, even highly educated people can be less numerate  and research has demonstrated that numeracy can be more predictive of comprehension and decisions than education and other demographic variables [122–125].
Numeracy is relatively distinct from other aspects of health literacy, general intelligence, and working memory [122, 126, 127] and can be acquired through instruction and deliberative practice . Educational systems that focus on mathematics and science education from an early age may contribute to higher numeracy levels in the general population [120, 129]. Short interventions aimed at improving patients’ numeracy and consequently their risk understanding might be useful, but, at present, research regarding the feasibility or efficacy of such interventions is lacking.
Higher numeracy, however, does not preclude the need for well-designed PtDAs. Patients need at least basic numeracy but the communicators need to be aware of and use an evidence-based approach to present information in ways that are simple and clear. Even the highly numerate can misunderstand inconsistent or difficult information formats such as those involving unequal denominators, unspecified reference classes, or conditional probabilities. Furthermore, situational factors that are often present in medical decision making, such as time pressure and high levels of stress, can make the processing of risk related information difficult even for highly numerate people . In addition, as individuals age, their numeracy levels tend to decline [110, 130]; this is important because older adults tend to make a disproportionate number of medical decisions relative to healthier younger adult populations. Finally, more numerate individuals may be more susceptible to some biases than the less numerate (e.g., evaluating a possible gain as more attractive when put into a context of a possible small loss than when presented alone [116, 131]. As a result, numeracy will usually help but sometimes may hinder patients’ abilities to use the most important dimensions in a decision. Unintended consequences of higher numeracy have not been considered yet in health contexts, but, ultimately, may be necessary to understand.
When designing a PtDA, understanding the abilities of prospective users can help in designing presentation formats that will maximize comprehension and use of important information in PtDAs. For example, among those with lower numeracy and sufficient graph literacy, visual displays can improve understanding [62, 65, 132, 133]. A number of measures of patients’ abilities relevant for understanding of risks have been developed in recent years (e.g., objective numeracy [121, 134, 126, 135, 136]; subjective numeracy ; graph literacy ; other aspects of the health literacy [138, 139]. Unfortunately, with few exceptions [71, 111], studies of how numeracy and graph literacy influence use and interpretation of numerical data in PtDAs are lacking, and hence sorely needed. In the absence of such data, it is difficult to assess whether tailoring PtDAs to patient’s abilities would result in any improved outcomes. Given that assessing patients’ numeracy and graph literacy in busy clinic settings may be difficult, it may be preferable to design PtDAs that benefit patients across the whole continuum of abilities, and to teach medical or other personnel to help patients make sense of the information.
Interactive web-based formats
The increasing prevalence of computers, tablets, and mobile devices creates new opportunities for interactive, web-based formats for communicating probability information. The literature in this area is sparse, and we are aware of no published studies that have examined use of such tools in actual PtDAs. Several experimental studies suggest, however, that web-based formats offer both opportunities and pitfalls. For example, in one study, participants presented with a treatment scenario were better calibrated in their perceptions of medication side effects when they created a bar graph of the risk instead of just viewing one . Another study found that a web-based, game-like, interactive risk graphic in which participants clicked in a matrix until they uncovered a risk event had the effect of reducing disparities in risk perceptions between more and less numerate participants . Such exercises could be seen as methods to increase patients’ active processing of risk information, which may lead to improved risk understanding. Indeed, the game-like interactive task elicited stronger emotional responses .
However, there are also considerable grounds for concern about interactive risk graphics. In 2002, Tversky, Morrison, and Betrancourt reviewed the literature on animated graphics of all types and noted that “the research on the efficacy of animated over static graphics is not encouraging” . More recently, research participants who used an interactive pictograph applet to visually graph provided risk numbers had significantly worse knowledge and made poorer decisions than participants who viewed static graphs .
Even without interactivity, animated graphics can use motion cues to reinforce gist messages. However, the evidence here is also mixed. One study found a dynamic scattered icon display increased recipients’ subjective uncertainty about a risk . Another study tested various types of animation in both grouped and scattered icon displays and found that they failed to improve participants’ ability to identify a dominant treatment option and sometimes significantly impeded performance .
In short, interactive web-based risk communication formats allow educators to use additional cues in risk communications. However, evidence is lacking to determine whether the techniques allowed by new technologies provide a net positive experience. Preliminary evidence suggests that, unless the motion cues reinforce the most critical gist message (e.g., the accumulation of risk over time), there remains significant risk that interactive or animated formats may degrade knowledge versus evidence-based static formats.
The science around risk communication is expanding rapidly, and there is good evidence that patients have a better understanding of risk if outcomes are presented as numbers. Yet there is an emerging awareness that how risk information is provided can improve people’s understanding or bias their risk perceptions. We highlight in Table S1 (Additional file 1) our key messages regarding each of the eleven major risk communication issues involved in the presentation of probabilities and related risk information.
For example, it is increasingly clear that different formats work better for different tasks, and PtDA developers should consider what cognitive tasks are required for each decision. Our review also makes it clear that it is important to include contextual information and to consider the numeracy and graph literacy skills of the PtDA users. Some format biases can be reduced by concurrent use of visual formats such as graphs and there are some formats which should be avoided as they consistently perform badly (e.g., “1 in x”).
This primer of risk communication for PtDA developers aims to provide a set of guiding principles based on the current evidence-base in this field. However, one of the challenges for PtDA developers is keeping up with this rapidly changing evidence base. The ongoing commitment to updates within the IPDAS Collaboration’s standards development process is one promising way of maintaining clear guidance regarding evidence-based communication approaches, but the IPDAS Collaboration faces the same sustainability challenges faced by many other international collaborative groups: it is always harder to keep a guidance document up to date than it is to create one in the first place. Nevertheless, the key approaches outlined in this paper should provide a “roadmap” for good risk communication in PtDAs and, indeed, in clinical practice more generally.
Given the fast pace of knowledge acquisition in this area, it is often difficult to identify which practices are “best.” Yet, the same knowledge base is increasingly able to identify communication practices which are known to cause problems and biases. We urge all PtDA developers and health practitioners who must communicate probabilistic risk information to patients to become aware of these adverse practices and to make sure they are not carried forward simply because of familiarity or past use.
Our work has highlighted some important gaps in the evidence base for risk communication, particularly in tailoring information, formats for conveying outcomes over time, conveying uncertainty, and the impact of interactive web-based formats. Other under-developed areas of research evidence include optimal formats for people with lower numeracy and graph literacy, and risk perceptions in different cultural contexts. Addressing these needs will require not merely targeted research efforts from selected investigators, but also a broad willingness of PtDA developers to embed experimental tests of different communication designs in their products. Only through a process of systematic, widespread investigation of risk communication methods and broad dissemination of the subsequent results can we hope to ensure that future PtDAs will present probabilistic information to patients in ways that are truly meaningful and useful to them as they make their medical decisions.
Presenting numeric outcomes of decisions is an important component of good PtDA design and improves patients’ accuracy of risk perception. Good evidence exists for strategies that can improve risk communication. Our guiding principles for numeric estimates aim to assist PtDA developers to create better tools for informed decision making. Adherence to the basic principles outlined herein should be considered a quality standard for PtDAs and related risk communication materials.
Publication of this supplement was partially funded by an unrestricted grant from the Informed Medical Decisions Foundation. Lyndal Trevena contributed the remaining funds. Administrative and editorial support for this supplement was also provided by The University of Texas MD Anderson Cancer Center.
This article has been published as part of BMC Medical Informatics and Decision Making Volume 13 Supplement 2, 2013: The International Patient Decision Aid Standards (IPDAS) Collaboration’s Quality Dimensions: Theoretical Rationales, Current Evidence, and Emerging Issues. The full contents of the supplement are available online at http://www.biomedcentral.com/bmcmedinformdecismak/supplements/13/S2.
- Sackett D, Rosenberg W, Muir Gray J, Haynes R, Richardson W: Evidence based medicine: what it is and what it isn't. BMJ. 1996, 312: 71-72. 10.1136/bmj.312.7023.71.PubMed CentralPubMedGoogle Scholar
- Haynes R, Devereaux P, Guyatt G: Clinical expertise in the era of evidence-based medicine and patient choice. Evid Based Med. 2002, 7: 36-38. 10.1136/ebm.7.2.36.Google Scholar
- Trevena LJ, Davey HM, Barratt A, Butow P, Caldwell P: A systematic review on communicating with patients about evidence. J Eval Clin Pract. 2006, 12 (1): 13-23. 10.1111/j.1365-2753.2005.00596.x.PubMedGoogle Scholar
- Stacey D, Bennett CL, Barry MJ, Col NF, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Legare F, Thomson R: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011, CD001431-10
- Gigerenzer G: What are natural frequencies?. BMJ. 2011, 343: 10.1136/bmj.d6386.Google Scholar
- Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz L, Woloshin S: Helping doctors and patients make sense of health statistics. Psychol Sci Public Interest. 2007, 8 (2): 53-96.PubMedGoogle Scholar
- Gigerenzer G, Galesic M: Why do single event probabilities confuse patients? Statements of frequency are better for communicating risk. BMJ. 2012, 344: e245-10.1136/bmj.e245.PubMedGoogle Scholar
- Peters E, Hart PS, Fraenkel L: Informing Patients: the Influence of Numeracy, Framing and Format of Side Effect information on Risk Perceptions. Med Decis Making. 2011, 31 (3): 432-436. 10.1177/0272989X10391672.PubMedGoogle Scholar
- Woloshin S, Schwartz L: Communicating data about the benefits and harms of treatment. Ann Intern Med. 2011, 155 (2): 87-96. 10.7326/0003-4819-155-2-201107190-00004.PubMedGoogle Scholar
- Cuite C, Weinstein N, Emmons K, Colditz G: A test of numeric formats for communicating risk probabilities. Med Decis Making. 2008, 28 (3): 377-384. 10.1177/0272989X08315246.PubMedGoogle Scholar
- Pighin S, Savadori L, Barilli E, Cremonesi L, Ferrari M, Bonnefon J-F: The 1-in-X effect on the subjective assessment of medical probabilities. Med Decis Making. 2011, 31 (5): 721-729. 10.1177/0272989X11403490.PubMedGoogle Scholar
- Garcia-Retamero R, Galesic M: Using plausible group sizes to communicate information about medical risks. Patient Educ Couns. 2011, 84 (2): 245-250. 10.1016/j.pec.2010.07.027.PubMedGoogle Scholar
- Akl E, Oxman A, Herrin J, Vist G, Terrenato I, Sperati F, Costiniuk C, Blank D, Schünemann H: Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Syst Rev. 2011, 3: CD006776-PubMedGoogle Scholar
- Covey J: A meta-analysis of the effects of presenting treatment benefits in different formats. Med Decis Making. 2007, 27 (5): 638-654. 10.1177/0272989X07306783.PubMedGoogle Scholar
- Halvorsen P, Selmer R, Kristiansen I: Different ways to describe the benefits of risk-reducing treatments. Ann Intern Med. 2007, 146 (12): 848-856. 10.7326/0003-4819-146-12-200706190-00006.PubMedGoogle Scholar
- Zikmund-Fisher B, Fagerlin A, Roberts T, Derry H, Ubel P: Alternate methods of framing information about medication side effects: Incremental risk versus total risk occurrence. J Health Commun. 2008, 13 (2): 107-124. 10.1080/10810730701854011.PubMedGoogle Scholar
- Schwartz L, Woloshin S, Welch H: Using a drug facts box to communicate drug benefits and harms. Ann Intern Med. 2009, 150 (8): 516-527. 10.7326/0003-4819-150-8-200904210-00106.PubMedGoogle Scholar
- Zikmund-Fisher B, Ubel P, Smith D, Derry H, McClure J, Stark A, Pitsch R, Fagerlin A: Communicating side effect risks in a tamoxifen prophylaxis decision aid: The debiasing influence of pictographs. Patient Educ Couns. 2008, 73 (2): 209-214. 10.1016/j.pec.2008.05.010.PubMed CentralPubMedGoogle Scholar
- Malenka D, Baron J, Johansen S, Wahrenberger J, Ross J: The framing effect of relative and absolute risk. J Gen Intern Med. 1993, 8 (10): 543-548. 10.1007/BF02599636.PubMedGoogle Scholar
- Naylor C, Chen E, Strauss B: Measured enthusiasm: does the method of reporting trial results alter perceptions of therapeutic effectiveness?. Ann Intern Med. 1992, 117 (11): 916-921. 10.7326/0003-4819-117-11-916.PubMedGoogle Scholar
- Tversky A, Kahneman D: The framing of decisions and the psychology of choice. Science. 1981, 211 (4481): 453-458. 10.1126/science.7455683.PubMedGoogle Scholar
- Moxey A, O'Connell D, McGettigan P, Henry D: Describing treatment effects to patients. J Gen Intern Med. 2003, 18 (11): 948-959. 10.1046/j.1525-1497.2003.20928.x.PubMed CentralPubMedGoogle Scholar
- Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G: Communicating statistical information. Science. 2000, 290 (5500): 2261-2262. 10.1126/science.290.5500.2261.PubMedGoogle Scholar
- Gigerenzer G, Hoffrage U: How to improve Bayesian reasoning without instruction: Frequency formats. Psychol Rev. 1995, 102 (4): 684-704.Google Scholar
- Hoffrage U, Gigerenzer G, Krauss S, Martignon L: Representation facilitates reasoning: what natural frequencies are and what they are not. Cognition. 2002, 84 (3): 343-352. 10.1016/S0010-0277(02)00050-1.PubMedGoogle Scholar
- Stacey D, Bennett C, Barry M, Col N, Eden K, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Légaré F, Thomson R: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2001, 10 (CD001431):
- Gattellari M, Ward J: Does evidence-based information about screening for prostate cancer enhance consumer decision-making? A randomised controlled trial. J Med Screen. 2003, 10 (1): 27-39. 10.1258/096914103321610789.PubMedGoogle Scholar
- Kuppermann M, Norton M, Gates E, Gregorich S, Learman L, Nakagawa S, Feldstein V, Lewis J, Washington A, Nease RF J: Computerised prenatal genetic testing decision-assisting tool. Obstet Gynecol. 2009, 113 (1): 53-63. 10.1097/AOG.0b013e31818e7ec4.PubMedGoogle Scholar
- Schapira M, VanRuiswyk J: The effect of an illustrated pamphlet decision-aid on the use of prostate cancer screening tests. J Fam Pract. 2000, 49 (5): 418-424.PubMedGoogle Scholar
- Wolf A, Schorling J: Does informed consent alter elderly patients' preferences for colorectal cancer screening? Results of a randomized trial. J Gen Intern Med. 2000, 15 (1): 24-30. 10.1046/j.1525-1497.2000.01079.x.PubMed CentralPubMedGoogle Scholar
- Lerman C, Biesecker B, Benkendorf J, Kerner J, Gomez-Caminero A, Hughes C, Reed M: Controlled trial of pretest education approaches to enhance informed decision-making for BRCA1 gene testing. J Natl Cancer Inst. 1997, 89 (2): 148-157. 10.1093/jnci/89.2.148.PubMedGoogle Scholar
- Woloshin S, Schwartz L, Welch H: The risk of death by age, sex, and smoking status in the United States: Putting health risks in context. J Natl Cancer Inst. 2008, 100: 845-853. 10.1093/jnci/djn124.PubMed CentralPubMedGoogle Scholar
- Peters E, Dieckmann N, Vastfjall D, Mertz C, Slovic P, Hibbard J: Bringing meaning to numbers: The impact of evaluative categories on decisions. J Exp Psycho App. 2009, 15 (3): 213-227.Google Scholar
- Zikmund-Fisher B, Fagerlin A, Keeton K, Ubel P: Does labeling prenatal screening test results as negative or positive affect a woman's responses?. Am J Obstet Gynecol. 2007, 197 (5): 528.e521-528.e526.Google Scholar
- Han P, Klein W, Arora N: Varieties of uncertainty in health care: a conceptual taxonomy. Med Decis Making. 2011, 31 (6): 828-838. 10.1177/0272989X10393976.PubMed CentralPubMedGoogle Scholar
- Lenert L, Cher D: Use of meta-analytic results to facilitate shared decision making. J Am Med Inform Assoc. 1999, 6 (5): 412-419. 10.1136/jamia.1999.0060412.PubMed CentralPubMedGoogle Scholar
- Baty B, Venne V, McDonald J, Croyle R, Halls C, Nash J, Botkin J: BRCA1 testing: Genetic counseling protocol development and counseling issues. J Genet Couns. 1997, 6 (2): 223-244. 10.1023/A:1025620404473.PubMedGoogle Scholar
- Schapira M, Nattinger A, McHorney C: Frequency or probability? A qualitative study of risk communication formats used in health care. Med Decis Making. 2001, 21 (6): 459-467. 10.1177/0272989X0102100604.PubMedGoogle Scholar
- Han P, Klein W, Killam B, Lehman T, Massett H, Freedman A: Representing randomness in the communication of individualized cancer risk estimates: Effects on cancer risk perceptions, worry, and subjective uncertainty about risk. Patient Educ Couns. 2012, 86 (1): 106-113. 10.1016/j.pec.2011.01.033.PubMed CentralPubMedGoogle Scholar
- Ancker J, Weber E, Kukafka R: Effects of game-like interactive graphics on risk perceptions and decisions. Med Decis Making. 2011, 31 (1): 130-142. 10.1177/0272989X10364847.PubMedGoogle Scholar
- Han P, Klein W, Lehman T, Killam B, Massett H, Freedman A: Communication of uncertainty regarding individualized cancer risk estimates: Effects and influential factors. Med Decis Making. 2011, 31 (2): 354-366. 10.1177/0272989X10371830.PubMed CentralPubMedGoogle Scholar
- Lipkus I, Klein W, Rimer B: Communicating breast cancer risks to women using different formats. Cancer Epidemiol Biomarkers Prev. 2001, 10 (8): 895-898.PubMedGoogle Scholar
- Mazor K, Dodd K, Kunches L: Communicating hospital infection data to the public: A study of consumer responses and preferences. Am J Med Qual. 2009, 24 (2): 108-115.PubMedGoogle Scholar
- Muscatello DJ, Searles A, Macdonald R, J L: Communicating population health statistics through graphs: A randomised controlled trial of graph design interventions. BMC Med. 4 (1): 33-
- Han P, Klein W, Lehman T, Massett H, Lee S, Freedman A: Laypersons’ responses to the communication of uncertainty regarding cancer risk estimates. Med Decis Making. 2009, 29 (3): 391-403. 10.1177/0272989X08327396.PubMed CentralPubMedGoogle Scholar
- Ellsberg D: Risk, ambiguity, and the savage axioms. Q J Econ. 1961, 75: 643-669. 10.2307/1884324.Google Scholar
- Camerer C, Weber M: Recent developments in modeling preferences: Uncertainty and ambiguity. J Risk Uncertain. 1992, 5 (4): 325-370. 10.1007/BF00122575.Google Scholar
- Kuhn K: Communicating uncertainty: Framing effects on responses to vague probabilities. Organ Behav Hum Dec. 1997, 71: 55-83. 10.1006/obhd.1997.2715.Google Scholar
- Viscusi W, Magat W, Huber J: Communication of ambiguous risk information. Theor Decis. 1991, 31 (2): 159-173.Google Scholar
- Garcia-Retamero R, Galesic M, Gigerenzer G: Do icon arrays help reduce denominator neglect?. Med Decis Making. 2010, 30 (6): 672-684. 10.1177/0272989X10369000.PubMedGoogle Scholar
- Garcia-Retamero R, Cokely E: Effective communication of risks to young adults: Using message framing and visual aids to increase condom use and STD screening. J Exp Psycho App. 2011, 17 (3): 270-Google Scholar
- Garcia-Retamero R, Galesic M: How to reduce the effect of framing on messages about health. J Gen Intern Med. 2010, 25 (12): 1323-1329. 10.1007/s11606-010-1484-9.PubMed CentralPubMedGoogle Scholar
- Fagerlin A, Wang C, Ubel P: Reducing the influence of anecdotal reasoning on people’s health care decisions: Is a picture worth a thousand statistics?. Med Decis Making. 2005, 25 (4): 398-405. 10.1177/0272989X05278931.PubMedGoogle Scholar
- Barbey A, Sloman S: Base-rate respect: From ecological rationality to dual processes. Beh Brain Sci. 2007, 30: 241-97.Google Scholar
- Sloman SA, Over D, Slovak L, Stibel J: Frequency illusions and other fallacies. Organ Behav Hum Dec. 2003, 91 (2): 296-309. 10.1016/S0749-5978(03)00021-9.Google Scholar
- Brase G: Pictorial representations in statistical reasoning. Appl Cognitive Psych. 2009, 23 (3): 369-381. 10.1002/acp.1460.Google Scholar
- Gurmankin A, Helweg-Larsen M, Armstrong K, Kimmel S, Volpp K: Comparing the standard rating scale and the magnifier scale for assessing risk perceptions. Med Decis Making. 2005, 25 (5): 560-570. 10.1177/0272989X05280560.PubMedGoogle Scholar
- Woloshin S, Schwartz L, Byram S, Fischhoff B, Welch H: A new scale for assessing perceptions of chance: a validation study. Med Decis Making. 2000, 20 (3): 298-307. 10.1177/0272989X0002000306.PubMedGoogle Scholar
- Paling J: Strategies to help patients understand risks. BMJ. 2003, 327 (7417): 745-748. 10.1136/bmj.327.7417.745.PubMed CentralPubMedGoogle Scholar
- Spiegelhalter D, Pearson M, Short I: Visualizing uncertainty about the future. Science. 2011, 333 (6048): 1393-1400. 10.1126/science.1191181.PubMedGoogle Scholar
- Galesic M, Garcia-Retamero R: Graph literacy: a cross-cultural comparison. Med Decis Making. 2011, 31: 444-457. 10.1177/0272989X10373805.PubMedGoogle Scholar
- Garcia-Retamero R, Galesic M: Who profits from visual aids: Overcoming challenges in people’s understanding of risks. Soc Sci Med. 2010, 70 (7): 1019-1025. 10.1016/j.socscimed.2009.11.031.PubMedGoogle Scholar
- Galesic M, Garcia-Retamero R, Gigerenzer G: Using icon arrays to communicate medical risks to low-numeracy people. Health Psychol. 2009, 28 (2): 210-216.PubMedGoogle Scholar
- Garcia-Retamero R, Galesic M: Communicating treatment risk reduction to people with low numeracy skills: A cross-cultural comparison. Am J Public Health. 2009, 99 (12): 2196-2202. 10.2105/AJPH.2009.160234.PubMed CentralPubMedGoogle Scholar
- Gaissmaier W, Wegwarth O, Skopec D, Müller A-S, Broschinski S, Politi M: Numbers can be worth a thousand pictures: Individual differences in understanding graphical and numerical representations of health-related information. Health Psychol. 2012, 31 (3): 286-296.PubMedGoogle Scholar
- Feldman-Stewart D, Kocovski N, McConnell B, Brundage M, Mackillop W: Perception of quantitative information for treatment decisions. Med DecisMaking. 2000, 20 (2): 228-238.Google Scholar
- Reyna V: A theory of medical decision making and health: Fuzzy-trace theory. Med Decis Making. 2008, 28 (6): 829-833. 10.1177/0272989X08327069.PubMed CentralPubMedGoogle Scholar
- Hawley S, Zikmund-Fisher B, Ubel P, Jankovic A, Lucas T, Fagerlin A: The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Educ Couns. 2008, 73 (3): 448-455. 10.1016/j.pec.2008.07.023.PubMedGoogle Scholar
- Lipkus I: Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. Med Decis Making. 2007, 27 (5): 696-713. 10.1177/0272989X07307271.PubMedGoogle Scholar
- Lipkus I, Hollands J: The visual communication of risk. J Natl Cancer Inst Monogr. 1999, 25: 149-163.PubMedGoogle Scholar
- Zikmund-Fisher B, Fagerlin A, Ubel P: Improving understanding of adjuvant therapy options by using simpler risk graphics. Cancer. 2008, 113 (12): 3382-3390. 10.1002/cncr.23959.PubMed CentralPubMedGoogle Scholar
- Zikmund-Fisher B, Fagerlin A, Ubel P: A demonstration of ''less can be more'' in risk graphics. Med Decis Making. 2010, 30 (6): 661-671. 10.1177/0272989X10364244.PubMed CentralPubMedGoogle Scholar
- Ancker J, Weber E, Kukafka R: Effect of arrangement of stick figures on estimates of proportion in risk graphics. Med Decis Making. 2011, 31 (1): 143-150. 10.1177/0272989X10369006.PubMedGoogle Scholar
- Stone E, Yates J, Parker A: Effects of numerical and graphical displays on professed risk-taking behavior. J Exp Psychol Appl. 1997, 3 (4): 243-256.Google Scholar
- Stone E, Sieck W, Bull B, Yates J, Parks S, Rush C: Foreground:background salience: Explaining the effects of graphical displays on risk avoidance. Organ Behav Hum Decis Process. 2003, 90 (1): 19-36. 10.1016/S0749-5978(03)00003-7.Google Scholar
- McCaffery K, Dixon A, Hayen A, Jansen J, Smith S, Simpson J: The Influence of Graphic Display Formaton the Interpretations of Quantitative Risk Information among Adultswith Lower Education and Literacy: A Randomized Experimental Study. Med Decis Making. 2012, 32: 532-544. 10.1177/0272989X11424926.PubMedGoogle Scholar
- Kurz-Milcke E, Gigerenzer G, Martignon L: Transparency in risk communication: Graphical and analog tools. Strategies for risk communication: Evolution, evidence, experience. Edited by: Tucker W, Ferson S, Finkel A, Long T, Slavin D, Wright P. 2008, New York: Blackwell, 1128: 18-28.Google Scholar
- Shah P, Hoeffner J: Review of graph comprehension research: Implications for instruction. Educ Psychol Rev. 2002, 14 (1): 47-69. 10.1023/A:1013180410169.Google Scholar
- Kosslyn S: Graph Design for the Eye and Mind. 2006, New York: Oxford University PressGoogle Scholar
- Noar S, Benac C, Harris M: Does tailoring matter? Meta-analytic review of tailored print health behaviour change interventions. Psychol Bull. 2007, 133: 673-693.PubMedGoogle Scholar
- Manne S, Coups E, Markowitz A, Meropol N, Haller D, J P, Jandorf L, Peterson S, Lesko S, Pilipshen S, et al: A randomized trial of generic versus tailored interventions to increase colorectal cancer screening among intermediate risk siblings. Ann Behav Med. 2009, 37: 207-217. 10.1007/s12160-009-9103-x.PubMed CentralPubMedGoogle Scholar
- Albada A, Ausems M, Bensing J, Van Dulmen S: Tailored information about cancer risk and screening: A systematic review. Patient Educ Couns. 2009, 77: 155-171. 10.1016/j.pec.2009.03.005.PubMedGoogle Scholar
- Bodurtha J, Quillin J, Tracy K, Borzelleca J, McClish D, Wilson D, Jones R, Quillin J, Bowen D: Mammography screening after risk-tailored messages: The Women Improving Screening through Education and Risk Assessment (WISER) randomized, controlled trial. J Womens Health. 2009, 18: 41-47. 10.1089/jwh.2007.0703.Google Scholar
- Lu C, Karon J, Sorich M: The importance of high-quality evidence of the long-term impact of nonfatal events used in randomized controlled trials: a case study of Parasugrel. Clin Pharmacol Ther. 2011, 90 (1): 27-29. 10.1038/clpt.2011.61.PubMedGoogle Scholar
- NICE: National Institute for Health and Clinical Excellence. Prasugrel for the treatment of acute coronary syndromes with percutaneous coronary intervention. NICE technology appraisal guidance 182. Edited by: National Institute for Health and Clinical Excellence. 2009, LondonGoogle Scholar
- Goldhaber-Fiebert J, Stout N, Goldie S: Empirically evaluating decision-analytic models. Value Health. 2010, 13 (5): 667-674. 10.1111/j.1524-4733.2010.00698.x.PubMedGoogle Scholar
- Stout N, Knudsen A, Kong C, McMahon P, Gazelle G: Calibration methods used in cancer simulation models and suggested reporting guidelines. Pharmacoeconomics. 2009, 27: 533-545. 10.2165/11314830-000000000-00000.PubMed CentralPubMedGoogle Scholar
- Tunis S, Stryer D, Clancy C: Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003, 290 (12): 1624-1632. 10.1001/jama.290.12.1624.PubMedGoogle Scholar
- Levin L, Goeree R, Levine M, Krahn M, Easty T, Brown A, Henry D: Coverage with evidence development: The Ontario experience. Int J Technol Assess Health Care. 2001, 27 (2): 159-168.Google Scholar
- Statin Choice. Mayo Knowledge and Encounter Research Unit, Mayo Clinic College of Medicine. [http://shareddecisions.mayoclinic.org/decision-aids-for-diabetes/cardiovascular-prevention/]
- Bennett C, Drake E, Hopkins L, O’Connor A: What can you do to prevent HPV and cervical cancer?: A decision aid for parents/guardians of girls in grade 8 in Ontario. Canadian Institute of Health. 2009, [http://decisionaid.ohri.ca/docs/das/HPV_vaccine.pdf]Google Scholar
- Heart disease: Should I have bypass surgery?. [https://www.healthwise.net/cochranedecisionaid/Content/StdDocument.aspx?DOCHWID=av2037]
- Vandemheen K, Aaron S, Tullis E, Poirier C: When your lung function is getting worse: Should you be referred for a lung transplant? A decision aid for adults with cystic fibrosis. 2009, [http://decisionaid.ohri.ca/docs/das/CF_group_a.pdf]Google Scholar
- Zikmund-Fisher B, Fagerlin A, Ubel P: What’s time got to do with it? Inattention to duration in interpretation of survival graphs. Risk Analysis. 2005, 25 (3): 589-595. 10.1111/j.1539-6924.2005.00626.x.PubMedGoogle Scholar
- Zikmund-Fisher B, Fagerlin A, Ubel P: Mortality versus survival graphs: Improving temporal consistency in perception of treatment effectiveness. Patient Educ Couns. 2007, 66: 100-107. 10.1016/j.pec.2006.10.013.PubMedGoogle Scholar
- Armstrong K, Schwartz J, Fitzergald G, Putt M, Ubel P: Effect of framing as gain versus loss on understanding and hypothetical treatment choices: Survival and mortality curves. Med Decis Making. 2002, 22: 76-83. 10.1177/0272989X0202200108.PubMedGoogle Scholar
- Redelmeier D, Rozin P, Kahneman D: Understanding patients’ decisions: Cognitive and emotional perspectives. JAMA. 1993, 270 (1): 72-76. 10.1001/jama.1993.03510010078034.PubMedGoogle Scholar
- McNair A, Brookes S, Davis C, Argyropoulos M, Blazeby J: Communicating the results of randomized clinical trials: Do patients understand multidimensional patient-reported outcomes?. J Clin Oncol. 2010, 28: 738-743. 10.1200/JCO.2009.23.9111.PubMedGoogle Scholar
- Rolison J, Hanoch Y, Miron-Shatz T: What do men understand about lifetime risk following genetic testing? The effect of context and numeracy. Health Psychol. 2012, 31 (4): 530-3. doi: 10.1037/a0026562PubMedGoogle Scholar
- National Cancer Institue: BRCA1 and BRCA2: Cancer Risk and Genetic Testing. [http://www.cancer.gov/cancertopics/factsheet/Risk/BRCA]
- Cochrane Musculoskeletal Group: Decision Aids. [http://musculoskeletal.cochrane.org/decision-aids]
- Sydney Health Decision Group: Making Decisions: Should I use hormone replacement therapy? (HRT). [http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/wh37.pdf]
- Healthwise: Achilles tendon rupture: Should I have surgery?. [https://www.healthwise.net/cochranedecisionaid/Content/StdDocument.aspx?DOCHWID=ug2998]
- Healthwise: Heart Failure: Should I get a Pacemaker (Cardiac Resychronization Therapy)?. [https://www.healthwise.net/cochranedecisionaid/Content/StdDocument.aspx?DOCHWID=uf9843]
- Mayo Clinic: Birth control. [http://www.mayoclinic.com/health/birth-control/MY01182]
- Betsch C, Ulshofer C, Renkewitz F, Betsch T: The Influence of Narrative v. Statistical Information on Perceiving Vaccination Risks. Med Decis Making. 2011, 31 (5): 742-53. 10.1177/0272989X11400419.PubMedGoogle Scholar
- Ubel PA, Jepson C, Baron J: The inclusion of patient testimonials in decision aids: effects on treatment choices. Med Decis Making. 2001, 21 (1): 60-68. 10.1177/0272989X0102100108.PubMedGoogle Scholar
- Shaffer V, Zikmund-Fisher B: All stories are not alike: A purpose-, content-, and valence-based taxonomy of patient narratives in decision aids. Med Decis Making. 2013, 33 (1): 4-13. 10.1177/0272989X12463266.PubMedGoogle Scholar
- Lipkus I, Peters E: Understanding the role of numeracy in health: proposed theoretical framework and practical insights. Health Educ Behav. 2009, 36: 1065-1081. 10.1177/1090198109341533.PubMed CentralPubMedGoogle Scholar
- Reyna V, Nelson W, Han P, Dieckmann N: How numeracy influences risk comprehension and medical decision making. Psychol Bull. 2009, 135: 943-973.PubMed CentralPubMedGoogle Scholar
- Lipkus I, Peters E, Kimmick G, Liotcheva V, Marcom P: Breast cancer patients' treatment expectations after exposure to the decision aid program adjuvant online: the influence of numeracy. Med Decis Making. 2010, 30: 464-473. 10.1177/0272989X09360371.PubMed CentralPubMedGoogle Scholar
- Dieckmann NF, Slovic P, Peters EM: The use of narrative evidence and explicit likelihood by decisionmakers varying in numeracy. Risk Anal. 2009, 29 (10): 1473-1488. 10.1111/j.1539-6924.2009.01279.x.PubMedGoogle Scholar
- Galesic M, Garcia-Retamero R: Communicating consequences of risky behaviors: Life expectancy versus risk of disease. Patient Educ Couns. 2011, 82: 30-35. 10.1016/j.pec.2010.02.008.PubMedGoogle Scholar
- Weinstein N, Atwood K, Puleo E, Fletcher R, Colditz G, Emmons K: Colon cancer: risk perceptions and risk communication. J Health Commun. 2004, 9: 53-65. 10.1080/10810730490271647.PubMedGoogle Scholar
- Woloshin S, Schwartz L, Black W, Welch H: Women’s perceptions of breast cancer risk: How you ask matters. Med Decis Making. 1999, 19: 221-229. 10.1177/0272989X9901900301.PubMedGoogle Scholar
- Peters E, Västfjäll D, Slovic P, Mertz C, Mazzocco K, Dickert S: Numeracy and decision making. Psychol Sci. 2006, 17: 407-413. 10.1111/j.1467-9280.2006.01720.x.PubMedGoogle Scholar
- Zikmund-Fisher B, Angott A, Ubel P: The benefits of discussing adjuvant therapies one at a time instead of all at once. Breast Cancer Res Treat. 2011, 129: 79-87. 10.1007/s10549-010-1193-4.PubMed CentralPubMedGoogle Scholar
- Reyna VF, Brainerd C: Numeracy, ratio bias, and denominator neglect in judgments of risk and probability. Learn Individ Differ. 2008, 18: 89-107. 10.1016/j.lindif.2007.03.011.Google Scholar
- Peters E: Beyond comprehension: The role of numeracy in judgments and decisions. Curr Dir Psychol Sci. 2012, 21: 31-10.1177/0963721411429960.Google Scholar
- Galesic M, Garcia-Retamero R: Statistical numeracy for health: A cross-cultural comparison with probabilistic national samples. Arch Intern Med. 2010, 170: 462-468. 10.1001/archinternmed.2009.481.PubMedGoogle Scholar
- Lipkus I, Samsa G, Rimer B: General performance on a numeracy scale among highly educated samples. Med Decis Making. 2001, 21: 37-44.PubMedGoogle Scholar
- Hibbard J, Peters E, Dixon A, Tusler M: Consumer competencies and the use of comparative quality information: It isn’t just about literacy. Med Care Res Rev. 2007, 64 (4): 379-394. 10.1177/1077558707301630.PubMedGoogle Scholar
- Peters E, Baker D, Dieckmann N, Leon J, Collins J: Explaining the effect of education on health: A field study in Ghana. Psychol Sci. 2010, 21 (10): 1369-1376. 10.1177/0956797610381506.PubMedGoogle Scholar
- Yin H, Wolf M, Dreyer B, Sanders L, Parker R: Evaluation of consistency in dosing directions and measuring devices for pediatric nonprescription liquid medications. JAMA. 2010, 304 (23): 2595-2602. 10.1001/jama.2010.1797.PubMedGoogle Scholar
- Kumar D, Sanders L, Perrin E, Lokker N, Patterson B, Gunn V, Finkle J, Franco V, Choi L, Rothman R: Parental understanding of infant health information: Health literacy, numeracy, and the Parental Health Literacy Activities Test (PHLAT). AcadPediatr. 2010, 10 (5): 309-316.Google Scholar
- Cokely E, Galesic M, Schulz E, Garcia-Retamero R: Measuring risk literacy: the Berlin Numeracy Test. Judgm Decis Mak. 2012, 7: 35-47.Google Scholar
- Liberali J, Reyna V, Furlan S, Stein L, Pardo S: Individual differences in numeracy and cognitive reflection, with implications for biases and fallacies in probability judgment. J Beh Dec Mak. 2012, 25: 361-381. 10.1002/bdm.752.Google Scholar
- Butterworth B: Mathematical expertise. Cambridge Handbook of Expertise and Expert Performance. Edited by: Ericsson K. 2006, Cambridge: CUP, 553-568.Google Scholar
- Stigler J, Gonzales P, Kawanaka T, Knoll S, Serrano A: The TIMSS videotape classroom study: Methods and findings from an exploratory research project on eighth-grade mathematics instruction in Germany, Japan, and the United States. Edited by: US Dept of Education, National Center for Education Statistics. Washington DC. 1999Google Scholar
- Peters E, Hess T, Västfjäll D, Auman C: Adult age differences in dual information processes: Implications for the role of affective and deliberative processes in older adults’ decision making. Perspect Psychol Sci. 2007, 2 (1): 1-23. 10.1111/j.1745-6916.2007.00025.x.PubMedGoogle Scholar
- Kahan D, Peters E, Wittlin M, Slovic P, Ouellette L, Braman D, Mandel G: The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Clim Change. 2012, 2: 732-745. 10.1038/nclimate1547.Google Scholar
- Lipkus IM: Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Making. 2007, 27 (5): 696-713. 10.1177/0272989X07307271.PubMedGoogle Scholar
- Zikmund-Fisher B, Smith D, Ubel P, Fagerlin A: Validation of the Subjective Numeracy Scale (SNS): Effects of low numeracy on comprehension of risk communications and utility elicitations. Med Decis Making. 2007, 27: 663-671. 10.1177/0272989X07303824.PubMedGoogle Scholar
- Schwartz L, Woloshin S, Black W, Welch G: The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med. 1997, 127: 966-971. 10.7326/0003-4819-127-11-199712010-00003.PubMedGoogle Scholar
- Schapira M, Walker C, Cappaert K, Ganschow P, Fletcher K, McGinley E, Del Pozo S, Schauer C, Tarima S, Jacobs E: The Numeracy Understanding in Medicine Instrument: : A measure of health numeracy developed using item response theory. Med Decis Making. 2012, 32: 852-865.Google Scholar
- Weller J, Dieckmann N, Tusler M, Mertz C, Burns W, Peters E: Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. J Beh Dec Mak. 2012, 26 (2): 198-212.Google Scholar
- Fagerlin A, Zikmund-Fisher B, Ubel P, Jankovic A, Derry H, Smith D: Measuring numeracy without a math test: development of the subjective numeracy scale. Med Decis Making. 2007, 27: 672-680. 10.1177/0272989X07304449.PubMedGoogle Scholar
- Baker W: The meaning and the measure of health literacy. J Gen Intern Med. 2006, 21: 878-883. 10.1111/j.1525-1497.2006.00540.x.PubMed CentralPubMedGoogle Scholar
- Schwartz L, Woloshin S, Welch G: Can patients interpret health information? An assessment of the medical data interpretation test. Medical Decision Making. 2005, 25: 290-300. 10.1177/0272989X05276860.PubMedGoogle Scholar
- Natter H, Berry D: Effects of active information processing on the understanding of risk information. Appl Cogn Psychol. 2005, 19: 123-135. 10.1002/acp.1068.Google Scholar
- Ancker J, Chan C, Kukafka R: Interactive graphics for expressing health risks: development and qualitative evaluation. J Health Commun. 2009, 14 (5): 461-475. 10.1080/10810730903032960.PubMed CentralPubMedGoogle Scholar
- Tversky B, Morrison J, Betrancourt M: Animation: can it facilitate?. Int J Hum Comput Stud. 2002, 57: 247-262. 10.1006/ijhc.2002.1017.Google Scholar
- Zikmund-Fisher B, Dickson M, Witteman H: Cool but counterproductive: interactive web-based risk communications can backfire. J Med Internet Res. 2011, 13 (3): e60-10.2196/jmir.1665. doi:10.2196/jmir.1665PubMed CentralPubMedGoogle Scholar
- Zikmund-Fisher B, Witteman H, Fuhrel-Forbis A, Exe N, Kahn V, Dickson M: Animated graphics for comparing two risks: a cautionary tale. J Med Internet Res. 2012, 14 (4): e106-10.2196/jmir.2030. doi: 110.2196/jmir.2030PubMed CentralPubMedGoogle Scholar