- Research article
- Open Access
- Open Peer Review
Risk communication formats for low probability events: an exploratory study of patient preferences
BMC Medical Informatics and Decision Making volume 8, Article number: 14 (2008)
Clear communication about the possible outcomes of proposed medical interventions is an integral part of medical care. Despite its importance, there have been few studies comparing different formats for presenting probabilistic information to patients, especially when small probabilities are involved. The purpose of this study was to explore the potential usefulness of several new small-risk graphic communication formats.
Information about the likelihoods of cancer and cancer prevention associated with two hypothetical cancer screening programs were used to create an augmented bar chart, an augmented grouped icon display, a flow chart, and three paired combinations of these formats. In the study scenario, the baseline risk of cancer was 53 per 1,000 (5.3%). The risk associated with cancer screening option A was 38 per 1,000 (3.8%) and the risk associated with screening option B was 29 per 1,000 (2.9%). Both the augmented bar chart and the augmented grouped icon display contained magnified views of the differences in cancer risk and cancer prevention associated with the screening programs. A convenience sample of 29 subjects (mean age 56.4 years; 76% men) used the Analytic Hierarchy Process (AHP) to indicate their relative preferences for the six formats using 15 sequential paired comparisons.
The most preferred format was the combined augmented bar chart + flow diagram (mean preference score 0.43) followed by the combined augmented icon + augmented bar chart format (mean preference score 0.22). The overall differences among the six formats were statistically significant: Kruskal-Wallis Chi Square = 141.4, p < 0.0001. The three combined formats all had statistically significant higher preferences scores than the single format displays (p < 0.05).
These findings suggest that patients may prefer combined, rather than single, graphic risk presentation formats and that augmented bar charts and icon displays may be useful for conveying comparative information about small risks to clinical decision makers. Further research to confirm and extend these findings is warranted.
Clear communication about the possible risks and benefits of proposed medical interventions has been an integral part of medical care since the doctrine of informed consent was adopted. In recent years, it has become increasingly more important due to changes in the accepted model of the doctor-patient relationship that promote more active involvement of patients in decisions about their care, the rise of evidence-based medical practice, and the increasing emphasis placed on preventive measures that are intended to reduce future health risks [1, 2]. It is likely to become more important in the future as new information about personal health risks becomes available through work in the "new sciences" such as medical genomics, metabolic profiling, and proteomics . Richard Smith, former editor of the British Medical Journal, recently called risk communication the "main work of doctors" .
Current recommendations for communicating information about uncertain future events emphasize the importance of presenting data in a balanced manner that avoids framing effects, provides baseline risk information, and uses graphic risk displays whenever possible. Recommended risk displays include bar charts, grouped icon displays (also known as dot or face diagrams), and flow diagrams [5–7]. The use of multiple risk presentation formats has also been suggested [7, 8].
Despite the importance of risk communication, there have been few empirical studies comparing different graphic formats for presenting probabilistic information to patients for use in shared decision-making [8–12]. In comparative studies, both bar charts and icon displays have been more effective than simple numeric statements and several other graphic formats for comparing the outcomes of alternative decision options [13, 14]. Ordered icon displays have also been shown to minimize decision biases due to vivid anecdotal information and to be well understood by patients over 75 years old [16, 17]. Flow diagrams, developed by Gigerenzer and colleagues, have been shown to help people better understand risk forecasts as well as more complex probabilistic data, such as the change in risk of disease after a positive or negative test result [18, 19].
A limitation of the currently available data is that most studies to date have compared risk displays using relatively common events with likelihoods of 20% or more. Outcomes with expected likelihoods of less than 5% occur frequently in medical decision making situations and in many cases have an important impact on medical decisions. Probably the most common of these low likelihood outcomes are risks of serious adverse events. For example, in 2004 the drug rofecoxib, a COX-2 inhibitor, was removed from the market in the United States due to concerns about an increase in the annual rate of myocardial infarctions from one per thousand to four per thousand, an absolute risk increase of three per thousand .
Small risks are difficult to depict in bar charts and icon displays because it is hard to include all of the necessary information in a single display. Studies with both bar charts and icon displays have shown that graphs that do not include information about the chances of both experiencing and not experiencing the outcome can lead to misinterpretation of the data presented [12, 16]. The need to properly illustrate this part-to-whole relationship is what makes it difficult to use these formats to convey information about low likelihood events
A possible solution to this problem is to use graphs that include a special section designed to highlight information about low likelihood events. This approach has been used successfully in two studies that elicited probability estimates from patients using graphs that included a "magnified" zone for probabilities less than 1% [22, 23]. Whether this approach can also be used to create effective graphic displays for conveying low likelihood information to clinical decision makers is currently unknown.
This study was designed as the first step in a series of investigations designed to address the question of how to most effectively present information about low likelihood events to patients and other clinical decision makers. The purpose of this study was to explore the potential usefulness of several new small-risk graphic communication formats.
The study population consisted of a convenience sample of middle-aged to elderly adults, recruited through word of mouth and distribution of informational materials, who volunteered to participate in a study of medical decision making. Patients were recruited from three separate sites in Rochester NY: a general maintenance shop, a weekly social gathering at an elderly retirement community, and a medical research institute contained within the University of Rochester. No patients had prior knowledge of the project or the study hypotheses. Formal health status was not assessed. However, subjects at the maintenance shop and research institute were working regularly at the time of the study and the subjects at the retirement community were well enough to attend a weekly social function. This study was a pilot study performed in connection with a larger project that was approved by the University of Rochester Research Subjects Review Board. The data were collected anonymously. All subjects gave consent to participate.
The study intervention was a computer-assisted comparison of six different graphic communication formats for displaying the differences in effectiveness of two hypothetical cancer screening programs. In the study scenario, the baseline risk of cancer was 53 per 1,000 (5.3%). The risk associated with cancer screening option A was 38 per 1,000 (3.8%) and the risk associated with screening option B was 29 per 1,000 (2.9%). These absolute risks and risk reductions are comparable to outcomes seen with currently recommended colorectal cancer screening tests . Although screening by itself does not reduce cancer risk, we did not include details about screening-related treatments to simplify the problem and focus attention on the graphic displays.
Three of the formats studied were based on currently recommended graphic displays: a flow diagram, a vertical bar chart, and a grouped icon display. The bar chart and icon display were augmented to include both an overall depiction of the possible outcomes associated with each screening option and a magnified view of the differences between them in cancer risk and likelihood of cancer prevention. The resulting diagrams are shown in Figures 1, 2, 3.
The other three formats included in the study were paired combinations of the three single formats: a flow diagram plus an augmented icon display, a flow diagram plus an augmented bar chart, and an augmented icon display plus an augmented bar chart. An example of a combined format is shown in Figure 4.
After a brief explanation of the study and a training exercise that involved judging the relative weights of three balls, the subjects were shown two of the six diagrams on a computer screen and asked to indicate if they liked both formats equally or if they preferred one to the other for comparing the effectiveness of the two screening options for preventing cancer. If one was preferred, they were asked to indicate their strength of preference using a dynamic slider that showed a range of preferences from equal to nine times more preferable. The relationships between different settings on the slider and relative strengths of preference were illustrated by linked horizontal bar and pie charts. An example of the screen used to make these comparisons is shown in Figure 5. This process was repeated until all six formats were compared, a total of 15 comparisons. Participants needed approximately 45 minutes to one hour to complete the study. The computer program used to perform these assessments is attached: see Additional file 1.
Quantitative preference scores summarizing the comparisons among the formats were calculated using the Analytic Hierarchy Process (AHP). To create the AHP-based numeric preference scores, each subject's comparisons were entered into a matrix and the normalized right eigenvector of the matrix was calculated. The resulting eigenvector values define the formats' preference scores. Because the eigenvector is normalized, the scores range from 0 to 1 and the sum of all scores equals 1. Scores are interpreted as indicating each format's relative strength of preference compared to the others, measured on a ratio scale.
The reliability of each subject's comparisons was assessed by calculating the consistency ratio, a standard measure of the reliability of the pairwise comparisons used to generate the preference scale that is routinely used in AHP applications . Consistency ratios run from 0 to 1, with 0 indicating a perfectly consistent set of comparisons. Traditionally, consistency ratios ≤ 0.10 have been considered acceptable; in practical applications this cut-off is often increased to ≤ 0.20 . For this study the latter cutoff was used and defined a priori.
We analyzed the data to see if there were significant differences among the six presentation formats in subject preferences. Because the response data were not normally distributed, we tested this hypothesis using Kruskal-Wallis analysis of variance. The level of statistical significance was set a priori as an alpha level of 0.05. Differences between individual formats were tested using the Bonferroni multiple comparison test to maintain an alpha of 0.05. All statistical analyses were done using NCSS 2000 .
The mean age of the study population was 56.4 years. There were 22 men and 7 women. The preference scores for the six presentation formats are summarized in the box plot shown in Figure 6. The most preferred format was the combined augmented bar chart + flow diagram with a mean preference score 0.43. This format is shown in Figure 4. The combined augmented icon display + bar chart was second most preferred with a mean preference score of 0.22. A small group of patients strongly preferred this format as indicated by the outlier circles on the box plot. The combined augmented icon display + flow chart had the third highest mean preference score, 0.15. The three single presentation formats were the least preferred with mean preference scores of 0.10 for the augmented bar chart, 0.07 for the flow diagram, and 0.04 for the augmented icon display.
The overall differences in preferences among the six formats were statistically significant: Kruskal-Wallis Chi Square = 141.4, p < 0.0001. The Bonferroni multiple comparison test indicated that the mean preference scores for all three combined formats were significantly different from all other options and the mean preference score for the augmented bar chart was significantly different from the mean score of the augmented icon display, (p < 0.05).
The consistency indices for the comparisons ranged from 0.08 to 0.20 with a mean of 0.135. All subjects met the predefined standard for acceptable consistency of 0.20 or less.
A recently issued set of guidelines for creating patient decision aids recommends the use of multiple risk presentation formats . These results support this recommendation. The most preferred format was a combined format and all three combined formats were more preferred than the three single format options included in the study.
We were unable to find previous studies that have assessed patient preferences for single versus combined graphic risk presentation formats. In a study that compared text formats for presenting treatment effectiveness information, Sheridan and colleagues found that patient understanding was worse for a combined format than several single formats . The discrepancy between this result and ours may reflect differences in the way people process textual versus graphical information [19, 20]. Alternatively, it could be due to the differences in the outcomes studied: preferences versus understanding.
Given the limited scope of this study, we do not know why the patients preferred the combined formats so strongly. The cost-benefit theory of decision strategy choice posits that the way information is displayed affects a person's choice of how they will process the information based on anticipated effort and anticipated accuracy . It is possible that the increased variety of information presentation provided by the combined formats led the patients to perceive that they could be used to make more accurate decisions with less effort than the single format displays. Additional research is warranted to confirm our results and to address additional outcomes including understanding of the information presented, cognitive effort, the rationale underlying risk presentation preferences, and the effect of presentation preferences on the decision making process.
This study also demonstrates the usefulness of using the Analytic Hierarchy Process as a technique for creating measurement scales in situations where no pre-existing scale exists. Although the AHP is most widely known as a multi-criteria decision making method, it is fundamentally a system of measurement that derives a ratio-level scale from a series of pairwise judgments [21–28]. In addition to producing a robust measurement scale, the AHP also yields an assessment of the consistency of the component judgments that can be used as an indicator of the reliability of the measurement process. The AHP therefore is a valuable assessment tool that can enhance our ability to investigate preferences and other important, but difficult to measure, concepts.
This study has several limitations. First, the only outcome studied was patient preference. At least two studies have shown that the most preferred risk formats are not necessarily the same as the most effective formats for risk communication in medical decision making [13, 14]. Assessing preferences as the initial step in evaluating new risk communication formats, however, has the advantage of providing easy to obtain information about an important dimension of a risk display that can be used to help plan additional investigations. Second, only two likelihoods for a single outcome were studied. We were, therefore, not able to determine if there is an interaction between the magnitude of the risk being presented and preferred display format. Third, the study population was a small volunteer convenience sample of people who were not facing a real decision. Fourth, the reproducibility of preferences over time was not tested. Finally, only one version of the newly created graphs was studied. Whether patient preferences are affected by different color schemes, axis formatting, the size of the display, and other design characteristics is unknown.
Despite these limitations, the results of this study suggest that patients may prefer combined, rather than single, graphic risk presentation formats and that augmented bar charts and icon displays may be useful for conveying comparative information about small risks to clinical decision makers. Further research to confirm and extend these findings is warranted.
Sheridan S, Harris R, Woolf S: Shared decisionmaking about screening and chemoprevention. Am J Prev Med. 2004, 26 (1): 67-80. 10.1016/j.amepre.2003.09.011.
Woolf SH, Chan EC, Harris R, Sheridan SL, Braddock CH, Kaplan RM, Krist A, O'Connor AM, Tunis S: Promoting informed choice: transforming health care to dispense knowledge for decision making. Ann Intern Med. 2005, 143 (4): 293-300.
Arab S, Gramolini AO, Ping P, Kislinger T, Stanley B, van Eyk J, Ouzounian M, MacLennan DH, Emili A, Liu PP: Cardiovascular Proteomics: Tools to Develop Novel Biomarkers and Potential Applications. Journal of the American College of Cardiology. 2006, 48 (9): 1733-1741. 10.1016/j.jacc.2006.06.063.
Smith R: Communicating risk: the main work of doctors. BMJ. 2003, 327 (7417):
Edwards A, Elwyn G, Mulley A: Explaining risks: turning numerical data into meaningful pictures. BMJ. 2002, 324 (7341): 827-830. 10.1136/bmj.324.7341.827.
Paling J: Strategies to help patients understand risks. BMJ. 2003, 327 (7417): 745-748. 10.1136/bmj.327.7417.745.
Holmes-Rovner M: International Patient Decision Aid Standards (IPDAS): beyond decision aids to usual design of patient education materials. Health Expect. 2007, 10 (2): 103-7. 10.1111/j.1369-7625.2007.00445.x.
Lipkus I: Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. mdm. 2007, 27 (5): 696-713.
Lipkus IM, Hollands JG: The visual communication of risk. J Natl Cancer Inst Monogr. 1999, 149-163. 25
Wills CE, Holmes-Rovner M: Patient comprehension of information for shared treatment decision making: state of the art and future directions. Patient Education and Counseling. 2003, 50 (3): 285-290. 10.1016/S0738-3991(03)00051-X.
Trevena LJ, Davey HM, Barratt A, Butow P, Caldwell P: A systematic review on communicating with patients about evidence. J Eval Clin Pract. 2006, 12 (1): 13-23. 10.1111/j.1365-2753.2005.00596.x.
Ancker JS, Senathirajah Y, Kukafka R, Starren JB: Design Features of Graphs in Health Risk Communication: A Systematic Review. J Am Med Inform Assoc. 2006, 13 (6): 608-618. 10.1197/jamia.M2115.
Feldman-Stewart D, Kocovski N, McConnell BA, Brundage MD, Mackillop WJ: Perception of quantitative information for treatment decisions. Med Decis Making. 2000, 20 (2): 228-238. 10.1177/0272989X0002000208.
Waters EA, Weinstein ND, Colditz GA, Emmons K: Formats for improving risk communication in medical tradeoff decisions. J Health Commun. 2006, 11 (2): 167-182. 10.1080/10810730500526695.
Chou R, Helfand M, Peterson K, Dana T, Roberts C: Comparative Effectiveness and Safety of Analgesics for Osteoarthritis. Comparative Effectiveness Review No. 4. 2006, Rockville, MD: Agency for Healthcare Research and Quality, [http://www.effectivehealthcare.ahrq.gov/reports/final.cfm]
Stone E, Sieck W, Bull B, Yates J, Parks S, Rush C: Foreground:background salience: explaining the effects of graphical displays on risk avoidance. Organizational Behavior and Human Decision Processes. 2003, 90 (1): 19-36. 10.1016/S0749-5978(03)00003-7.
Winawer SJ, Fletcher RH, Miller L, Godlee F, Stolar MH, Mulrow CD, Woolf SH, Glick SN, Ganiats TG, Bond JH: Colorectal cancer screening: clinical guidelines and rationale. Gastroenterology. 1997, 112 (2): 594-642. 10.1053/gast.1997.v112.agast970594.
Sheridan SL, Pignone MP, Lewis CL: A randomized comparison of patients' understanding of number needed to treat and other common risk reduction formats. J Gen Intern Med. 2003, 18 (11): 884-892. 10.1046/j.1525-1497.2003.21102.x.
Kleinmuntz D, Schkade D: Information displays and decision processes. Psychological Science. 1993, 4 (4): 221-227. 10.1111/j.1467-9280.1993.tb00265.x.
Sanfey A, Hastie R: Does evidence representation format affect judgment? An experimental evaluation of displays of data for judgment. Psychological Science. 1998, 9 (2): 99-103. 10.1111/1467-9280.00018.
Saaty T: The Analytic Hierarchy Process. 1980, New York: McGraw-Hill Book Co
Saaty T: Axiomatic foundations of the analytic hierarchy process. Management Science. 1986, 32 (7): 841-855.
Saaty T: How to make a decision: The Analytic Hierarchy Process. Interfaces. 1994, 24: 19-43.
Saaty T: Decision Making for Leaders. 2001, Pittsburgh, PA: RWS Publications, 3
Dolan JG, Isselhardt BJ, Cappuccio JD: The analytic hierarchy process in medical decision making: a tutorial. Med Decis Making. 1989, 9 (1): 40-50. 10.1177/0272989X8900900108.
Gass S: Model world: When is a number a number?. Interfaces. 2001, 31 (5): 93-103.
Forman E, Gass S: The Analytic Hierarchy Process – An Exposition. Operations Research. 2001, 49: 469-486. 10.1287/opre.49.4.469.11231.
Forman E, Selly M: Decision by objectives. 2001, Hackensack, NJ: World Scientific
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/8/14/prepub
This study was supported by a grant from the National Cancer Institute: 1R01CA112366-01A1. The authors are indebted to the four peer reviewers for their insightful comments and suggestions regarding earlier drafts of this manuscript.
The author(s) declare that they have no competing interests.
JD conceived and designed the study, performed the data analysis, and drafted the manuscript. SI created the data collection computer interface, recruited study subjects, administered the study interviews and compiled the results. Both authors read and approved the final manuscript.