Developing and user-testing Decision boxes to facilitate shared decision making in primary care - a study protocol
© Giguere et al; licensee BioMed Central Ltd. 2011
Received: 17 January 2011
Accepted: 9 March 2011
Published: 9 March 2011
Applying evidence is one of the most challenging steps of evidence-based clinical practice. Healthcare professionals have difficulty interpreting evidence and translating it to patients. Decision boxes are summaries of the most important benefits and harms of diagnostic, therapeutic, and preventive health interventions provided to healthcare professionals before they meet the patient. Our hypothesis is that Decision boxes will prepare clinicians to help patients make informed value-based decisions. By acting as primers, the boxes will enhance the application of evidence-based practices and increase shared decision making during the clinical encounter. The objectives of this study are to provide a framework for developing Decision boxes and testing their value to users.
We will begin by developing Decision box prototypes for 10 clinical conditions or topics based on a review of the research on risk communication. We will present two prototypes to purposeful samples of 16 family physicians distributed in two focus groups, and 32 patients distributed in four focus groups. We will use the User Experience Model framework to explore users' perceptions of the content and format of each prototype. All discussions will be transcribed, and two researchers will independently perform a hybrid deductive/inductive thematic qualitative analysis of the data. The coding scheme will be developed a priori from the User Experience Model's seven themes (valuable, usable, credible, useful, desirable, accessible and findable), and will include new themes suggested by the data (inductive analysis). Key findings will be triangulated using additional publications on the design of tools to improve risk communication. All 10 Decision boxes will be modified in light of our findings.
This study will produce a robust framework for developing and testing Decision boxes that will serve healthcare professionals and patients alike. It is the first step in the development and implementation of a new tool that should facilitate decision making in clinical practice.
Evidence-based medicine refers to the conscientious, explicit, and judicious use of current best evidence when making decisions about health care . Practicing evidence-based medicine means integrating a practitioner's personal clinical expertise with the best clinical evidence available from scientific research . Applying evidence in clinical practice is one of the most challenging aspects of evidence-based practice, in part because physicians find it difficult to interpret evidence . More specifically, physicians do not always correctly estimate the benefits and harms of the interventions they commonly recommend . Physicians are also unclear about how best to discuss the benefits and harms of treatment with patients , this despite evidence that how health professionals present clinical information influences patients' perception of risks-and ultimately, decisions about treatment [5, 6].
It is clear, then, that strategies to help physicians easily access and understand evidence of the benefits and harms of healthcare interventions are needed if physicians are to better communicate this knowledge to their patients. Shared decision making (SDM) is a promising avenue for applying scientific evidence in clinical practice. SDM help healthcare professionals and patients make joint decisions based on two elements: the best evidence of the benefits and harms of all available options, and patients' values and preferences in regard to those options, benefits, and harms . By clarifying values and improving patients' feeling of being adequately informed , SDM reduces the overuse of screening or treatment options not clearly associated with health benefits for all . Patients' active participation in decision making has also been associated with favourable outcomes such as better quality of life  and higher patient satisfaction [11–13] without increasing consultation times .
SDM can be facilitated using patient decision aids. Patient decision aids are "interventions designed to help patients make specific and deliberative choices among options (including the status quo) by providing (at the minimum) information on the options and outcomes relevant to a person's health status and implicit methods to clarify values," in particular values about the benefits and harms of the options . Most patient decision aids are designed so that patients can work through them on their own (at home or somewhere else outside of the consultation): for that reason, they can generally be described as patient-mediated interventions. Only a few have been designed for the clinician to use to facilitate decision making at the point of care [15–17].
This paper reports on the study protocol of the first step of our research program: developing and user-testing Decision boxes. More specifically, we present (1) how we will select clinical topics for 10 Decision boxes, (2) how we will develop the Decision boxes, (3) how we will evaluate users' perception of the format and content of the Decision boxes, and (4) how we will pre-test the questionnaires to be used in the implementation study that will follow. This first study will provide a robust framework for the development and testing of Decision boxes.
Phase 1: Developing the Decision boxes
1. Selecting the clinical topics
A panel consisting of seven of the researchers involved in this project (including four practicing family physicians) will select 10 clinical topics they perceive as relevant to primary care practice, including three relevant genomic topics. They will base their selection of the clinical topics on the following criteria:
The health treatment or screening decision should not have a single 'best' choice, i.e., decisions should be considered 'close calls' because there is scientific uncertainty about outcomes or because a choice requires trading off benefits and harms .
The clinical condition should be commonly encountered in primary care.
A strong body of clinical research on the benefits and harms of the intervention, and ideally an appraisal of the quality of the evidence, should be available for incorporation into the Decision box.
The intervention should be offered in at least one Canadian province.
2. Developing the format and content of the prototypes
Information included in the Decision boxes
Each Decision box will begin by describing the intervention for which a decision is required. It will then identify the population targeted by the intervention and clarify the decision to be considered. The box will supply research-based information about treatment options and their benefits, harms, probabilities, and uncertainties. It will integrate the best evidence available from primary research studies, systematic reviews and synopses: for that reason, we can describe it as summary-level evidence [20, 21]. The box will specify the subjects studied and the study duration. The box's design will facilitate comparison of the benefits and harms of each option and will, to the extent possible, depict the benefits and harms of each option in equal detail .
Risk communication formats
Each Decision box will present research-based information on the probabilities associated with each option as a combination of numbers, graphics and narrative statements. Although the evidence about the best risk communication formats is still not conclusive [6, 23], a number of reviews recommend combining graphs with verbal and numerical formats to improve the reader's understanding of probabilistic information. Supplementing text with graphics has been proposed as a means to reduce the influence of less-relevant textual information, to improve decision-making and to increase accuracy when comparing probabilities . Providing multiple formats may also help a broader audience with various numeracy levels make more accurate assessments of risks .
We will express the numerical probabilities of the benefits and harms of an intervention as absolute risk, communicated as natural frequencies with a constant denominator [5, 6, 26, 27]. Natural frequencies facilitate inferences on the likelihood of having a disease because they carry implicit information about baseline risks, reduce the number of computations required to determine the positive predictive value of a test, and correspond to the way that humans have experienced statistical information for most of the history of mankind . Screening test results that include false positives will be segmented into pieces (prevalence, test sensitivity, positive predictive value of a test) to improve understanding of risk estimates [28, 29].
There are two reasons why graphics are recommended to present the probability of harm: graphics lead to more risk aversion than numerical probability information alone, and graphics allow the observer to process information more effectively then when numbers are presented on their own . Research suggests that simple bar charts are preferable to more complex presentations of data . Because no single graphical format performs optimally in all situations , the different Decision boxes may use different formats to display information. We will prefer part-to-whole bar graphs to convey conditional probabilities, ratios, and proportions, as these graphs are believed to invoke automatic visual area processing and proportion judgments and help viewers attend to mathematical proportions .
Because the Decision boxes seek to prepare clinicians to communicate probabilities to their patients, we will supplement numerical and graphical information with a short narrative statement describing the absolute risk differences between the options available [29, 32].
Similarly to the Drug facts box (Dr Lisa Schwartz, personal communication), we will reserve the bottom of each Decision box to present confidence in the results, using an approach adapted from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for reporting the consistency of results, the indirectness of evidence, study limitations, and imprecision .
Phase 2: User-testing the Decision boxes
1. Focus groups with users
To explore users' perceptions of the content and format of the Decision boxes and to seek suggestions for improvement, we will conduct two focus groups with family physicians and four focus groups with patients (Figure 3). Physicians will be recruited by five members of the research team through their professional networks (ML, FL, MC, PP, RG). Eligible patients will be first identified by the healthcare professionals and support staff of the clinics of the network. Support staff will ask them for their permission to be contacted by a member of the research team who will then offer them to participate in the study.
To explore physicians' perceptions, we will present two Decision box prototypes to a purposeful sample of 16 family physicians distributed in two focus groups: a French-speaking group and an English-speaking group. Each focus group will comprise eight physicians. To explore patients' perceptions, we will present the same two Decision boxes to a purposeful sample of patients from each target population (the population of patients with the clinical condition addressed by the Decision box). We will use a maximum variation strategy to populate the samples , each of which will form a distinct focus group. Each Decision box will thus be presented to 16 patients of the target population (eight French-speaking and eight English-speaking patients, distributed in a French-speaking group and an English-speaking group), for a total of 32 patients in four focus groups.
To enrich our understanding of the group's views, we will collect demographic data from all participants at the beginning of each focus group. The focus groups with physicians will last approximately two hours; those with patients will last approximately one hour. The focus groups will be moderated by experienced interviewers (one for the French-speaking groups and one for the English-speaking groups). Two observers (AG, PP or ML) will take notes on the process and the content of the discussions so that participants can be identified when the interviews are transcribed . One observer (AG) will be present during every focus group discussion, to ensure consistency of approach. Each discussion will be audiotaped and professionally transcribed. Ethical approvals for this project were given by the research ethics committees of the Centre de Recherche du Centre Hospitalier Universitaire de Quebec (reference number #S10-12-114) and by McGill University (reference number #A12-E82-10B).
2. Testing the feasibility of the questionnaire for family physicians
Following their focus group, family physicians will be asked to pre-test a self-administered questionnaire to be used in a larger study on the implementation of the Decision boxes, to take place later. Pretesting will allow us to evaluate physicians' understanding of the content of the questionnaire and the feasibility of having the physicians complete the questionnaire under field conditions. The questionnaire will be made available in English and in French. It will collect the respondent's sociodemographic characteristics and will evaluate the respondent's interest in the clinical topic using a visual analogue scale ranging from "no interest" to "deep interest". The questionnaire will include the information sub-scale of the Decisional Conflict Scale , the Information Assessment Method , and a scale based on the Theory of Planned Behaviour to evaluate physicians' intention to use the Decision boxes in their practice and to share decisions equally with their patients . Open-ended questions on the relevance of the questions and respondents' ease of understanding will be added at the end. We will record the time needed for physicians to complete the questionnaire.
Two individuals (one research assistant and one researcher) will independently perform a hybrid deductive/inductive thematic qualitative data analysis of each focus group discussion using specialized software (NVivo 9) . A first analysis will be performed by a research assistant (one in French, one in English), and a second analysis will be performed by one of the researcher (AG). Any disagreements between the research assistants and the researcher will be discussed until consensus is reached. Having the same researcher perform the second analysis for all of the sites will ensure consistency. The coding scheme will be developed following the User Experience Model mentioned earlier  by identifying what users experienced as barriers or facilitators to the use of the Decision boxes and using this information to help clinicians share decisions with their patients. The deductive thematic analysis will apply attributes derived a priori from the seven facets of the User Experience Model : valuable, usable, credible, useful, desirable, accessible and findable. The inductive thematic analysis will integrate new themes, suggested by the data, into the scheme. We will compare the phenomena observed to emphasize a common tangent and will work out tree structures and matrices for the analysis. Other members of the research team will corroborate the findings by scrutinizing the analysis and ensuring that the new themes, tree structures and matrices are representative of the initial data analysis and codes assigned. We will triangulate key findings by referring to publications on the design of tools to improve the communication of risk [23, 42].
We will also perform descriptive statistical analyses of the answers to the questionnaire. To test the feasibility of the questionnaire, we will evaluate response rates and the time required to complete the questionnaire. We will also perform a descriptive analysis of respondents' comments about the relevance of the questions and how well they understood them.
This study will create a framework for the production of Decision boxes designed to facilitate decision making about preference-sensitive health interventions. It will produce 10 bilingual Decision boxes on common clinical topics in primary care, and it will generate robust evidence of family physicians' and patients' perceptions of the content and format of Decision box prototypes. This preliminary work is crucial to the implementation of Decision boxes in clinical practice.
The methodology proposed here has several strengths. First, we will ground our development of the Decision boxes in a thorough review of the rich evidence base on risk communication. Second, we will develop the boxes in both of Canada's official languages and have them tested by English-speakers and French-speakers alike. And third, we will have the boxes tested by both parties to SDM during the clinical encounter: that is, by both family physicians and patients. The limitations of our study lie in users' evaluation of only two of the 10 Decision boxes we will have developed, a limitation necessitated by the restricted availability of funds. The authors are confident, however, that many of the comments gathered on the two Decision boxes-namely, comments regarding the boxes' format-will apply to Decision boxes on any clinical topic.
We plan to conduct another study that evaluates the effect of the Decision boxes on the integration of evidence-based and SDM principles in practice. We hypothesize that exposing family physicians to a theory-based strategy  to implement Decision boxes will prime the physicians to better communicate the benefits and harms of the available options to their patients. This communication will cause patients to become more involved in decisions concerning their health. By helping to implement Decision boxes in clinical practice, the subsequent steps of this research program will impact not only the communication of scientific data to physicians, but also the communication between physicians and patients, leading physicians to make more judicious use of current best evidence when making clinical decisions together with their patients.
This work was supported by the Canadian Institutes for Health Research (CIHR), CIHR Institute of Genetics, the CIHR Institute of Health Services and Policy Research, and the APOGÉENet/CanGèneTest Research and Knowledge Network on Genetic Services and Policy (grant # ETG92250). The same network also provided AG with a postdoctoral training fellowship. We would like to thank Drs Lisa Schwartz and Steve Woloshin for their advice on the development and evaluation of the Decision box. We would also like to acknowledge writing assistance obtained from Ms. Jennifer Petrela.
- Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn't. Brit Med J. 1996, 312: 71-72.PubMed CentralView ArticlePubMed
- Coumou HC, Meijman FJ: How do primary care physicians seek answers to clinical questions? A literature review. J Med Libr Assoc. 2006, 94: 55-60.PubMed CentralPubMed
- Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S: Helping Doctors and Patients Make Sense of Health Statistics. Psychological Science in the Public Interest. 2007, 8: 53-96.PubMed
- Edwards A, Matthews E, Pill R, Bloor M: Communication about risk: diversity among primary care professionals. Fam Pract. 1998, 15: 296-300. 10.1093/fampra/15.4.296.View ArticlePubMed
- Ancker JS, Senathirajah Y, Kukafka R, Starren JB: Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc. 2006, 13: 608-618. 10.1197/jamia.M2115.PubMed CentralView ArticlePubMed
- Edwards A, Elwyn G, Covey J, Matthews E, Pill R: Presenting risk information - A review of the effects of "framing'' and other manipulations on patient outcomes. J HealthCommun. 2001, 6: 61-82.
- Charles C, Gafni A, Whelan T: Decision-making in the physician-patient encounter: revisiting the shared treatment decision-making model. Soc Sci Med. 1999, 49: 651-661. 10.1016/S0277-9536(99)00145-8.View ArticlePubMed
- O'Connor AM, Bennett CL, Stacey D, Barry M, Col NF, Eden KB, Entwistle VA, Fiset V, Holmes-Rovner M, Khangura S, Llewellyn-Thomas H, Rovner D: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2009, CD001431-3
- Evans R, Edwards A, Brett J, Bradburn M, Watson E, Austoker J, Elwyn G: Reduction in uptake of PSA tests following decision aids: systematic review of current aids and their evaluations. Patient Educ Couns. 2005, 58: 13-26. 10.1016/j.pec.2004.06.009.View ArticlePubMed
- Hack TF, Degner LF, Watson P, Sinha L: Do patients benefit from participating in medical decision making? Longitudinal follow-up of women with breast cancer. Psychooncology. 2006, 15: 9-19. 10.1002/pon.907.View ArticlePubMed
- Merenstein D, Diener-West M, Krist A, Pinneger M, Cooper LA: An assessment of the shared-decision model in parents of children with acute otitis media. Pediatrics. 2005, 116: 1267-1275. 10.1542/peds.2005-0486.View ArticlePubMed
- Perneger TV, Charvet-Berard A, Perrier A: Patient assessments of the most important medical decision during a hospitalization. J Gen Intern Med. 2008, 23: 1659-1665. 10.1007/s11606-008-0736-4.PubMed CentralView ArticlePubMed
- van Tol-Geerdink JJ, Leer JW, van Lin E, Schimmel EC, Huizenga H, van Daal WAJ, Stalmeier PFM: Offering a treatment choice in the irradiation of prostate cancer leads to better informed and more active patients, without harm to well-being. Int J Radiat Oncol Biol Phys. 2008, 70: 442-448. 10.1016/j.ijrobp.2007.06.058.View ArticlePubMed
- Duncan E, Best C, Hagen S: Shared decision making interventions for people with mental health conditions. Cochrane Database Syst Rev. 2010, CD007297-1
- Legare F, Labrecque M, Leblanc A, Njoya M, Laurier C, Cote L, Godin G, Thivierge RL, O'Connor A, St-Jacques S: Training family physicians in shared decision making for the use of antibiotics for acute respiratory infections: a pilot clustered randomized controlled trial. Health Expect. 2010,
- Mann DM, Ponieman D, Montori VM, Arciniega J, McGinn T: The Statin Choice decision aid in primary care: a randomized trial. Patient Educ Couns. 2010, 80: 138-140. 10.1016/j.pec.2009.10.008.View ArticlePubMed
- Whelan T, Levine M, Willan A, Gafni A, Sanders K, Mirsky D, Chambers S, O'Brien MA, Reid S, Dubois S: Effect of a decision aid on knowledge and treatment decision making for breast cancer surgery: a randomized trial. JAMA. 2004, 292: 435-441. 10.1001/jama.292.4.435.View ArticlePubMed
- Schwartz LM, Woloshin S, Welch HG: The drug facts box: providing consumers with simple tabular data on drug benefit and harm. Med Decis Making. 2007, 27: 655-662. 10.1177/0272989X07306786.View ArticlePubMed
- Schwartz LM, Woloshin S, Welch HG: Using a drug facts box to communicate drug benefits and harms: two randomized trials. Ann Intern Med. 2009, 150: 516-527.View ArticlePubMed
- Dicenso A, Bayley L, Haynes RB: Accessing pre-appraised evidence: fine-tuning the 5 S model into a 6 S model. Evid Based Nurs. 2009, 12: 99-101. 10.1136/ebn.12.4.99-b.View ArticlePubMed
- Haynes B: Of studies, syntheses, synopses, summaries, and systems: the "5S" evolution of information services for evidence-based healthcare decisions. Evid Based Nurs. 2007, 10: 6-7. 10.1136/ebn.10.1.6.View ArticlePubMed
- Elwyn G, O'Connor AM, Bennett C, Newcombe RG, Politi M, Durand MA, Drake E, Joseph-Williams N, Khangura S, Saarimaki A, Sivell S, Stiel M, Bernstein SJ, Col N, Coulter A, Eden K, Harter M, Rovner MH, Moumjid N, Stacey D, Thomson R, Whelan T, van der Weijden T, Edwards A: Assessing the quality of decision support technologies using the International Patient Decision Aid Standards instrument (IPDASi). PLoS One. 2009, 4: e4705-10.1371/journal.pone.0004705.PubMed CentralView ArticlePubMed
- Lipkus IM: Numeric, verbal, and visual formats of conveying health risk: Suggested best practices and future recommendations. Med Decis Mak. 2007, 27: 696-713. 10.1177/0272989X07307271.View Article
- Ancker JS, Kaufman D: Rethinking health numeracy: a multidisciplinary literature review. J Am Med Inform Assoc. 2007, 14: 713-721. 10.1197/jamia.M2464.PubMed CentralView ArticlePubMed
- Garcia-Retamero R, Galesic M: Who profits from visual aids: Overcoming challenges in people's understanding of risks. Soc Sci Med. 2010, 70: 1019-1025. 10.1016/j.socscimed.2009.11.031.View ArticlePubMed
- Covey J: A meta-analysis of the effects of presenting treatment benefits in different formats. Med Decis Making. 2007, 27: 638-654. 10.1177/0272989X07306783.View ArticlePubMed
- Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G: Medicine. Communicating statistical information. Science. 2000, 290: 2261-2262. 10.1126/science.290.5500.2261.View ArticlePubMed
- Gigerenzer G, Hoffrage U: How to Improve Bayesian Reasoning Without Instruction: Frequency Formats. Psychol Rev. 1995, 102: 684-704. 10.1037/0033-295X.102.4.684.View Article
- Visschers VH, Meertens RM, Passchier WW, de Vries NN: Probability information in risk communication: a review of the research literature. Risk Anal. 2009, 29: 267-287. 10.1111/j.1539-6924.2008.01137.x.View ArticlePubMed
- Edwards A, Elwyn G, Mulley A: Explaining risks: turning numerical data into meaningful pictures. Brit Med J. 2002, 324: 827-830. 10.1136/bmj.324.7341.827.PubMed CentralView ArticlePubMed
- Lipkus IM, Hollands JG: The visual communication of risk. J Natl Cancer Inst Monogr. 1999, 149-163.
- Gramling R, Irvin JE, Nash J, Sciamanna C, Culpepper L: Numeracy and medicine: key family physician attitudes about communicating probability with patients. J Am Board Fam Pract. 2004, 17: 473-10.3122/jabfm.17.6.473.View ArticlePubMed
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Brit Med J. 2008, 336: 924-926. 10.1136/bmj.39489.470347.AD.PubMed CentralView ArticlePubMed
- Patton M, (Ed.): Qualitative research and evaluation methods. 2002, Thousand Oaks, CA: Sage Publications, 3
- User experience Design. [http://www.semanticstudios.com/publications/semantics/000029.php]
- Rosenbaum SE, Glenton C, Nylund HK, Oxman AD: User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010, 63: 607-619. 10.1016/j.jclinepi.2009.12.013.View ArticlePubMed
- O'Connor AM: Validation of a decisional conflict scale. Med Decis Making. 1995, 15: 25-30.View ArticlePubMed
- Krueger R, Casey M: Focus groups: A practical guide for applied research. 2008, Pine Forge Pr
- Pluye P, Grad RM, Johnson-Lafleur J, Bambrick T, Burnand B, Mercer J, Marlow B, Campbell C: Evaluation of email alerts in practice: Part 2 - validation of the information assessment method. J Eval Clin Pract. 2010, 16: 1236-1243. 10.1111/j.1365-2753.2009.01313.x.View ArticlePubMed
- Ajzen I: The theory of planned behavior. Organ behav hum dec. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.View Article
- Fereday J, Muir-Cochrane E: Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods. 2006, 5: 80-
- Graham ID, Logan J, Bennett CL, Presseau J, O'Connor AM, Mitchell SL, Tetroe JM, Cranney A, Hebert P, Aaron SD: Physicians' intentions and use of three patient decision aids. BMC Med Inform Decis Mak. 2007, 7: 20-10.1186/1472-6947-7-20.PubMed CentralView ArticlePubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/11/17/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.