This article has Open Peer Review reports available.
Variability of the impact of adverse events on physicians’ decision making
© Cozmuta et al.; licensee BioMed Central Ltd. 2014
Received: 17 September 2013
Accepted: 28 May 2014
Published: 25 September 2014
Physicians frequently differ in their treatment recommendations. However, few studies have examined the reasons underlying these differences. The objective of this study was to examine whether physicians vary in the importance they attach to specific adverse events for two treatment options found in recent randomized controlled trials to have equivalent efficacy and overall toxicity.
A Max-Diff survey was administered to physicians attending a national scientific conference to quantify the influence of 23 specific adverse events on decision making related to two treatment options for vasculitis. This approach was chosen because it results in greater item discrimination compared to rating scales. We used Hierarchical Bayes modeling to generate the relative importance score for each adverse event and examined the association between physicians’ characteristics and the five most influential factors.
118 physicians completed the survey. The mean age (SD) was 48 years (10); 68% were male and 81% reported spending the majority of time in clinical practice. There was significant variability in the ratings of the relative importance of all adverse events, except those that were mild and easily reversible. We found a positive correlation between increasing physician age with ratings of sepsis (r = 0.29, p = 0.002) and opportunistic infection (r = 0.23, p = 0.016), and an inverse association between age with progressive multifocal leukoencephalopathy (r = - 0.28, p = 0.003). Physician sex, work setting, location, and number of patients with vasculitis seen per year were not associated with the influence of specific adverse events on decision making.
Our findings demonstrate that physicians differ substantially in how they perceive the importance of specific adverse events which may help explain observed unwarranted variability in physicians’ recommendations in clinical practice. Further efforts are needed to ensure that the reasons underlying variability in physicians’ recommendations are transparent.
Physicians often differ in their treatment recommendations. Extant reports of patients receiving significantly different recommendations by physicians practicing in different locations have been published both in the medical literature and the lay press. This variability persists even among experts who are well versed in the same literature as illustrated by the publication of conflicting guidelines by separate organizations . However, few studies have systematically examined the reasons underlying these differences. Variability in recommendations likely occurs more often under conditions of uncertainty where there are few high quality studies available to inform decisions . In these situations, indirect evidence is frequently relied upon to inform decision making in clinical practice. This is especially true for less common conditions, such as vasculitis.
Anti-neutrophil cytoplasmic antibodies (ANCA)-associated vasculitis (AAV) refers to a group of life- and organ-threatening diseases characterized by inflammation of small blood vessels, most commonly involving the skin, lungs, and kidneys. When vasculitis was first described, glucocorticoids were the only available treatment. The introduction of cyclophosphamide in the 1970’s resulted in significant improvements in overall survival of patients with vasculitis. However, cyclophosphamide is associated with significant toxicity including infection, infertility, and cancer, and physicians have long sought safer alternatives.
In 2010, two randomized controlled trials, Rituximab in ANCA-Associated Vasculitis (RAVE) and Rituximab versus Cyclophosphamide in ANCA-Associated Vasculitis (RITUXVAS) [3, 4], demonstrated that rituximab is as effective as cyclophosphamide at inducing remission in patients with moderate to severe disease. Moreover, there were no differences in the rate of adverse events between the two groups. These results provided physicians with an alternative treatment option for AAV for the first time in approximately 40 years.
Given the equivalent rates of remission and toxicity induced by both treatment regimens, the decision of which treatment to initiate (assuming the patient has no contraindications to either option) is likely strongly influenced by judgments related to the impact of specific adverse events. The RAVE trial was designed as a non-inferiority trial because physicians believed rituximab to be a potentially safer alternative. This belief was based on a large body of indirect evidence generated from randomized trials and observational studies of patients with different diseases or using different treatment regimens demonstrating a lower risk of infection associated with rituximab compared to cyclophosphamide [5–26]. Moreover, unlike cyclophosphamide, rituximab is not associated with a risk of infertility or cancer. However, the RAVE and RITUXVAS trials, which are the only head-to-head comparisons of cyclophosphamide and rituximab in any disease, demonstrated no significant differences in toxicity between the two treatment arms, including serious infection or rates of hospitalization. Several of the major toxicities of concern for cyclophosphamide, including ovarian failure and bladder cancer, usually manifest after either higher cumulative doses of cyclophosphamide or longer follow-up times than present in the trials under discussion. Despite these limitations, the RAVE and RITUXVAS trials led not only to the approval of rituximab for use in AAV by the US Food and Drug Administration and regulatory agencies in other countries, but also rapid widespread adoption of the drug’s use as an remission induction agent in AAV.
We administered a MaxDiff survey to a convenience sample of physicians attending the 2010 American College of Rheumatology National Scientific Meeting . The survey was designed using Sawtooth Software (Sawtooth Software SSI Web version 6.4) and included 23 adverse events associated with either RTX and/or cyclophosphamide. Participants were given the following instructions:
“On the following pages you will be asked to consider (given your knowledge about the severity and probability of each of the adverse events) how much each adverse event influences your decision about which medication to prescribe for a 50 year old post-menopausal woman with newly diagnosed severe ANCA-associated vasculitis. Assume the patient is hospitalized with pulmonary and renal involvement”. We described the patient as being a postmenopausal woman so as to eliminate fertility concerns, which are extremely variable and difficult to control for in a standardized scenario. The survey asked respondents to choose the most important item from 20 sets containing different combinations of five items from the master list of 23 adverse events (see example in Figure 1).
The survey design ensured that each item 1) was shown approximately an equal number of times, 2) was paired with each other item an equal number of times, and 3) appeared in each position an equal number of times. We also collected data on the following physician characteristics: age, sex, number of years of experience, work setting, and the number of patients with vasculitis seen per year.
We used Hierarchical Bayes modeling to generate the mean relative importance score for each adverse event. The relative importance scores sum to 100, and reflect the impact of each characteristic on physicians’ decision making. For each respondent, utilities (zero-centered values) were calculated for each level of each attribute using Hierarchical Bayes (HB) modeling (Sawtooth CBC/HB system for hierarchical Bayes estimation version 4.0). HB modeling has the advantage that it can better incorporate heterogeneity between respondents’ choices. In HB modeling, the sample averages (prior information) are used to update the individual utilities in a number of iterations until the sample averages stop changing between iterations. After this convergence, the cycle is run several thousand more times and the estimates of each iteration are saved and averaged. Raw scores were rescaled to a 0 to 100 on a ratio scale . The associations between physician characteristics and ratings were made using Pearson’s correlation coefficient and chi-square tests for continuous and categorical variables, respectively. This study qualified for exemption as determined by the Yale Human Investigation Committee.
One hundred and eighteen physicians completed the survey: 75 from North America and 22 from Europe. The mean age (SD) was 48 (10) years; 68% were male; 81% reported spending the majority of time in clinical practice; 39% work in an academic setting; 7% see fewer than 1 patient with AAV per year, 46% between 1 and 5 patients with AAV per year; 22% between 6 and 10 patients with AAV per year, and 25% see more than 10 patients with AAV per year.
Relative importance scores for each adverse event
Bone marrow suppression
Serious infusion reaction
Progressive multifocal leukoencephalopathy
Interstitial lung disease
Serious skin reaction
Mild infusion reaction
Mild skin reaction
The findings from this study demonstrate that physicians differ significantly in the impact that specific adverse events have on their decision making. With the exception of reversible and easily treatable adverse events, histograms showed wide distributions for all adverse events related to the two treatment options examined. Ratings were not related to clinical experience. However, we did find differences by age group, with physicians greater than 56 years old being more influenced by the risks of all infection-related adverse events, except for progressive multifocal leukoencephalopathy, than their younger counterparts. While the data do not explain the reasons underlying this pattern, it is possible that the differential impact between unspecified serious infections and progressive multifocal leukoencephalopathy is due to the relative recent attention focusing on the latter [31–33].
The RAVE  and RITUXVAS  head-to-head randomized controlled trials demonstrated no significant differences in the risk of infection between the two regimens. Despite these results, physicians ranked the risk of infection as the most important factor influencing their treatment decision. This finding was especially pronounced among older physicians. Physicians may undervalue risk estimates generated from randomized trials, because selective eligibility criteria, strict protocols, and short-term follow-up minimize adverse events. In addition, many of the risks associated with immunosuppressive medications are rare and would not be expected to occur in trials with relatively small numbers of patients. Small numbers and short duration of follow-up limit the usefulness of the toxicity data generated from head-to-head randomized controlled trials, however, the use of indirect data also has significant limitations. Most notably, the indirect data available for cyclophosphamide were generated from trials using higher doses of the drug and for much longer periods of time compared to the regimens currently used. This is particularly important because the most worrisome adverse events associated with cyclophosphamide (i.e. infection, infertility, and malignancy) are dose dependent.
There are several important limitations of this study. The use of standardized scenarios, enabled us to systematically vary patient characteristics, but this approach cannot replicate the complexity of decision making in clinical practice. In addition, although we surveyed a large number of physicians, most of whom spend the majority of their time in clinical practice, the study population represents a convenience sample of consecutively approached volunteers, which limits generalizability. Future research should examine whether differences in decision making predict differences in treatment choices made in clinical practice.
Ideally, despite physicians differing in their judgments, patients should all receive the same information describing potential treatment options and the reasons underlying variability in physicians’ recommendation should be transparent. While we did not measure actual treatment decisions, these findings suggest that physicians differ substantially in how they perceive the importance of specific adverse events which may help explain the frequently observed unwarranted variability in physicians’ recommendations.
Research reported in this publication was also supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases, part of the National Institutes of Health, under Award Number AR060231-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Dr. Fraenkel is supported by NIAMS K24 AR060231-01.
Dr. Merkel has conducted research partially funded by Genentech, Inc.
- Kachalia A, Mello MM: Breast cancer screening: conflicting guidelines and medicolegal risk. JAMA. 2013, 309: 2555-2556. 10.1001/jama.2013.7100.View ArticlePubMedGoogle Scholar
- GRADE Working Group: Grading quality of evidence and strength of recommendations. BMJ. 2004, 328: 1490-1497.View ArticlePubMed CentralGoogle Scholar
- Stone JH, Merkel PA, Spiera R, Merkel PA, Spiera R, Seo P, Langford CA, Hoffman GS, Kallenberg CGM, St Clair EW, Turkiewicz A, Tchao NK, Webber L, Ding L, Sejismundo LP, Mieras K, Weitzenkamp D, Ikle D, Seyfert-Margolis V, Mueller M, Brunetta P, Allen NB, Fervenza FC, Geetha D, Keogh KA, Kissin EY, Monach PA, Peikert T, Stegeman C, Ytterberg SR, Specks U, for the RAVE-ITN Research Group: Rituximab versus cyclophosphamide for ANCA-associated vasculitis. N Engl J Med. 2010, 363: 221-232. 10.1056/NEJMoa0909905.View ArticlePubMedPubMed CentralGoogle Scholar
- Jones RB, Tervaert JWC, Hauser T, Luqmani R, Morgan MD, Peh CA, Savage CO, Segelmark M, Tesar V, van Paassen P, Walsh D, Walsh M, Westman K, Jayne DRW, for the European Vasculitis Study Group: Rituximab versus cyclophosphamide in ANCA-associated renal vasculitis. N Engl J Med. 2010, 363: 211-220. 10.1056/NEJMoa0909169.View ArticlePubMedGoogle Scholar
- Anolik JH, Barnard J, Cappione A, Pugh-Bernard AE, Felgar RE, Looney RJ, Sanz I: Rituximab improves peripheral B cell abnormalities in human systemic lupus erythematosus. Arthritis Rheum. 2004, 50: 3580-3590. 10.1002/art.20592.View ArticlePubMedGoogle Scholar
- Balow JE: Choosing treatment for proliferative lupus nephritis. Arthritis Rheum. 2002, 46: 1981-1983. 10.1002/art.10466.View ArticlePubMedGoogle Scholar
- Boren EJ, Cheema GS, Naguwa SM, Ansari AA, Gershwin ME: The emergence of progressive multifocal leukoencephalopathy (PML) in rheumatic diseases. J Autoimmun. 2008, 30: 90-98. 10.1016/j.jaut.2007.11.013.View ArticlePubMedGoogle Scholar
- Boumpas DT, Austin HA, Balow JE, Vaughan EM, Yarboro CH, Klippel JH, Steinberg AD: Controlled trial of pulse methylprednisolone versus two regimens of pulse cyclophosphamide in severe lupus nephritis. Lancet. 1992, 340: 741-745. 10.1016/0140-6736(92)92292-N.View ArticlePubMedGoogle Scholar
- Cohen P, Pagnoux C, Mahr A, Arène JP, Mouthon L, Le Guern V, André MH, Gayraud M, Jayne D, Blöckmans D, Cordier JF, Guillevin L, French Vasculitis Study Group: Churg-Strauss syndrome with poor-prognosis factors: a prospective multicenter trial comparing glucocorticoids and six or twelve cyclophosphamide pulses in forty-eight patients. Arthritis Rheum. 2007, 57: 686-693. 10.1002/art.22679.View ArticlePubMedGoogle Scholar
- De Groot K, Harper L, Jayne DR, Flores Suarez LF, Gregorini G, Gross WL, Luqmani R, Pusey CD, Rasmussen N, Sinico RA, Tesar V, Vanhille P, Westman K, Savage CO, EUVAS (European Vasculitis Study Group): Pulse versus daily oral cyclophosphamide for induction of remission in antineutrophil cytoplasmic antibody-associated vasculitis. Ann Intern Med. 2009, 150: 670-680. 10.7326/0003-4819-150-10-200905190-00004.View ArticlePubMedGoogle Scholar
- De Groot K, Rasmussen N, Bacon PA, Tervaert JW, Feighery C, Gregorini G, Gross WL, Luqmani R, Jayne DR: Randomized trial of cyclophosphamide versus methotrexate for induction of remission in early systemic antineutrophil cytoplasmic antibody-associated vasculitis. Arthritis Rheum. 2005, 52: 2461-2469. 10.1002/art.21142.View ArticlePubMedGoogle Scholar
- Fauci AS, Haynes BF, Katz P, Wolff SM: Wegener’s granulomatosis: prospective clinical and therapeutic experience with 85 patients for 21 years. Ann Internal Med. 1983, 98: 76-85. 10.7326/0003-4819-98-1-76.View ArticleGoogle Scholar
- Faurschou M, Sorensen IJ, Mellemkjaer L, Loft AG, Thomsen BS, Tvede N, Baslund B: Malignancies in Wegener’s granulomatosis: incidence and relation to cyclophosphamide therapy in a cohort of 293 patients. J Rheumatol. 2008, 35: 100-105.PubMedGoogle Scholar
- Ginzler EM, Dooley MA, Aranow C, Ginzler EM, Dooley MA, Aranow C, Kim MY, Buyon J, Merrill JT, Petri M, Gilkeson GS, Wallace DJ, Weisman MH, Appel GB: Mycophenolate mofetil or intravenous cyclophosphamide for lupus nephritis. N Engl J Med. 2005, 53: 2219-2228.View ArticleGoogle Scholar
- Gunnarsson I, Sundelin B, Jonsdottir T, Jacobson SH, Henriksson EW, van Vollenhoven RF: Histopathologic and clinical outcome of rituximab treatment in patients with cyclophosphamide-resistant proliferative lupus nephritis. Arthritis Rheum. 2007, 56: 1263-1272. 10.1002/art.22505.View ArticlePubMedGoogle Scholar
- Houssiau FA, Vasconcelos C, D’Cruz D, Sebastiani GD, Garrido Ed Ede R, Danieli MG, Abramovicz D, Blockmans D, Mathieu A, Direskeneli H, Galeazzi M, Gül A, Levy Y, Petera P, Popovic R, Petrovic R, Sinico RA, Cattaneo R, Font J, Depresseux G, Cosyns JP, Cervera R: Immunosuppressive therapy in lupus nephritis: the Euro-Lupus Nephritis Trial, a randomized trial of low-dose versus high-dose intravenous cyclophosphamide. Arthritis Rheum. 2002, 46: 2121-2131. 10.1002/art.10461.View ArticlePubMedGoogle Scholar
- Jayne D: Review article: progress of treatment in ANCA-associated vasculitis. Nephrology. 2009, 14: 42-48. 10.1111/j.1440-1797.2009.01101.x.View ArticlePubMedGoogle Scholar
- Jayne DR, Rasmussen N: Treatment of antineutrophil cytoplasm autoantibody-associated systemic vasculitis: initiatives of the European community systemic vasculitis clinical trials study group. Mayo Clin Proc. 1997, 72: 737-747. 10.1016/S0025-6196(11)63594-5.View ArticlePubMedGoogle Scholar
- Merrill JT, Neuwelt CM, Wallace DJ, Shanahan JC, Latinis KM, Oates JC, Utset TO, Gordon C, Isenberg DA, Hsieh HJ, Zhang D, Brunetta PG: Efficacy and safety of rituximab in moderately-to-severely active systemic lupus erythematosus: the randomized, double-blind, phase II/III systemic lupus erythematosus evaluation of rituximab trial. Arthritis Rheum. 2010, 62: 222-233. 10.1002/art.27233.View ArticlePubMedPubMed CentralGoogle Scholar
- Mok CC, Ho CTK, Chan KW, Lau CS, Wong RWS: Outcome and prognostic indicators of diffuse proliferative lupus glomerulonephritis treated with sequential oral cyclophosphamide and azathioprine. Arthritis Rheum. 2002, 46: 1003-1013. 10.1002/art.10138.View ArticlePubMedGoogle Scholar
- Mukhtyar C, Flossmann O, Hellmich B, Bacon P, Cid M, Cohen-Tervaert JW, Gross WL, Guillevin L, Jayne D, Mahr A, Merkel PA, Raspe H, Scott D, Witter J, Yazici H, Luqmani RA, European Vasculitis Study Group (EUVAS): Outcomes from studies of antineutrophil cytoplasm antibody associated vasculitis: a systematic review by the European league against rheumatism systemic vasculitis task force. Ann Rheum Dis. 2008, 67: 1004-1010. 10.1136/ard.2007.071936.View ArticlePubMedGoogle Scholar
- Radis CD, Kahl LE, Baker GL, Wasko MC, Cash JM, Gallatin A, Stolzer BL, Agarwal AK, Medsger TA, Kwoh CK: Effects of cyclophosphamide on the development of malignancy and on long-term survival of patients with rheumatoid arthritis a 20-year followup study. Arthritis Rheum. 1995, 38: 1120-1127. 10.1002/art.1780380815.View ArticlePubMedGoogle Scholar
- Somers EC, Marder W, Christman GM, Ognenovski V, McCune WJ: Use of a gonadotropin-releasing hormone analog for protection against premature ovarian failure during cyclophosphamide therapy in women with severe lupus. Arthritis Rheum. 2005, 52: 2761-2767. 10.1002/art.21263.View ArticlePubMedGoogle Scholar
- Stasi R, Stipa E, Del Poeta G, Amadori S, Newland AC, Provan D: Long-term observation of patients with anti-neutrophil cytoplasmic antibody-associated vasculitis treated with rituximab. Rheumatology. 2006, 45: 1432-1436. 10.1093/rheumatology/kel098.View ArticlePubMedGoogle Scholar
- Talar-Williams C, Hijazi YM, Walther MM, Linehan WM, Hallahan CW, Lubensky I, Kerr GS, Hoffman GS, Fauci AS, Sneller MC: Cyclophosphamide-induced cystitis and bladder cancer in patients with Wegener Granulomatosis. Ann Intern Med. 1996, 124: 477-484. 10.7326/0003-4819-124-5-199603010-00003.View ArticlePubMedGoogle Scholar
- Tamimoto Y, Horiuchi T, Tsukamoto H, Otsuka J, Mitoma H, Kimoto Y, Nakashima H, Muta K, Abe Y, Kiyohara C, Ueda A, Nagasawa K, Yoshizawa S, Shimoda T, Harada M: A dose-escalation study of rituximab for treatment of systemic lupus erythematosus and Evans’ syndrome: immunological analysis of B cells, T cells and cytokines. Rheumatology. 2008, 47: 821-827. 10.1093/rheumatology/ken071.View ArticlePubMedGoogle Scholar
- van Herk H, Poortinga YH, Verhallen TMM: Response styles in rating scales. J Cross Cult Psychol. 2004, 35: 346-360. 10.1177/0022022104264126.View ArticleGoogle Scholar
- Flynn T, Louviere J, Peters T, Coast J: Best-worst scaling: what it can do for health care research and how to do it. J Health Econ. 2007, 26: 171-189. 10.1016/j.jhealeco.2006.04.002.View ArticlePubMedGoogle Scholar
- The MaxDiff/Web v6.0 Technical Paper:http://www.sawtoothsoftware.com/download/techpap/maxdifftech.pdf,
- Orme B: Hierarchical Bayes regression analysis: technical paper. 2003, Sequim: Sawtooth Software, Available from: http://www.sawtoothsoftware.com Google Scholar
- Post Marketing Reviews: Rituximab (marketed as Rituxan): Progressive Multifocal Leukoencephalopathy (PML). 2007, Available from: http://www.fda.gov/Drugs/DrugSafety/DrugSafetyNewsletter/ucm115974.htm Google Scholar
- Bharat A, Xie F, Baddley JW, Beukelman T, Chen L, Calabrese L, Delzell E, Grijalva CG, Patkar NM, Saag K, Winthrop KL, Curtis JR: Incidence and risk factors for progressive multifocal leukoencephalopathy among patients with selected rheumatic diseases. Arthritis Care Res. 2012, 64: 612-615. 10.1002/acr.21564.View ArticleGoogle Scholar
- Molloy ES: PML and rheumatology: the contribution of disease and drugs. Clevel Clin J Med. 2011, 78 (Suppl 2): S28-S32.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/14/86/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.