Impact of an interactive web tool on patients’ intention to receive COVID-19 vaccination: a before-and-after impact study among patients with chronic conditions in France
BMC Medical Informatics and Decision Making volume 21, Article number: 228 (2021)
In France, about 30% of the population refuses COVID-19 vaccination outright, and 9 to 40% are hesitant. We developed and evaluated an interactive web tool providing transparent and reliable information on the benefits and risks of COVID-19 vaccination.
The most recent scientific data at the time of the study were implemented into an interactive web tool offering individualized information on the risks of COVID-19 infection-related events versus vaccination-related serious adverse events. The tool was evaluated during a before-and-after impact study nested in ComPaRe, a French e-cohort of adult patients with chronic conditions. Primary outcome was the proportion of patients intending to receive vaccination after using the tool, among those not intending to receive it at baseline.
Between January 8 and 14, 2021, we enrolled 3152 patients in the study [mean age 55.2 (SD: 16.9), 52.9% women and 63% with ≥ 2 chronic conditions]. Before consulting the tool, 961 (30.5%) refused to be vaccinated until further data on efficacy/safety was obtained and 239 (7.5%) outright refused vaccination. Among these 1200 patients, 96 (8.0%, number needed to treat: 12.5) changed their mind after consulting the tool and would subsequently accept vaccination.
Our interactive web tool represents a scalable method to help increase the intent to receive COVID-19 vaccination among patients with chronic conditions and address vaccine hesitancy. Since April 2021, our tool has been embedded on the official webpage of the French Government for COVID-19 information.
Global vaccination against SARS-CoV-2 has raised hopes of putting an end to the current global COVID-19 pandemic. Yet, vaccine hesitancy, which is defined as a “delay in acceptance or refusal of vaccination despite availability of vaccination services” could compromise this endeavor . Vaccine hesitancy has been an issue in France long before the COVID-19 pandemic, and the country ranked among the ten least confident countries regarding vaccine safety in a survey conducted across 67 countries in 2016 . The vaccination campaign for COVID-19 in France is not spared from vaccine hesitancy, and a recent study showed that 30% of people in the working-age population would refuse COVID-19 vaccination outright, and 9 to 40% of them are hesitant . Common reasons for vaccine refusal and hesitancy include concerns about the vaccine safety and effectiveness, or a need for more information, especially given the fast development of the vaccines . More recently, concerns on the safety of two Adenovirus based vaccines (from AstraZeneca and Johnson & Johnson) may deter people who are already hesitant and give room for the spread of misinformation .
In this context, decision aids based on transparent and accurate scientific information could help physicians and patients better estimate the benefits and risks of vaccination, and make informed choices regarding vaccination.
In this study, we aimed at assessing the impact of an interactive web tool on COVID-19 vaccination's benefits and risks on the intention to receive vaccination among individuals with chronic conditions, at a very early stage of the vaccination campaign in France.
Our study comprised two stages: first, the development of an interactive web tool offering individualized information on the benefits and risks of COVID-19 vaccination, and second, the assessment of its impact on intention to receive vaccination during a before-and-after impact study.
Development of the tool
From December, 26th 2020 to January, 6th 2021, two epidemiologists and one primary care physician involved in the vaccination campaign (VTT, SS, PR) developed an interactive web tool that aimed at offering individualized information on the risks of death, hospitalization, symptom persistence at 2 months, in case of COVID-19 infection, with and without vaccination, and on the risks vaccination-related serious adverse events. The tool was based on the most recent scientific data available at that time and focused on evidence-based methods to counter antivaccination attitudes [6,7,8,9,10]. Its design was inspired by existing decision aids and its content followed the recommendations from the International Patient Decision Aid Standards (IPDAS) [11, 12]. Output of the tool could be personalized according to gender, age, and types of vaccine and used 10 000-person pictographs to illustrate the absolute risk reduction and the serious adverse effects associated with vaccination (Additional file 1). Three patients aged 33 to 78 years old pilot-tested and helped improve the tool’s clarity and wording before its release at https://cress-umr1153.fr/covid_vaccines/ (Fig. 1).
Assessment of the impact of the tool on patients’ intention to receive COVID-19 vaccination
We evaluated the impact of the tool on patients’ intention to receive COVID-19 vaccination during a before-and-after impact study nested in ComPaRe (Community of Patients for Research), a nationwide e-cohort of patients with chronic conditions, in France (www.compare.aphp.fr) .
Participants were adult patients (> 18 years old) reporting having at least one chronic condition, defined as any medical condition requiring healthcare for at least 6 months. Participants who reported a previous SARS-CoV2 infection (laboratory confirmed or not) were excluded from the study because, at the time of the study, they were not yet eligible for vaccination. Participants provided written informed consent before participation. ComPaRe was approved by the Institutional Review Board of Hôtel-Dieu Hospital, Paris (IRB: 0008367). Participants were not exposed to any form of harm as a result of their participation in this study.
Data collected and outcomes
Before consulting the tool, participants were asked to select their intention to receive COVID-19 vaccination from four response options (“Yes, with any vaccine,” “Yes, but not with all vaccines,” “No, I prefer to wait for more vaccine efficacy/safety data,” and “No, I don’t want to be vaccinated at all”). Then, they were presented the web interactive tool, embedded in the study questionnaire, and invited to test it. Finally, after consulting the tool, participants were immediately asked to reassess their intention to receive COVID-19 vaccination, using the same four response options as before.
The primary outcome was the proportion of patients intending to be vaccinated after using the tool, among those not intending so at baseline. Secondary outcomes included respondents’ perception of the tool's usefulness and of the importance of vaccination at the individual and population levels, assessed by rating scales ranging from 0 (not useful/important) to 100 (extremely useful/important).
Demographic and clinical characteristics of participants (i.e., age, sex, educational level, comorbidities, number of people living with the participant) were collected as part of the ComPaRe baseline data collection.
All analyses used a weighted dataset obtained by calibration on margins with weights for age, gender, and educational level derived from national census data describing the French population of patients with chronic conditions . We performed a post-hoc multivariable logistic regression, accounting for the weighted data, to investigate the association between change of mind and patients’ characteristics. Variables included in the model were sex, age (as a continuous variable), high educational level, household with > 1 adult (including the patient), household with ≥ 1 children, living with a person over 65 years old, living with a person having a chronic condition and responses to the “before” assessment. Analyses were performed on complete cases only. P < 0.05 was considered statistically significant.
All statistical analyses involved the use of R v3.6.3 (http://www.r-project.org, R Foundation).
From January 8 to 14, 2021, we sent an opt-in invitation to 20,940 members of the ComPaRe e-cohort and 3603 patients volunteered to participate (422 declined to participate). Finally, 3152 patients were included in the analyses (451 were further excluded because they reported a history of COVID-19 infection [n = 444], or did not complete the “after” questionnaire [n = 7]) (Fig. 2). In the weighted dataset, their mean age was 55.2 (SD: 16.9) and 1666 (52.9%) were women. About 63% of patients had multimorbidity (the mean number of conditions was 2.8 (SD: 2.7). Patients’ conditions included, among others, high blood pressure (n = 642, 20.4%), diabetes (n = 412, 13.1%); asthma (n = 151, 4.8%), cancer (n = 263, 8.3%), and osteoarthritis (n = 205, 6.5%). Complete characteristics of participants are reported in the Table 1.
Before consulting the tool, 1570 (49.8%) patients stated they would accept COVID-19 vaccination by any vaccine, 382 (12.1%) would accept at least one vaccine, 961 (30.5%) would refuse vaccination without more vaccine efficacy/safety data, and 239 (7.5%) would outright refuse vaccination. Patients’ perceptions of the importance of vaccination at the individual and population levels reflected their intention to receive vaccination but patients refusing to receive vaccination usually rated the importance of vaccination at population level higher than at individual level (Additional file 2).
Among the patients who did not intend to be vaccinated at baseline (n = 1200 in the weighted data), 8.0% (n = 96 in the weighted data, Number needed to treat = 12.5) changed their mind after consulting the tool and reported that they would accept vaccination. These patients were mostly patients who indicated that they would refuse vaccination unless they had further data on vaccine efficacy and/or safety (97%, n = 93 in the weighted data). Patients’ characteristics significantly associated with a change of mind were (1) responding that they required further data on vaccine efficacy and/or safety (Odds Ratio 9.49, 95% confidence interval [CI] 3.00 to 29.96) and (2) high educational level (Odds Ratio 1.74, 95% CI 1.01 to 2.99) (Additional file 3).
Patients who did not intend to be vaccinated at baseline found the tool moderately useful (mean ratings of 53.4 [SD = 1.8] and 35.6 [3.5] by patients asking for further data on vaccine efficacy and/or safety and by patients outright refusing vaccination, respectively). Among these patients, the tool slightly increased patients’ ratings of the importance of vaccination at individual level (mean increase of 4.0 [1.4] and 0.9 [0.4] points out of 100, for patients asking for further data on vaccine efficacy and/or safety and for patients outright refusing vaccination, respectively). However, the increase of their ratings of the importance of vaccination at population level was limited (mean increase of 2.3 [1.1] and 0.4 [1.2], respectively) (Additional file 4).
An interactive web tool presenting individualized information about the benefits and risks of COVID-19 vaccination moderately increased intention to be vaccinated among individuals with chronic conditions who initially intended to decline. As of the 28th of June 2021, about 33 million people in France (out of 64 million) received a first injection . Yet, similar to other western countries, a glass ceiling begins to be reached and waiting lines are getting shorter in many vaccination centers. Persuading hesitant persons to receive vaccination is therefore of importance to achieve herd immunity.
Our tool was designed as a standalone, scalable, and effective method to address the need for transparent communication to help people make informed decisions based on reliable scientific data. The number needed to treat (i.e., the number of patients unsure or refusing to receive vaccination who, after consulting the tool, change their minds) was 12.5. Since the tool was released, more than 174,000 people have accessed it. The tool has been disseminated by the media in France and embedded on the official webpage of the French Government for COVID-19 information, since April 2021 . It was enriched with data from Astra Zeneca’s and Johnson and Johnson’s vaccines and specific information regarding the risk of blood clots with low platelets have been added when users select the vaccine from Astra Zeneca [17,18,19].
Participants who changed their mind after consulting our tool were mainly those who asked for more information about the efficacy and safety of COVID-19 vaccination before seeing the tool. There were only 3 patients who outright refused vaccination changed their mind after consulting the tool. Similar to our results, other studies showed that outright vaccination refusal was strongly associated with a lower perceived severity of COVID-19 if infected, and that only a negligible proportion of those people would accept vaccination even if presented with an ideal vaccine in terms of efficacy and safety [3, 20]. This underlines that classic strategies to counter antivaccination attitudes (e.g., presenting data of the risks of not vaccinating) will not be effective in this population and that different strategies, such as incentivizing vaccination or complex combined strategies, should be considered .
Strengths of the study included the use of a before-and-after design, that was adapted for demonstrating the immediate impacts of the tool . Indeed, as the before and after assessments were at the same time, it was unlikely that factors other than consulting the tool would have influenced patients’ intention to receive vaccination, thus limiting the biases inherent in this type of design. Second, we used statistical methods to enhance the representativeness of estimates and account for the sampling bias associated with the use of an online cohort of patients.
Our study was limited by several factors. First, the response rate was low, potentially owing to the very short recruitment period (7 days). Second, the tool was evaluated by itself, rather than during medical visits when clinicians could address patients’ questions and fears to enable shared decision-making in a person-centered approach. Our tool is only a necessary but not sufficient part of a shared decision-making intervention. Third, the tool required access to the Internet, which limited its impact on people with low literary or on those not comfortable engaging with online surveys. Fourth, our tool was evaluated before the recent concerns about a potential link between rare adverse effects and vaccination (e.g., blood clots and Adenovirus-based vaccines or myocarditis and mRNA vaccines) [5, 23]. Finally, our study highlighted that most patients who changed their minds were vaccine hesitant. For patients who outright refused vaccination, mistrust roots deeper than the lack of information on vaccine’s efficacy or benefits and is often connected with defiance towards the government or by deeper fears, unlikely to be modified by an evidence-based online intervention . In this population, the use of decision aids during clinical encounters needs to be complemented by multi-faceted strategies based on restoring trust in healthcare providers and health authorities, as well as motivational interviewing during visits that best addresses patients’ values.
We developed a simple and scalable interactive web tool on the benefits and risks of COVID-19 vaccination. For every 12.5 person unsure or refusing to receive vaccination, consulting our tool convinced one person to receive vaccination.
Availability of data and materials
All data collected for the study, including individual participant data and a data dictionary is available for research purposes, under the rules of the ComPaRe e-cohort described here: https://compare.aphp.fr. Please contact the corresponding author for further information.
MacDonald NE. Vaccine hesitancy: definition, scope and determinants. Vaccine. 2015;33(34):4161–4.
Larson HJ, de Figueiredo A, Xiahong Z, Schulz WS, Verger P, Johnston IG, Cook AR, Jones NS. The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine. 2016;12:295–301.
Schwarzinger M, Watson V, Arwidson P, Alla F, Luchini S. COVID-19 vaccine hesitancy in a representative working-age population in France: a survey experiment based on vaccine characteristics. Lancet Public Health. 2021;6:e210–21.
Fisher KA-O, Bloomstone SJ, Walder JA-O, Crawford SA-O, Fouayzi HA-O, Mazor KM. Attitudes toward a potential SARS-CoV-2 vaccine: a survey of U.S. adults. Ann Intern Med. 2020;173:964–73 (1539–3704 (Electronic)).
Cines DB, Bussel JB. SARS-CoV-2 vaccine-induced immune thrombotic thrombocytopenia. N Engl J Med. 2021;384:2254–6.
Salje H, Tran Kiem C, Lefrancq N, Courtejoie N, Bosetti P, Paireau J, Andronico A, Hozé N, Richet J, Dubost CL, et al. Estimating the burden of SARS-CoV-2 in France. Science (New York, NY). 2020;369(6500):208–11.
Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, Pérez Marc G, Moreira ED, Zerbini C, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603–15.
Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, Diemert D, Spector SA, Rouphael N, Creech CB, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2020;384:403–16.
Sudre CH, Murray B, Varsavsky T, Graham MS, Penfold RS, Bowyer RC, Pujol JC, Klaser K, Antonelli M, Canas LS, et al. Attributes and predictors of long COVID. Nature Med. 2021;27:626–31.
Horne Z, Powell D, Hummel JE, Holyoak KJ. Countering antivaccination attitudes. Proc Natl Acad Sci U S A. 2015;112(33):10321–4.
Weymiller AJ, Montori VM, Jones LA, Gafni A, Guyatt GH, Bryant SC, Christianson TJ, Mullan RJ, Smith SA. Helping patients with type 2 diabetes mellitus make treatment decisions: statin choice randomized trial. Arch Intern Med. 2007;167(10):1076–82.
Elwyn G, O’Connor A, Stacey D, Volk R, Edwards A, Coulter A, Thomson R, Barratt A, Barry M, Bernstein S, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ Clin Res Ed. 2006;333(7565):417.
Tran VT, Ravaud P. COllaborative open platform E-cohorts for research acceleration in trials and epidemiology. J Clin Epidemiol. 2020;124:139–48.
Direction de la recherche dé, de l’évaluation et des statistiques.: L’état de santé de la population en France—RAPPORT 2017. In: Paris: Ministère des Solidarités et de la Santé—République Française; 2017.
VaccinTracker: Combien de français ont été vaccinés contre la Covid19? https://covidtracker.fr/vaccintracker/.
Quels sont les bénéfices et risques des vaccins contre la Covid-19? https://www.gouvernement.fr/quels-sont-les-benefices-et-risques-des-vaccins-contre-la-covid-19.
Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, Aley PK, Angus B, Baillie VL, Barnabas SL, Bhorat QE, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet (London, England). 2020;397:99–111.
Sadoff J, Gray G, Vandebosch A, Cárdenas V, Shukarev G, Grinsztejn B, Goepfert PA, Truyers C, Fennema H, Spiessens B, et al. Safety and efficacy of single-dose Ad26.COV2.S vaccine against Covid-19. N Engl J Med. 2021;384:2187–201.
AstraZeneca’s COVID-19 vaccine: benefits and risks in context. https://www.ema.europa.eu/en/news/astrazenecas-covid-19-vaccine-benefits-risks-context.
Kaplan RM, Milstein A. Influence of a COVID-19 vaccine’s effectiveness and safety profile on vaccination acceptance. Proc Natl Acad Sci U S A. 2021;118(10):e2021726118.
Wilf-Miron R, Myers V, Saban M. Incentivizing vaccination uptake: the “Green Pass” proposal in Israel. JAMA. 2021;325(15):1503–4.
Sedgwick P. Before and after study designs. BMJ Clin Res Ed. 2014;349:g5074.
Kim HW, Jenista ER, Wendell DC, Azevedo CF, Campbell MJ, Darty SN, Parker MA, Kim RJ. Patients with acute myocarditis following mRNA COVID-19 vaccination. JAMA Cardiol. 2021. https://doi.org/10.1001/jamacardio.2021.2828.
COCONEL Group. A future vaccination campaign against COVID-19 at risk of vaccine hesitancy and politicisation. Lancet Infect Dis. 2020;20(7):769–70.
The authors thank Elise Diard, Isabelle Pane and Carolina Riveros for work in ComPaRe and Jo Ann Cahn for editing the manuscript.
No specific funding was obtained for this work.
Ethics approval and consent to participate
Participants provided written informed consent before participation. ComPaRe was approved by the Institutional Review Board of Hôtel-Dieu Hospital, Paris (IRB: 0008367).
Consent for publication
The manuscript contains no details, images, or videos relating to an individual person.
The authors declare no competing interests and no financial associations that may be relevant or seen as relevant to the submitted manuscript. The authors have no association with commercial entities that could be viewed as having an interest in the general area of the submitted manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Data used to develop the tool.
Patients’ perception of the importance of vaccination before consulting the tool.
Demographic and clinical characteristics of patients changing their minds towards COVID-19 vaccination after consulting the tool. 3b: Association between participants’ characteristics and change of mind after consulting the tool.
Patients’ perceptions of the tool’s usefulness and of the importance of vaccination at individual and population level after using the tool.
About this article
Cite this article
Tran, VT., Sidorkiewicz, S., Péan, C. et al. Impact of an interactive web tool on patients’ intention to receive COVID-19 vaccination: a before-and-after impact study among patients with chronic conditions in France. BMC Med Inform Decis Mak 21, 228 (2021). https://doi.org/10.1186/s12911-021-01594-8