- Research article
- Open Access
- Open Peer Review
Barriers to the use of reminder/recall interventions for immunizations: a systematic review
BMC Medical Informatics and Decision Makingvolume 12, Article number: 145 (2012)
Although many studies have demonstrated the benefits of reminder/recall (RR) measures to address patient under-immunization and improve immunization coverage, they are not widely implemented by healthcare providers. We identified providers’ perceived barriers to their use from existing literature.
We conducted a systematic review of relevant articles published in English between January 1990 and July 2011 that examined the perceptions of healthcare providers regarding barriers to tracking patient immunization history and implementing RR interventions. We searched MEDLINE, PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, Academic Search Premier, and PsychINFO. Additional strategies included hand-searching the references of pertinent articles and related reviews, and searching keywords in Google Scholar and Google.
Ten articles were included; all described populations in the United States, and examined perceptions of family physicians, pediatricians, and other immunization staff. All articles were of moderate-high methodological quality; the majority (n=7) employed survey methodology. The most frequently described barriers involved the perceived human and financial resources associated with implementing an RR intervention, as well as low confidence in the accuracy of patient immunization records, given the lack of data sharing between multiple immunization providers. Changes to staff workflow, lack of appropriate electronic patient-tracking functionalities, and uncertainty regarding the success of RR interventions were also viewed as barriers to their adoption.
Although transitioning to electronic immunization records and registries should facilitate the implementation of RR interventions, numerous perceived barriers must still be overcome before the full benefits of these methods can be realized.
Although immunization is the most effective defense against vaccine preventable diseases, vaccine coverage is suboptimal in many populations[1–3]. Barriers to immunization have been well-studied and include anti-vaccination sentiments, difficulty accessing a healthcare provider (HCP) to provide immunization, public perception of a lack of endorsement by physicians, and, for non-publicly funded vaccines, cost. However, one of the most commonly cited reasons for low coverage is patients or their caregivers being unaware that one or more vaccines are due or overdue[4–8].
To address this prevalent issue, standalone and multi-faceted interventions have been developed to improve vaccine coverage, both to remind individuals of upcoming immunizations and to recall those for whom immunizations are overdue. Such interventions have been termed "reminder/recall" (RR) measures, and methods include automated or personal telephone calls, postcards, letters, and text messages to patients[9–13]. RR interventions can also be directed at immunization providers using processes such as reminders attached by nurses or receptionists to patient charts and, with the growth of electronic medical records (EMRs) and registries, through computerized alerts[14, 15]. Regardless of whom the RR systems are targeting, they have been demonstrated to increase vaccine coverage by 5-20%[16–18].
The benefits of RR interventions may also extend to data quality, since such measures may prompt patients to update other personal information, thereby leading to improved accuracy and completeness of the entire patient record. Furthermore, patients who are under-immunized often do not participate in other recommended preventive care activities; immunization-based RR can improve this by providing opportunities for HCP-patient interactions.
RR interventions start with the identification of patients who are due or overdue for immunization, involving either comparisons of electronic records with a system-embedded schedule or manual chart reviews where electronic records are not available. Although electronic systems should facilitate the efficient identification of under-immunized patients and those soon to be due, only 15-25% of physicians identify patients who are not up-of-date with vaccinations and implement an RR intervention[21–23]. It is important to understand why, despite the known effectiveness of RR measures and the increasing availability of this functionality associated with growing EMR use, these practices are not being adopted by immunization providers. We therefore conducted a systematic review to explore what providers perceive as barriers to the utilization of RR interventions.
Prior to conducting the review, our study team defined its parameters:
Study scope: Our search focused on summarizing the barriers identified by immunization providers (including physicians, nurses, and pharmacists) toward implementing either provider-directed RR interventions (measures to deliver reminders to immunization providers) or patient-directed interventions (client interventions initiated by immunization providers). Methodologies to elicit provider views may include surveys, focus groups, or interviews.
Search process: The systematic search retrieved articles using criteria developed based on input from the study team, keywords used in related previous reviews, and consultation with an information specialist.
Quality assessment: Each paper was evaluated using an amalgamation of previous tools to assess the rigour of methodology and the quality of reporting.
Barrier summary: We extracted the reported barriers and grouped them where possible, based on any noted commonalities.
Literature search and study selection
We searched MEDLINE, PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, Academic Search Premier, and PsychINFO for articles published in January 1990 through July 2011 on barriers to the use of immunization RR interventions, using various combinations of Medical Subject Headings terms (e.g., immunization, immunization programs, vaccination, appointment and schedules, reminder systems, parental notification) and keywords (e.g., recall, reminder, barrier, attitude, behavior, adopt). Search criteria were developed based on consultation with a library information specialist, as well as reviewing the keywords used in previous reviews of RR interventions. Full search criteria are described in Table1. Articles were considered eligible for evaluation if they were in English, contained original data, and described studies using quantitative and/or qualitative methodologies to identify the barriers perceived by immunization staff towards implementing RR interventions for childhood and/or adult immunizations. The perceived barriers could be towards any type of RR intervention for immunization directed at patients/their caregivers or healthcare providers. We excluded reviews, editorials, commentaries, and practice guidelines, as well as conference abstracts and other non-full text publications.
We entered the titles of relevant articles into the PubMed “related articles” feature and also hand-searched the bibliographies of review papers, relevant articles, and systematic reviews from the Cochrane Database of Reviews of Effectiveness to identify additional pertinent papers. We used Google Scholar and Google to search the grey literature for government reports and other documents.
After duplicate articles were removed, all titles and abstracts were reviewed for relevance (Figure1). Those articles with abstracts pertaining to our main objectives were then reviewed in full-text. JP conducted the primary review; a second reviewer (SQ) was consulted to verify the final list of articles, as well as to confer on all articles for which there was uncertainty as to whether inclusion criteria were met. A consensus on each of these articles was reached.
Methodological quality assessment
JP and SQ independently assessed the methodological rigour of all eligible articles based on a modification of tools designed to assess qualitative and quantitative studies (the Critical Appraisal Skills Programme (CASP) appraisal tool and Centre for Evidence-Based Management Survey Scoring System, respectively)[24, 25]. Articles were evaluated and classified as high/moderate/low quality based on the following 14 elements: clear research aims; appropriate research design; recruitment strategy; representativeness of population; avoidance of selection bias; data collection; sample size considerations; sufficient response rate; validity of measurements; ethical considerations; rigour of data analysis; significance testing; clarity of findings; and applicability of research. We scored one point for each element, for a maximum of 14; articles deemed to be of high methodological quality had ≥ 12 points, moderate = 8 – 11 points, and low = < 8 points. Rating disagreements were infrequent and were resolved through discussion.
A structured review form was then used by both reviewers to abstract data including country, study population, study methodology, and RR intervention(s) studied. The reviewers also independently abstracted any immunization providers’ perceived barriers towards the use of RR measures identified in each paper, and conferred to ensure consistency and completeness. Together, the reviewers then stratified these barriers where appropriate, based on commonalities.
The structured database searches yielded 1,774 titles. Eight articles were identified as meeting the inclusion criteria. Based on a review of the reference lists of pertinent papers, two additional articles were identified, for a total of 10.
Five of the 10 articles employed surveys[26–30], one used semi-structured interviews, two involved a series of focus groups[32, 33], and two described a mixed-methodology design (Table2)[23, 34]. All described studies in American populations.
Four articles examined the perceptions of either pediatricians solely or in addition to other HCPs[23, 28, 31, 32] while the remaining articles focused on family physicians and other staff (nurse practitioners, health informatics experts, other clinicians, etc.)[26, 27, 29, 30, 33, 34]. Six studies identified barriers to the use of RR interventions associated with specific immunization registries and information systems[25, 28–31, 34].
All articles were found to be of moderate (n=6) to high (n=4) methodological quality; there was 100% agreement between the two reviewers on this rating. All articles reported research aims, employed appropriate study design as well as recruitment strategies, utilized a suitable research design, provided sufficient data analysis, and described results with good applicability. While the majority of the other methodological elements examined were also well reported across the studies, only five studies described a satisfactory response rate for their quantitative or qualitative study.
Given the strong methodological quality of the 10 articles, we felt confident in including all in this review.
Although HCPs acknowledged the improvement in immunization coverage associated with RR interventions, they identified several concerns with implementing such measures in their practice settings. These perceived barriers have been grouped into broad themes below.
Financial and human resource constraints were consistently identified as barriers to RR implementation across the majority of articles. Tracking immunization histories of patients was considered very time-consuming, requiring concerted staff time and cost to complete. This was particularly true for those without electronic systems, thereby necessitating manual chart audits to identify those due or overdue for vaccinations[23, 26–28, 31, 32].
For computerized systems, the need for technical assistance, such as support for programming EMRs/registries with immunization algorithms to identify those eligible or overdue for certain vaccines, was also described as an added challenge to the implementation of RR measures[26, 27]. For such systems, human resource barriers also included the training needs of staff: respondents described a lack of knowledge regarding how to implement an RR intervention, and uncertainty regarding optimal designs, particularly for staff with limited computer skills. In one article, respondents expressed concern that using an RR measure could lead to the perception of unprofessionalism, should reminders be sent to patients who already have an immunization appointment scheduled for the near future. As a safeguard against this, a cross-check of the appointment schedules for under-immunized patients was added to the workflow prior to issuing reminders, which also added extra time. An additional identified barrier was the increased work load for already busy staff that may result from the appointments of patients identified through the RR process.
While financial costs were frequently identified as a barrier to RR implementation in the reviewed articles, they were only broadly discussed; costs related to increased human resources (administrative staff to implement tracking activities, immunization staff to carry out vaccinations, and technical staff to support the interventions) as well as upgraded system functionality, where the intervention was integrated into an electronic system, were described[23, 27, 31].
One of the most frequently mentioned barriers to using an RR intervention was the perception of a lack of reliable vaccination data on which to base reminders and recalls[28, 31, 32]. Identified reasons that a single provider would not have comprehensive immunization records included patients having recently moved, having no regular primary care physician and consequently seeing multiple HCPs, or seeking immunization from a different source such as a public health setting or a school-based clinic. The lack of integrated health systems to facilitate data sharing between multiple providers has resulted in concerns; even if an RR intervention was implemented, it may yield poor results, given that those who were recorded as being under-immunized at their "medical home" may have simply sought the vaccination from a separate provider[28, 32].
This perception of poor data quality also extended to contact information, resulting in the concern that the RR intervention would not reach the intended patient. This was also an issue where immunization information was recorded more than once (i.e., in charts as well as a registry); when neither are consistently updated, it is challenging to identify the best source of accurate patient information[31, 33].
Workflow barriers were identified as minor concerns associated with the implementation of RR, and included instances where the initiation of an RR activity involved new staff responsibilities, such as checking a state registry for patient immunization status or verifying information.
A workflow-related barrier to the use of computerized clinical alerts for immunization is the perception that pop-ups may be disruptive to the patient visit. Since the HCP may already have multiple health issues to address with the patient, the potential volume of such alerts may appear overwhelming, resulting in all messages being ignored.
Two articles described barriers to RR intervention implementation within specific systems and registries[31, 34]; these concerns likely apply to other electronic clinical systems. There was a perception that a single system may not reflect the appropriate immunization algorithms for all of its users, thereby rendering RR interventions ineffective.
Complicated systems such as those with different record formats for inpatients and outpatients, or certain high-risk groups, were also perceived to increase the time to implement a reminder alert comprehensively across all eligible patients.
Privacy issues associated with the system in use were described in one article: one-third of survey respondents expressed concerns regarding whether using tracking systems to identify patients due or overdue for immunizations and implementing an RR intervention may result in breaches of patient confidentiality.
Varying expectations of the utility of RR interventions were identified by two articles as barriers to its adoption[30, 31]. In one study that examined RR specifically for adolescent vaccination, users perceived that caregivers may not be receptive to interventions, perhaps viewing immunizations as less urgent for this age group, compared to younger cohorts.
The same article mentioned the belief that for other populations, an RR intervention should result in vastly improved coverage; if substantial benefits were not realized after initial use, unmet high expectations were identified as posing a potential barrier to the sustainability of the method.
Expectations about healthcare responsibilities also appear to affect willingness to adopt an RR intervention. In a study examining perceptions of RR adopters versus non-adopters, the latter group was less likely to consider all aspects of immunization delivery to be the responsibility of the healthcare system as a whole. There is no impetus to initiate RR interventions when HCPs believe that it is someone else’s duty to remember when a patient’s immunization is due.
Although monitoring of patient immunization records continues to expand as HCPs transition to electronic documentation, our review has demonstrated that there are many barriers to the implementation of RR interventions. While several of these issues relate to the perceived workflow changes associated with RR measures, most result from the current limitations surrounding immunization record-keeping.
The human resource requirements expected for RR interventions were identified as a significant barrier to their use in several of the articles reviewed; the perception that additional staff time is required to properly implement such measures is likely intensified by uncertainty over who should be implementing RR features, if anyone. Some HCPs may feel that parents/guardians respond best to notes sent from their child’s physician, others may perceive that measures to ensure appropriate immunization levels are best handled by public health, while still others view remembering immunizations are due as the responsibility of the patient or their caregiver. Efficient RR implementation requires coordination between physicians and local public health agencies to ensure that every individual is accounted for, including those without a primary care physician. Those HCPs with the view that an individual’s health is a shared responsibility across the health system rather than by a single provider seemed to have better acceptance of RR interventions, and perhaps a greater willingness to allocate staff time for their conduct.
Cost was also identified as a significant barrier to implementing RR methods. There are a range of interventions available including automated telephone and letter reminders to patients, based on manual chart audits or computerized systems programmed to determine the dates that immunizations are due and generate reminder messages accordingly. Previous studies have compared these methods by outcomes including financial resources required, finding that automated dialing is typically more cost-effective. However, this may depend on the start-up costs needed (for example, in the case of automated calls, the purchase of dialer equipment) and may vary based on target population[35–38]. Certain tools such as computerized reminders to alert the HCP that their patient is due for a vaccination may be a built-in functionality of the EMR system or immunization information system used, and therefore could potentially represent a cost-saving measure.
Optimal use of RR interventions begins with the accurate tracking of patient immunization information; where an electronic system is being used, identifying those individuals eligible for vaccination requires access to an electronic immunization schedule which has been integrated into that system. However, even where a single national vaccine schedule is recommended, implementation may vary locally. This issue is exacerbated in countries where every region is responsible for developing their own schedule and the variations between regions can be considerable; when EMR or other electronic systems are available nationally or are in use in more than one region, RR programming becomes complex. Additionally, if a practice adheres to even slightly different immunization schedules and age eligibilities than those specified in regional recommendations (and incorporated into the system in use), it will be challenging to quickly identify the patients that the practice defines as due or overdue for one or more vaccines. Improved communication between system developers, healthcare providers, and immunization program managers is vital to ensure that users have access to the appropriate schedules in their systems. The use of EMRs or immunization information systems with a region-specific immunization schedule is a key first step in initiation of RR activities if they are to be the responsibility of clinicians.
The studies we reviewed suggest that HCPs lack confidence in the accuracy and completeness of their patient immunization records. Immunization records can become fragmented between local public health departments (for school and non-school based immunizations), primary care physicians, and other HCPs who administer vaccines. Therefore the completeness of any one source of immunization information becomes suspect, limiting the value of any RR intervention based on these data. However, checking multiple places for a patient’s immunization documentation is impractical. A single regional immunization registry that is updated each time a vaccine is administered would be very beneficial. Based on the results of a systematic review, in 2010, the Task Force on Community Preventive Services at the United States Centers for Disease Control and Prevention recommended the use of immunization information systems as a means to increase vaccination rates, through numerous functionalities including RR interventions directed at clients and providers. Using an immunization registry as the source of "truth" would ensure that RR notices are based on complete, up-to-date information from all relevant sources[31, 32]. Additionally, since RR interventions will typically be initiated by the primary care provider, the transfer of information from registry to provider could facilitate the process. The benefits of a single source of vaccination information may extend to client safety; a recent study found that children with more than one immunization provider have a higher rate of over-immunization than those with one provider only.
Several limitations should be considered when interpreting the results of this review. It is possible that our search strategies may not have identified every pertinent article, particularly those only available in grey literature. Our restriction to English-language−only articles also may have excluded some relevant studies. Additionally, as all articles included in the final review were based on American populations, it is possible that our results may not extend to countries that differ significantly from the U.S. in terms of health care and immunization delivery. We were unable to identify a single tool that was appropriate for assessing quality of both quantitative and qualitative studies; as the one we developed and used for this review is an amalgamation of published resources, we are confident in its comprehensiveness, but acknowledge that it is not yet validated. Finally, we were unable to include a synthesis of results beyond a descriptive summary given the low yield of relevant articles combined with the nature of our research question.
Despite many routine childhood and adult vaccines being publicly funded for various risk groups in some regions, vaccine coverage is still sub-optimal[1–3]. Although RR interventions have been shown to be an effective means of improving coverage, our review has summarized several reasons that these measures have not been adopted in vaccination settings. Improved training and knowledge about RR interventions will help immunization staff address some of these issues. However, others require changes in data infrastructure, and are more challenging to overcome. Current immunization data collection processes are mainly piecemeal, eroding provider confidence in any one data repository being complete and accurate; to facilitate successful implementation of tracking functionalities, a single trusted source of up-to-date immunization data is required. Until such time that all providers are mandated to update their regional immunization registry for each administered vaccine, improved communication is required among those involved in immunization delivery, including public and private healthcare providers as well as registry and EMR developers, to ensure the comprehensiveness of data collected, system utility, and the incorporation of immunization schedule algorithms that correspond to prospective users’ jurisdictions. The use of RR interventions will be optimized and the benefits best realized when such modifications are made to address current challenges.
PCIRN Vaccine Coverage Theme Group members are: Julie Bettinger, Nicole Boulianne, Stephanie Brien, David Buckeridge, Larry Chambers, Natasha Crowcroft, Lois Crowe, Shelley Deeks, Michael Finkelstein, Maryse Guay, Jemila Hamid, Christine Heidebrecht, Faron Kolbe, Jeff Kwong, Allison McGeer, Jennifer Pereira, Susan Quach, Sherman Quan, Beate Sander, Chris Sikora, Anne-Luise Winter.
Centers for Disease Control and Prevention: US Vaccination Coverage Reported via NIS. 2011, http://www.cdc.gov/vaccines/stats-surv/nis/default.htm Accessed on Oct 1, 2012
Environics Research Group for the Public Health Agency of Canada: Canadian Adult National Immunization Coverage Survey. HC POR # POR-05-75:http://resources.cpha.ca/CCIAP/data/544e.pdf.,
Frescura AM, Belzak L: Results from the 2006 National Immunization Coverage Survey. Can J Infect Dis Med Microbiol. 2008, 19 (5): 357-
Mieczkowski TA, Wilson SA: Adult pneumococcal vaccination: a review of physician and patient barriers. Vaccine. 2002, 20 (9–10): 1383-1392.
Kimmel SR, Burns IT, Wolfe RM, Zimmerman RK: Addressing immunization barriers, benefits, and risks. J Fam Pract. 2007, 56 (2 Suppl): S61-S69.
Thomas M, Kohli V, King D: Barriers to childhood immunization: findings from a needs assessment study. Home Health Care Serv Q. 2004, 23 (2): 19-39. 10.1300/J027v23n02_02.
McCormick LK, Bartholomew LK, Lewis MJ, Brown MW, Hanson IC: Parental perceptions of barriers to childhood immunization: results of focus groups conducted in anurban population. Health Education Res. 1997, 12 (3): 355-362. 10.1093/her/12.3.355.
Johnson DR, Nichol KL, Lipczynski K: Barriers to adult immunization. Am J Med. 2008, 121 (7 Suppl 2): S28-S35.
Szilagyi PG, Schaffer S, Barth R, Shone LP, Humiston SG, Ambrose S: Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Arch Pediatr Adolesc Med. 2006, 160 (2): 157-163. 10.1001/archpedi.160.2.157.
Kellerman RD, Allred CT, Frisch LE: Enhancing influenza immunization. Postcard and telephone reminders and the challenge of immunization site shift. Arch Fam Med. 2000, 9 (4): 368-372. 10.1001/archfami.9.4.368.
Mason BW, Donnelly PD: Targeted mailing of information to improve uptake of measles, mumps, and rubella vaccine: a randomized controlled trial. Commun Dis Public Health. 2000, 3 (1): 67-68.
Smith DM, Zhou X, Weinberger M, Smith F, McDonald RC: Mailed reminders for area-wide influenza immunization: a randomized controlled trial. J Am Geriatr Soc. 1999, 47 (1): 1-5.
Ahlers-Schmidt CR, Chesser A, Hart T, Paschal A, Nguyen T, Wittler RR: Text messaging immunization reminders: feasibility of implementation with low-income parents. Prev Med. 2010, 50 (5–6): 306-307.
Szilagyi PG, Rodewald LE, Savageau J, Yoos L, Doane C: Improving influenza vaccination rates in children with asthma: a test of a computerized reminder system and an analysis of factors predicting vaccination compliance. Pediatrics. 1992, 90 (6): 871-875.
Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA: Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009, 124 (1): 159-169. 10.1542/peds.2008-2823.
Szilagyi PG, Bordley C, Vann JC, Chelminski A, Kraus RM, Margolis PA: Effect of patient reminder/recall interventions on immunization rates: a review. JAMA. 2000, 284 (14): 1820-1827. 10.1001/jama.284.14.1820.
Jacobson VJ, Szilagyi P: Patient reminder and patient recall systems to improve immunization rates. Cochrane Database Syst Rev. 2005, 3: 003941-
Guide to Community Preventive Services: Universally recommended vaccinations: client reminder & recall systems (abbreviated). http://www.thecommunityguide.org/vaccines/universally/clientreminder.html. Last updated: Feb 2008
Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF: A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999, 103 (1): 31-38. 10.1542/peds.103.1.31.
Hillestad H, Bigelow J, Bower A, Girosi F, Meili R, Scoville R: Can Electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff. 2005, 24 (5): 1103-1117. 10.1377/hlthaff.24.5.1103.
Szilagyi PG, Rodewald LE, Humiston SG, Hager J, Roghmann KJ, Doane C: Immunization practices of pediatricians and family physicians in the United States. Pediatrics. 1994, 94 (4 Pt 1): 517-523.
Schaffer SJ, Humiston SG, Shone LP, Averhoff FM, Szilagyi PG: Adolescent immunization practices: a national survey of US physicians. Arch Pediatr Adolesc Med. 2001, 155 (5): 566-571.
Tierney CD, Yusuf H, McMahon SR, Rusinak D, O'Brien M, Massoudi MS: Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics. Pediatrics. 2003, 112 (5): 1076-1082. 10.1542/peds.112.5.1076.
Public Health Resource Unit: 10 questions to help you make sense of qualitative research. http://www.sph.nhs.uk/sph-files/casp-appraisal-tools/Qualitative%20Appraisal%20Tool.pdf. Accessed on Jan 14, 2012
Center for Evidence-Based Management: Critical questions for a survey. http://www.cebma.org/wp-content/uploads/Critical-Appraisal-Questions-for-a-Survey.pdf. Accessed on Jan 15, 2012
Clark SJ, Cowan AE, Bartlett DL: Private provider participation in statewide immunization registries. BMC Public Health. 2006, 6: 33-10.1186/1471-2458-6-33.
Deutchman M, Brayden R, Siegel CD, Beaty B, Crane L: Childhood immunization in rural family and general practices: current practices, perceived barriers and strategies for improvement. Ambulat Child Health. 2000, 6 (3): 181-189. 10.1046/j.1467-0658.2000.00087.x.
Dombkowski KJ, Leung SW, Clark SJ: Provider attitudes regarding use of an immunization information system to identify children with asthma for influenza vaccination. J Public Health Manag Pract. 2007, 13 (6): 567-571.
Yarnall KS, Rimer BK, Hynes D, Watson G, Lyna PR, Woods-Powell CT: Computerized prompts for cancer screening in a community health center. J Amer Board of Fam Pract. 1998, 11 (2): 96-104.
Fung CH, Woods JN, Asch SM, Glassman P, Doebbeling BN: Variation in implementation and use of computerized clinical reminders in an integrated healthcare system. Am J Manag Care. 2004, 10 (11 Pt 2): 878-885.
Saville AW, Albright K, Nowels C, Barnard J, Daley MF, Stokley S: Getting under the hood: exploring issues that affect provider-based recall using an immunization information system. Acad Pediatr. 2011, 11 (1): 44-49. 10.1016/j.acap.2010.12.009.
Birmingham E, Catallozzi M, Findley SE, Vawdrey DK, Kukafka R, Stockwell MS: FluAlert: a qualitative evaluation of providers' desired characteristics and concerns regarding computerized influenza vaccination alerts. Prev Med. 2011, 52 (3–4): 274-277.
Humiston SG, Albertin C, Schaffer S, Rand C, Shone LP, Stokley S: Health care provider attitudes and practices regarding adolescent immunization: a qualitative study. Patient Educ Counsel. 2009, 75: 121-127. 10.1016/j.pec.2008.09.012.
Wallace C, Hatzakis M, Legro MW, Goldstein B: Understanding a VA preventive care clinical reminder: lessons learned. SCI Nurs. 2004, 21 (3): 149-152.
Franzini L, Rosenthal J, Spears W: Cost-effectiveness of childhood immunization reminder/recall systems in urban private practices. Pediatrics. 2000, 106 (Suppl 1): 177-183.
Suh CA, Saville A, Daley MF: Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012, 129 (6): e1437-e1445. 10.1542/peds.2011-1714.
Lieu TA, Black SB, Ray P: Computer-generated recall letters for underimmunized children: how cost-effective?. Pediatr Infect Dis J. 1997, 16: 28-33. 10.1097/00006454-199701000-00007.
Dini EF, Linkins RW, Sigafoos J: The impact of computer-generated messages on childhood immunization coverage. Am J Prev Med. 2000, 18 (2): 132-139. 10.1016/S0749-3797(99)00086-0.
Fiks AG: Designing computerized decision support that works for clinicians and families. Curr Probl Pediatr Adolesc Health Care. 2011, 41 (3): 60-88. 10.1016/j.cppeds.2010.10.006.
Centers for Disease Control and Prevention: Partners and organization support for IIS. http://www.cdc.gov/vaccines/programs/iis/resources-refs/partner-org-support.html. Created on May 15, 2012; accessed on July 10, 2012
Darden PM, Gustafson KK, Nietert PJ, Jacobson RM: Extra-immunization as a clinical indicator for fragmentation of care. Public Health Rep. 2011, 126 (Suppl 2): 48-59.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/12/145/prepub
The Canadian Association for Immunization Research and Evaluation provided networking assistance. We are also grateful for the assistance from the Library Information Specialist at Public Health Ontario, Beata Pach. This study was supported by an operating grant from the Public Health Agency of Canada and the Canadian Institutes of Health Research (IRR – 96974). Dr. Kwong was supported by a University of Toronto Department of Family and Community Medicine Clinician Scientist Award.
There are no competing interests to report.
All authors (JP, SQ, CH, SDQ, FK, MF and JCK) participated in the conception and design of this review. JP and SQ conducted the review. All authors were involved in either drafting the manuscript or providing revisions. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.