- Research article
- Open Access
Scoping review of toolkits as a knowledge translation strategy in health
BMC Medical Informatics and Decision Making volume 14, Article number: 121 (2014)
Significant resources are invested in the production of research knowledge with the ultimate objective of integrating research evidence into practice. Toolkits are becoming increasingly popular as a knowledge translation (KT) strategy for disseminating health information, to build awareness, inform, and change public and healthcare provider behavior. Toolkits communicate messages aimed at improving health and changing practice to diverse audiences, including healthcare practitioners, patients, community and health organizations, and policy makers. This scoping review explores the use of toolkits in health and healthcare.
Using Arksey and O’Malley’s scoping review framework, health-based toolkits were identified through a search of electronic databases and grey literature for relevant articles and toolkits published between 2004 and 2011. Two reviewers independently extracted data on toolkit topic, format, target audience, content, evidence underlying toolkit content, and evaluation of the toolkit as a KT strategy.
Among the 253 sources identified, 139 met initial inclusion criteria and 83 toolkits were included in the final sample. Fewer than half of the sources fully described the toolkit content and about 70% made some mention of the evidence underlying the content. Of 83 toolkits, only 31 (37%) had been evaluated at any level (27 toolkits were evaluated overall relative to their purpose or KT goal, and 4 toolkits evaluated the effectiveness of certain elements contained within them).
Toolkits used to disseminate health knowledge or support practice change often do not specify the evidence base from which they draw, and their effectiveness as a knowledge translation strategy is rarely assessed. To truly inform health and healthcare, toolkits should include comprehensive descriptions of their content, be explicit regarding content that is evidence-based, and include an evaluation of the their effectiveness as a KT strategy, addressing both clinical and implementation outcomes.
Each year, considerable human and financial resources are devoted to the development of research knowledge in various sectors of health (e.g., clinical, biomedical, health services, population health). The ultimate objective on the research continuum is to integrate research evidence into practice to inform and improve health outcomes. To this end, efforts in the rapidly developing field of knowledge translation (KT) have focused on bridging research to practice. Knowledge translation efforts in health strive to ensure that knowledge users are aware of and use research evidence to inform health and healthcare decision-making . In other words, knowledge users need to access research evidence and know what to do with the knowledge shared in order to impact health and clinical care, whether through increased awareness, sharing knowledge, or facilitating behavior/practice change.
Effectiveness of KT strategies
A range of KT strategies have been used with varied effectiveness to increase awareness and utilization of research evidence in health, including but not limited to printed educational materials (e.g., guidelines for practice, audio-visual materials, electronic publications, and multifaceted combinations of these elements in the form of toolkits), educational meetings (e.g., conferences, workshop, lectures), educational outreach (or academic detailing, which supports change through one-on-one engagement between expert and practitioner), local opinion leaders (providers who are educationally influential), audit and feedback (any summary of clinical performance of healthcare to change behavior over a specified period of time), and reminders (patient or encounter specific prompts). KT strategies can also be tailored, taking into account identified barriers to change, and/or multifaceted, which capture interventions having two or more components . Despite over 300 systematic reviews of professional behavior change conducted by Cochrane Effective Practice and Organisation of Care Group (EPOC), evidence on the effectiveness of different strategies remains incomplete .
Most of the reviews included in Bero et al.’s systematic review identified modest improvements in performance resulting from these common KT interventions . Generally, consistently effective interventions include reminders, multifaceted interventions, and interactive educational meetings. KT strategies having variable effectiveness include audit and feedback, local opinion leaders, local consensus processes, and patient mediated interventions. KT strategies with little or no effect included educational materials and didactic education meetings. Multifaceted interventions appear to be more effective than single KT interventions ,, although others have shown that single KT strategies can be as effective at changing knowledge (but not practice) as multifaceted ones when they utilized tailored and targeted messaging .
The vision of mapping which KT strategies are effective for which audiences and in what contexts remains elusive. The field is in need of more detailed guidance on how to best share research evidence in a way that promotes and enables its use. Some of this guidance is emerging from taxonomies of KT strategies developed recently to facilitate greater consistency in strategy naming, definition, level of detail and specification of the target user in research on KT strategy effectiveness, with the ultimate goal of enhancing the synthesis and application of findings across a body of research ,. Context appears to be a defining feature in KT for practice change, and it is highly likely that no single KT strategy will prove to be universally effective across contexts. Recent developments in the KT field have identified a range of contextual factors that are implicated in successful practice change  and this emerging evidence can shape how we explore the effectiveness of KT strategies to maximize their impact. On the practice side, organizational efforts to integrate research findings into practice are on the rise in a range of practice settings yet such efforts often do not include an evaluation component to identify whether the strategy met the intended KT goal. Indeed, both research and practice change endeavors rarely explore both clinical and implementation outcomes stemming from practice change initiatives, a situation that lends itself to a Type III error. Practice change initiatives need to evaluate the fidelity with which clinical interventions are implemented in order to accurately discern whether a failure to demonstrate intended clinical outcomes is due to poor implementation or weaknesses inherent in the clinical intervention itself .
Educational meetings and materials
Educational meetings and materials tend to be the most widely used KT strategies in health and healthcare ; they are simple and well known, incur the least costs, and are feasible to use in many contexts. However, despite wide spread use, research on the effectiveness of educational materials and meetings is inconclusive. This may be due to differences in the characteristics of the educational materials, including their attractiveness, content, format, mode of delivery, timing, frequency, and complexity of targeted behaviour change or other KT goal . A Cochrane systematic review on the use of educational materials to change the practice of healthcare professionals and patient health outcomes showed that 98% of the studies compared the use of educational materials to no intervention, making it difficult to ascertain the effectiveness of printed educational materials relative to other KT strategies . Overall, printed educational materials were found to have a small positive effect on professional practice outcomes and an unknown effect on patient outcomes, as 91% of the studies did not report on patient outcomes. Although educational materials are widely used, we still lack primary research on their effectiveness, and how their content, format and delivery can be optimized to increase their effectiveness. Given that educational materials are, and will likely continue to be widely used to share knowledge and change practice, optimizing their use should be a priority in the health field.
Educational materials are often packaged as “toolkits”. Coined in the 1980’s, the term ‘toolkit’ has been widely used for decades. As in other fields, no definitive toolkit format has been established in the context of health. The term has been used to describe the bundling of a combination of educational materials including templates, instruction sheets, literature reviews, videos, and posters, presented in a variety of formats (hard copy, web). Toolkits have been used to inform and improve health behaviors for diverse audiences, including health practitioners, patients, community and health organizations, policy makers, and for the public. Toolkits have gained popularity as a KT strategy, particularly in health, and the Internet has provided fertile ground for toolkit dissemination. For example, a recently completed study of evidence-based interventions to increase booster seat use lends itself to dissemination in a web-based toolkit format for multiple target audiences that are highly likely to access information on the Web [Barwick, Bruce, Fuselli, Stein, & Barac: Impact and sustainability of targeted booster seat activities among community organizations in injury prevention, submitted].
Given the pervasiveness of toolkits to disseminate health evidence and change practice, it is important to map the evidence for their effectiveness to inform decisions about their potential use. To this end, the present study focuses on toolkits as a KT strategy. We conducted a scoping review of published and grey literature to identify how toolkits have been operationalized, and to describe their content, evidence underlying their content, format, and evidence of effectiveness. Our focus was on toolkits that were developed to disseminate health information, as well as change healthcare provider behavior (i.e., both health- and healthcare-focused). As is the purview of scoping reviews, we sought to assess whether sufficient research is available to warrant a more traditional systematic review on health toolkits and, where gaps are noted, to issue recommendations for future research on toolkit development and effectiveness. This type of review is appropriate given the emerging nature of evidence on toolkits and the absence of syntheses on this topic .
The review followed Arksey and O’Malley scoping review framework  and the PRISMA flow diagram for reporting standards in systematic reviews and meta-analyses . Arskey and O’Malley’s framework includes five key phases: (i) identifying the research question; (ii) identifying relevant studies; (iii) study selection; (iv) charting the data; and (v) collating, summarizing and reporting the results. Ethics approval was exempt for the present study because no data collection was required.
While no single definition of the term toolkit has gained wide acceptance, for the purposes of this review ‘toolkit’ was defined as the packaging of multiple resources that codify explicit knowledge, such as templates, pocket cards, guidelines, algorithms, summaries, and that are geared to knowledge sharing, educate, and/or facilitate behavior change. For example, How Schools Can Help Students Recover from Traumatic Experiences is an online toolkit designed to support schools that want to help students recover from traumatic experiences such as natural disasters, exposure to violence, abuse or assault, terrorist incidents, war and refugee experiences . The toolkit lists existing programs for each trauma type, providing program goal, program delivery and implementation requirements to facilitate the comparison of different programs available to schools having this goal.
The review sought to assess the scope of toolkits available in the field of health and healthcare, so search terms were relatively broad. Electronic databases and websites were searched by a library and information scientist. The research team compiled a broad list of terms pertinent to health and healthcare. These terms were used by the library scientist in various combinations to search electronic databases of peer-reviewed and grey literature. Searches were limited to English language publications from 2004 to 2011. Papers were included if they reported on toolkits used for disseminating health information and informing and/or changing healthcare provider behavior. Given the goal of mapping how toolkits have been used as a KT strategy, study design was left unspecified. The following electronic databases were searched: MEDLINE, PsychINFO, CINAHL, ERIC and those available in Scholar’s Portal. Common terms across all databases included health education and health promotion. Each database contained unique relevant terms used to create search strings. The term ‘toolkit’ was included as a keyword as it is not used in the controlled vocabulary for these databases. The following headings were used in combination with ‘tool kit’ or ‘toolkit’ to create search strings: health education, health promotion, consumer health information, preventative health services, information resources, child safety, passenger safety, patient safety, health programs and educational programs.
Grey literature was searched using Scirus (now retired), a scientific search engine that searches journals as well as websites. The search terms included toolkit or tool kit, health, child, pediatric, youth. Searches were limited to the following subject areas: medicine, psychology, social and behavioral sciences and sociology.
Source selection was carried out in two stages. Figure 1 illustrates the process of searching and selecting toolkit publications and web-based material. In stage 1, abstracts for all material identified through the search process were located. Two reviewers independently reviewed each title and abstract and consensus was sought on proceeding to review each source in full. Initial exclusion criteria were: 1) toolkits with only one component (e.g., tool, resource); 2) toolkits outside health or education; or 3) toolkits under development. Decision to proceed to full review occurred if: 1) both reviewers agreed to include the abstract; 2) both reviewers found insufficient information to make a determination; 3) both reviewers agreed not to include abstract. Duplicate materials were flagged and conflicts (n = 5) as to whether or not to include an abstract for full review were discussed and resolved. Articles proceeded to full review if they were in the first or second category.
In stage 2 of the scoping review all included and undecided sources were located and reviewed in full. Initial exclusion criteria were again applied and new exclusion criteria emerged; development of exclusion criteria during the review stage is a unique feature of scoping reviews and differentiates scoping reviews from systematic reviews . Given the broad nature of the search it was unlikely that all exclusion criteria could be pre-determined. Sources were further eliminated if they: 1) did not contain the word kit or toolkit, and 2) had insufficient information to make a determination about its inclusion.
Two reviewers independently read all 139 sources identified for full review, with the exception of one article that could not be located. The reviewers periodically conferred on inclusion or exclusion criteria. Conflicting decisions (n = 7) were resolved through reviewer discussion, culminating in the inclusion of 83 sources.
Data extraction and analysis
Two reviewers independently abstracted the following data from each source: toolkit format, toolkit topic (as specified by the toolkit authors), target audience(s) (e.g., doctors, nurses, etc.), toolkit content, evidence underlying toolkit content, KT goal of the toolkit, whether toolkit had been evaluated, evaluation approach, evaluation design, evaluation outcome. A unique aspect of our review was a focus on whether the toolkit content was evidence-based, which we assessed based on information provided by the authors. Publication details and abstracted data were subsequently transferred to a detailed spreadsheet, and authors met to discuss and synthesize the findings. During meetings, the authors reviewed the information extracted from the sources and organized it into larger categories determined through group consensus discussions.
Breadth and format of toolkits used in health and healthcare practice
Of 253 citations initially identified, 139 met initial inclusion criteria and 83 toolkits were included in the final sample. Among the 83 toolkits, the majority (n = 68) were identified in journal articles, and 15 toolkits were located online in their entirety with no related peer-reviewed publication.
Toolkit topics were categorized as clinical, general health and safety, developmental and mental health, technology-related, advocacy, politics, and cultural awareness (see Table 1). Most toolkits (n = 67, 81%) were related to clinical or general health and safety issues (e.g., disease prevention and care, oral health, end of life care). All toolkit topics relating to healthcare practice by healthcare practitioners were considered clinical, as were hospital-specific toolkits with topics related to the practice of patient care in a hospital setting (e.g., infection control, operating room fire prevention, discharge practice, in-patient falls, etc.). Health topics relevant to the general population were categorized as general health and safety (i.e., seat belt use, sun safety).
With respect to target audiences, most toolkits (n = 70, 84%) were directed at healthcare practitioners, such as physicians and nurses (see Table 1). Among these, 4 toolkits also targeted patients. Community audiences (e.g., church groups) and health decision makers/policy makers (e.g., organization managers, government officials) were the target of a further 21 (25%) toolkits. School staff, parents and caregivers, and other audiences such as researchers comprised the target for the remaining 16 (19%) toolkits. Many toolkits had more than one target audience, and, consequently, more than one KT goal.
The majority of toolkits (n = 57, 69%) were designed to inform practice change (e.g., changing clinical or organizational practices and procedures, such as needle stick prevention, cancer screening protocols) targeting healthcare practitioners. Educating or sharing knowledge and informing were the main KT goals for 40 (48%) toolkits (e.g., public awareness campaigns such as sun safety) and typically the target audience was parents, caregivers and school staff. Finally, informing policy and decision-making was the KT goal for 10 (12%) toolkits (e.g., Scotland’s national approach to improving mental health services).
Toolkit content was well described in only 36 (43%) of the sources reviewed (see Table 2). All online toolkits fully described toolkit components, while the remaining 57% of sources stated the KT goal and the target audience(s) but did not adequately describe the materials included. Most toolkits included a combination of written materials (e.g., tip sheets, information sheets, guidelines; a toolkit for health professionals and their patients with pre-diabetes ), with fewer incorporating audio-visual materials such as CDs and DVDs along with written information (e.g., toolkit for healthcare practitioners to help prevent meningococcal disease ). Five toolkits included tools such as a pedometer and Body Mass Index wheel in addition to written materials (e.g., a proper exercise and nutrition toolkit for healthcare practitioners ).
Evidence underlying toolkit content: is the toolkit based on research evidence?
We examined whether the 83 toolkits were evidence-based (i.e., reference or inclusion of evidence underlying the toolkit), and found that 72% included some mention of the evidence supporting the toolkit, while the remaining 28% made no reference to evidence. The type of evidence underlying the development of the 60 toolkits varied widely. Here, evidence took the form of literature scan/reviews (n = 31), qualitative data (i.e., focus groups, interviews or stakeholder surveys; n = 17), expert panels (n = 13), and evidence-based guidelines (n = 14). Toolkits supported by evidence from randomized controlled trials were much less common (n = 3). Toolkits were sometimes supported by multidisciplinary specialists’ discussions, theories or conceptual frameworks, best-practice approaches, reviews of institutional documentation, national reports, and observations of existing practices. Most toolkits relied on a combination of these sources and none of the toolkits specified the evidence base underlying each individual toolkit element; if they were supported by evidence, this was not made clear in the source.
Effectiveness of toolkits: did the toolkit achieve the intended KT goal(s)?
We also examined whether (1) toolkits as a whole were evaluated relative to their purpose or KT goal, and whether (2) toolkit components were evaluated for specific outcomes. Information regarding evaluation of the toolkit either as a whole or for its components was only available for 31 of the 83 toolkits. Importantly, none of the online toolkits included information about toolkit effectiveness. Table 3 summarizes evaluation type and outcome for the 31 toolkits that were evaluated. Among these, four evaluated the individual toolkit components, while the remaining 27 toolkits were evaluated in their entirety. Toolkit evaluations were typically carried out via interviews and surveys and, more rarely, via focus groups. The most comprehensive evaluation was detailed in High School Coaches’ Assessments, Intentions to Use and Use of a Concussion Prevention Toolkit, which consisted of an information letter and brochure, reference cards, fact sheets, posters and a video. This was the only toolkit that provided outcome information regarding evaluation of the individual components and reported an almost 100% satisfaction rate.
Where possible, information was collected regarding the type of evaluation conducted on the toolkit: process (i.e., evaluation aimed at testing and monitoring the process of toolkit use), and outcome (i.e., evaluation aimed at assessing changes in practice and knowledge as a result of the toolkit use). For instance, the Toolkit for New Parents, designed for parenting education, has been evaluated both in terms of process (i.e., interviews conducted with state officials, administrators, and mothers to assess the toolkit’s use and customization details) and outcome (i.e., knowledge assessment at baseline and at 2- and 14-months follow-up in mothers who did or did not receive the toolkit).
With respect to overall outcome evaluation, the majority (n = 21) of evaluated toolkits reported that the toolkit was satisfactory, useful, or resulted in an intention to change practice. For instance, 92% of the nurses who received and tried out the Proper Exercise and Nutrition Toolkit planned to incorporate it into their practice . Results for two toolkit evaluations showed inconclusive outcomes (i.e., Safety of the Land Toolkit designed to increase rural children’s knowledge about safety ) and modest initial positive changes that were not sustained at follow-up (i.e., the Diabetes Literacy and Numeracy Education Toolkit designed to improve glycemic control and self-efficacy in patients with diabetes ). In the first case, there was wide variability in children’s perception and understanding of the message communicated through the safety toolkit components, with some children completely misinterpreting the visual materials provided.
Toolkits as a KT strategy have popular appeal, particularly given the ease with which they can be disseminated on the Internet in an engaging and multimodal manner to a wide variety of audiences. This study set out to scope the literature on toolkits in health and healthcare, to explore their breadth of topic, format, target, content, underlying evidence and their effectiveness as KT strategies. The majority of toolkits focus on clinical and general health and safety topics such as disease prevention and care, oral health, and end of life care. Toolkits permeate both published and grey literature and tend to be targeted to healthcare practitioners, although many also target community-based knowledge users, health decision makers and policy makers. Toolkit content was adequately described in fewer than half of the sources reviewed. Although about 70% of the sources mentioned the evidence underlying the toolkit content, there was a wide range of variability in what constituted this evidence. The most commonly used sources of evidence included literature review findings, qualitative data, expert panels, and practice guidelines. Of the 83 toolkits included in the study, only 31 had been evaluated in any way. Among these, the majority focused on the effectiveness of the toolkit as a whole, and, very rarely, on the effectiveness of the individual components. Among evaluated toolkits, outcomes were generally positive related to their intended KT goal.
To our knowledge, this is the first scoping review on toolkit use in health and healthcare. Our findings indicate that there is sufficient research on this topic to warrant a full systematic review. In addition, the findings reveal several knowledge gaps which can inform how to best share research evidence in a way that optimizes its use. First, evaluation findings suggest toolkits can be an effective KT strategy, but more systematic evaluations are needed to capture impacts beyond intention to use or user satisfaction, which were the most commonly employed indicators of outcome. Systematic evaluations of toolkits that aim to change knowledge, attitudes, practice and, where applicable, patient outcomes, are crucial for understanding their full impact on health practice and behavior, and ultimately strengthen their utility. This finding aligns with results from the Cochrane review on the use of educational materials to change the practice of healthcare professionals and patient outcomes . This gap is particularly relevant for the online toolkits reviewed here, of which none included information about effectiveness.
Second, comparisons of toolkits with other KT strategies using comprehensive designs will help us to elucidate the contexts in which toolkits are useful and the relative advantage of this type of KT strategy. Toolkit research must also include full documentation of the toolkit materials, a limitation noted in 60% of the sources reviewed herein. This is informative for toolkit development, assessing toolkit feasibility and effectiveness, and comparisons to other toolkits and KT educational approaches. Finally, given that toolkits aim to translate knowledge, it is essential to make explicit the type and quality of evidence that is being translated and that underlies each component within the toolkit; in other words, the evidence on which the toolkit elements are based.
We harken back to the conclusion of an earlier review  that noted that the effectiveness of KT strategies is highly dependent on context and that no single KT strategy has been shown to be universally effective. In light of this finding, evaluation of whether any KT strategy achieves its knowledge translation goal (e.g., build awareness, share knowledge, impact policy, change behavior or practice) and the subtle contextual factors that facilitate or hinder this outcome, is an essential component of the KT process, and is lacking in our current state of KT practice and science.
We note two limitations. First, we reviewed toolkits developed prior to 2011 and it is likely that other toolkits have been published in the last three years. This presents an opportunity for other authors to extend the review when conducting a more systematic review. Additionally, findings are limited to studies published in English. Despite these limitations, this scoping review presents a comprehensive profile of toolkit use in health and healthcare, and contributes to advancing knowledge in the field of KT. Addressing the above noted gaps in future research has the potential to optimize the effectiveness of toolkits as a strategy to share knowledge and change practice.
Toolkits are proving to be desirable, accessible, and useful but they lack scientific rigor with respect to the evidence underlying their content, and evaluation of their overall effectiveness. To the extent that toolkits are focused on practice or behavior change, it is imperative that they be supported by research evidence, which was not the case for all toolkits included in this review. Furthermore, fewer than half the toolkits identified as relevant for this review included any evaluation of the toolkit at any level. Moreover, if we are to develop confidence in the ability of toolkits to change practice and/or behavior, we must evaluate and accumulate evidence of their impacts. Only then can we be certain we are behaving ethically, supported by evidence, and with the knowledge that the toolkit is both effective and a defensible investment of time and resources.
Grimshaw J, Eccles M, Lavis J, Hill S, Squires J: Knowledge translation of research findings. Implement Sci. 2012, 7: 50-10.1186/1748-5908-7-50.
Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998, 317: 465-10.1136/bmj.317.7156.465.
Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group: Effective implementation of research into practice: an overview of systematic reviews of health literature. BMC Res Notes. 2011, 4: 212-10.1186/1756-0500-4-212.
LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M: The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012, 12: 751-10.1186/1471-2458-12-751.
Colquhoun H, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, et al: Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. Implement Sci. 2014, 9: 51-10.1186/1748-5908-9-51.
Mazza D, Bairstow P, Buchan H, Paubrey Chakraborty S, Van Hecke O, Grech C, et al: Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implement Sci. 2013, 8: 32-10.1186/1748-5908-8-32.
Damschroder LJ, Lowery JC: Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013, 8: 51-10.1186/1748-5908-8-51.
Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S: A conceptual framework for implementation fidelity. Implement Sci. 2007, 2: 40-10.1186/1748-5908-2-40.
Giguere A, Legare F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A, et al: Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012, 10:
Levac D, Colquhoun H, O’Brien KK: Scoping studies: advancing the methodology. Implement Sci. 2010, 5: 69-10.1186/1748-5908-5-69.
Arskey H, O’Malley L: Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005, 8 (1): 19-32. 10.1080/1364557032000119616.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group:P referred R eporting I tems for S ystematic Reviews and M eta-A nalyses: The PRISMA Statement. PLoS Med. 2009, 6 (6): e1000097-10.1371/journal.pmed.1000097.
Jaycox L, Morse L, Tanielian T, Stein B: How Schools Can Help Students Recover from Traumatic Experiences: A Tool Kit for Supporting Long-Term Recovery. 2006, RAND Gulf States Policy Institute, Santa Monica, CA
Evans P, Greaves C, Winder R, Fearn-Smith J, Campbell J: Development of an educational ‘toolkit’ for health professionals and their patients with prediabetes: the WAKEUP study (Ways of Addressing Knowledge Education and Understanding in Pre-diabetes). Diabet Med. 2007, 24 (7): 770-7. 10.1111/j.1464-5491.2007.02130.x.
Butler K: The development of an evidence-based toolkit to prevent meningococcal disease in college students. Fam Community Health. 2007, 30 (2): 93-111. 10.1097/01.FCH.0000264407.36013.e7.
Kopp MK, Hornberger C: Proper exercise and nutrition kit: use of obesity screening and assessment tools with underserved populations. J Pediatr Nurs. 2008, 23 (1): 58-64. 10.1016/j.pedn.2007.11.006.
Sawyer R: High school coaches’ assessments, intention to use, and use of a concussion prevention toolkit. Centers for Disease Control and Prevention’s Heads Up: Concussion in High School Sports. Health Promot Pract. 2010, 1: 34-43. 10.1177/1524839907309377.
Neuhauser L, Constantine W, Constantine N, Sokal-Gutierrez K, Obarski S, Clayton L, et al: Promoting prenatal and early childhood health: evaluation of a statewide materials-based intervention for parents. Am J Public Health. 2007, 97 (10): 1813-9. 10.2105/AJPH.2006.089532.
Bryant L, Hoon EA: Rural children’s perceptions of child farm safety printed communication strategies. Rural Remote Health. 2007, 7 (4): 734-
Cavanaugh K, Wallstone KA, Gebretsadik T, Shintani A, Huizinga MM, Davis D, et al: Addressing literacy and numeracy to improve diabetes care: two randomized controlled trials. Diabetes Care. 2009, 32 (12): 2149-55. 10.2337/dc09-0563.
Moores PS: Engaging community partners to promote healthy behaviours in young children. Can J Public Health. 2010, 101 (5): 369-73.
Chagpar A, Banez C, Lopez R, Cafazzo JA: Challenges of hand hygiene in healthcare: the development of a tool kit to create supportive processes and environments. Healthc Q. 2010, 13: 59-66. 10.12927/hcq.2010.21968.
Rhodes S: Strong Bones RN, RU? A national partnership results in an exciting toolkit for school nurses. NASN Sch Nurse. 2010, 25 (6): 285-7.
Goslar PW, Silvers M, Strever T, Judkins D, Segebarth P, Lerma C: Lessons from a statewide pilot of “the battle of the belts project” in a state without a primary seat belt law. J Trauma Inj Infect Crit Care. 2009, 67 (Suppl 1): S62-6. 10.1097/TA.0b013e3181ac175c.
Hanni KD, Garcia E, Ellemberg C, Winkleby M: Targeting the taqueria: implementing healthy food options at Mexican American restaurants. Health Promot Pract. 2009, 10 (Suppl 2): 91S-9. 10.1177/1524839908331268.
Emmons KM, Geller AC, Viswanath V, Rutsch L, Zwirn J, Gorham S, et al: The SunWise Policy intervention for school-based sun protection: a pilot study. J Sch Nurs. 2008, 24 (4): 215-21. 10.1177/1059840508319627.
Wall HK, Beagan BM, O’Neill HJ, Foell KM, Boddie-Willis CL: Addressing stroke signs and symptoms through public education: the Stroke Heroes Act FAST campaign. Prev Chronic Dis. 2008, 5 (2): A49-
Seguin RA, Economos CD, Hyatt R, Palombo R, Reed PNT, Nelson ME: Design and national dissemination of the Strong Women Community Strength Training Program. Prev Chronic Dis. 2008, 5 (1): A25-
Gallivan J, Greenberg R, Leontos C: Role of food and nutrition professionals in stemming the diabetes epidemic. J Am Diet Assoc. 2007, 107 (8): 1394-7. 10.1016/j.jada.2007.05.005.
Stewart D, Law M, Burke-Gaffney J, Missiuna C, Rosenbaum P, King G, et al: Keeping it together: an information KIT for parents of children and youth with special needs. Child: Care Health Dev. 2006, 32 (4): 493-500.
Hunt C, Rankine P, Blackmore R: A health toolkit for people with learning disabilities. Nurs Times. 2006, 102 (12): 34-6.
Wolff K, Cavanaugh K, Malone R, Hawk V, Gregory BP, Davis D, et al: The Diabetes Literacy and Numeracy Education Toolkit (DLNET): Materials to facilitate diabetes education and management in patients with low literacy and numeracy skills. Diabetes Educ. 2009, 35 (2): 233-45. 10.1177/0145721709331945.
Lennox N, Taylor M, Rey-Conde T, Bain C, Boyle FM, Purdie DM: Ask for it: Development of a health advocacy intervention for adults with intellectual disability and their general practitioners. Health Promot Int. 2004, 19 (2): 167-75. 10.1093/heapro/dah204.
Venguer T, Pick S, Fishbein M: Health education and agency: a comprehensive program for young women in the Mixteca region of Mexico. Psychol Health Med. 2007, 12 (4): 389-406. 10.1080/13548500601010250.
Vreeland B, Minsky S, Gara MA, Toto AM, Kosseff CO, Verna M, et al: Solutions for wellness: results of a manualized psychoeducational program for adults with psychiatric disorders. Am J Psychiatr Rehabil. 2010, 13 (1): 55-72. 10.1080/15487761003673032.
Caperchione C, Coulson F: The WellingTONNE Challenge Toolkit: using the RE-AIM framework to evaluate a community resource promoting healthy lifestyle behaviours. Health Educ J. 2010, 69 (1): 126-34. 10.1177/0017896910363301.
Auger C, Demers L, Desrosiers J, Giroux F, Ska B, Wolfson C: Applicability of a toolkit for geriatric rehabilitation outcomes. Disabil Rehabil. 2007, 29 (2): 97-109. 10.1080/09638280600731540.
Powers RJ, Wirtschafter D: Prevention of Group B Streptococcus early-onset disease: a toolkit by the California Perinatal Quality Care Collaborative. J Perinatol. 2010, 30 (2): 77-87. 10.1038/jp.2009.95.
Byszewski AM, Graham ID, Amos S, Man-Son-Hing M, Dalziel WB, Marshall S, et al: A continuing medical education initiative for Canadian primary care physicians: the Driving and Dementia Toolkit: a pre- and postevaluation of knowledge, confidence gained, and satisfaction. J Am Geriatr Soc. 2003, 51 (10): 1484-9. 10.1046/j.1532-5415.2003.51483.x.
Leape LL, Rogers G, Hanna D, Griswold P, Federico F, Fenn CA, et al: Developing and implementing new safe practices: voluntary adoption through state-wide collaboratives. Qual Saf Health Care. 2006, 15 (4): 289-95. 10.1136/qshc.2005.017632.
Berkley-Patton J, Bowe-Thompson C, Bradley-Ewing A, Hawes S, Moore E, Williams E, et al: Taking It to the Pews: a CBPR-Guided HIV Awareness and Screening Project with Black Churches. AIDS Educ Prev. 2010, 22 (3): 218-37. 10.1521/aeap.2010.22.3.218.
Schultz D, Barnes-Proby D, Chandra A, Jaycox LH, Maher E, Pecora P: Toolkit for Adapting Cognitive Behavioral Intervention for Trauma in Schools(CBITS)or Supporting Students Exposed to Trauma(SSET) for Implementation with Youth in Foster Care. Technical Report, RAND Corporation. 2010, 45-
Boyer-Chu L, Wooley S, SFEASHAKOH: Give It a Shot! Toolkit for Nurses and Other Immunization Champions Working with Secondary Schools. 2008
Mitchko J, Huitric M, Sarmiento K, Hayes G, Pruzan M, Sawyer R: CDC’s approach to educating coaches about sports-related concussion. Am J Health Educ. 2007, 38 (2): 99-103. 10.1080/19325037.2007.10598951.
US Environmental Protection Agency Protection Agency: SunWise Meteorologist ToolKit ERINST. USEPAPA. 2007, 53-
Isman B: Healthy People 2010: Oral Health Toolkit. NIDCR. 2007, 208-
Hepburn KS: Building culturally & linguistically competent services to support young children, their families, and school readiness. Annie E Casey Foundation. 2004, 146-
Harris R, Miske S, Attig G: Embracing diversity: Toolkit for creating inclusive learning-friendly environments. UNESCO Bangkok. 2004, 342-
Hertweck P, Hillard PJ, Bacon J, Gold M: Toolkit for adolescents. J Pediatr Adolesc Gynecol. 2004, 17 (3): 199-203. 10.1016/j.jpag.2004.03.038.
Wilkie DJ, Lin YC, Judge MKM, Shannon SE, Corless IB, Farber SJ, et al: TNEEL workshop: interactive methods for teaching end-of-life care. Nursing Educ Perspect. 2004, 25 (1): 26-9.
Bender BG, Dickinson P, Rankin A, Wamboldt FS, Zittleman L, Westfall JM: The Colorado asthma toolkit program: a practice coaching intervention from the high plains research network. J Am Board Fam Med. 2011, 24 (3): 240-8. 10.3122/jabfm.2011.03.100171.
Clark L, Calvillo E, dela Cruz F, Fongwa M, Kools S, Lowe J, et al: Cultural competencies for graduate nursing education. J Prof Nurs. 2011, 27 (3): 133-9. 10.1016/j.profnurs.2011.02.001.
Mandal S, Ruggles R, Leonardi G, Goodfellow F, Bradley N, Murray V: Developing best practice response to carbon monoxide incidents: a toolkit for health protection frontline staff. Public Health. 2011, 125 (3): 148-56. 10.1016/j.puhe.2010.11.011.
Zuyev L, Benoit AN, Chang FY, Dykes PC: Tailored prevention of inpatient falls: development and usability testing of the fall tips toolkit. CIN. 2011, 29 (2): 93-100.
Beune EJAJ, Haafkens JA, Bindels PJE: Barriers and enablers in the implementation of a provider-based intervention to stimulate culturally appropriate hypertension education. Patient Educ Couns. 2011, 82 (1): 74-80. 10.1016/j.pec.2010.02.015.
Murray E, May C, Mair F: Development and formative evaluation of the e-health implementation toolkit. BMC Med Inform Decis Mak. 2010, 10 (1): 61-10.1186/1472-6947-10-61.
Kolin MM, Minnier T, Hale KM, Martin SC, Thompson LE: Fall initiatives: redesigning best practice. J Nurs Adm. 2010, 40 (9): 384-91. 10.1097/NNA.0b013e3181ee4468.
El-Ghorr A, Cameron R, Fleming M, McKechnie L, Thomson D, Doherty S: Scotland’s national approach to improving mental health services: integrated care pathways as tools for redesign and continuous quality improvement. Int J Care Pathw. 2010, 14 (2): 57-64. 10.1258/jicp.2010.010014.
Simpson J, Kniahnicki S, Quigley-Hobbs K: Developing an orientation toolkit for new public health nurse hires for Ontario’s changing landscape of public health practice. Nurs Leadersh. 2010, 23: 113-24. 10.12927/cjnl.2010.21751.
Baldwin J, Brodrick A, Cowley S, Mason N: An innovative toolkit to support normal birth. J Midwifery Women’s Health. 2010, 55 (3): 270-2. 10.1016/j.jmwh.2009.11.013.
Shah BR, Bhattacharyya O, Yu C, Mamdani M, Parsons JA, Straus SE, et al: Evaluation of a toolkit to improve cardiovascular disease screening and treatment for people with type 2 diabetes: Protocol for a cluster-randomized pragmatic trial. Trials. 2010, 11: 44-10.1186/1745-6215-11-44.
Kellagher M, Simpson J: Workload and workforce planning: developing a learning toolkit. Nurs Manag (Harrow). 1994, 17 (1): 32-4. 10.7748/nm2010.04.17.1.32.c7646.
Turner S, Chase DL, Milne R, Cook A, Hicks NJ: The health technology assessment adaptation toolkit: description and use. Int J Technol Assess Health Care. 2009, 25 (Suppl 2): 37-41. 10.1017/S0266462309990663.
Williams I, Mears G, Raisor C, Wilson J: An emergency medical services toolkit for improving systems of care for stroke in North Carolina. Prev Chronic Dis. 2009, 6 (2): A67-
Morris CD, Waxmonsky JA, May MG, Giese AA: What do persons with mental illnesses need to quit smoking? Mental health consumer and provider perspectives. Psychiatr Rehabil J. 2009, 32 (4): 276-84. 10.2975/32.4.2009.276.284.
Hewitt AM, Spencer SS, Ramloll R, Trotta H: Expanding CERC beyond public health: sharing best practices with healthcare managers via virtual learning. Health Promot Pract. 2008, 9 (Suppl 4): 83S-7. 10.1177/1524839908319090.
Byrd J, Thompson L: “It’s safe to ask”: promoting patient safety through health literacy. Healthc Q. 2008, 11 (Spec No 3): 91-4. 10.12927/hcq.2008.19656.
Gallivan J, Greenberg R, Leontos C: Role of food and nutritional professionals in stemming the diabetes epidemic. J Am Diet Assoc. 2007, 107 (8): 1394-7. 10.1016/j.jada.2007.05.005.
Sheridan DJ: Labeling solutions and medication in sterile procedural settings. Joint Commision J Qual Patient Saf. 2006, 32 (5): 276-82.
Brand C, Cox S: Systems for implementing best practice for a chronic disease: management of osteoarthritis of the hip and knee. Intern Med J. 2006, 36 (3): 170-9. 10.1111/j.1445-5994.2006.01018.x.
Hoffman Y, Everly GS, Werner D, Livet M, Madrid PA, Pfefferbaum B, et al: Identification and evaluation of mental health and psychosocial preparedness resources from the Centers for Public Health Preparedness. JPHMP. 2005, 11: S138-42.
Bust P, Gibb AGF, Brace C: Implementing a health management toolkit in the construction industry. Econ Ergonomics. 2008, 31: 197-202.
Lee JA, Zierler BK: Venous thromboembolism safety toolkit: a systems approach to safe practice interventions. Consum Centered Comput Supported Care Healthy People. 2006, 55 (8): 1281-6.
New toolkit for water and healthy ageing in care homes. Journal of the Royal Society for the Promotion of Health 2006, 126(1):4–4.
Paterson BL, Sock LA, LeBlanc D, Brewer J: Ripples in the water: A toolkit for Aboriginal people on hemodialysis. CANNT J. 2010, 20 (1): 20-8.
Petruccio C, Mills Shaw KR, Boughman J, Fernandez C, Harlow I, Kruesi M, et al: Healthy choices through family history: a community approach to family history awareness. Community Genet. 2008, 23 (4): 732-43.
Escoffery C, Glanz K, Elliott T: Process evaluation of the Pool Cool Diffusion Trial for skin cancer prevention across 2 years. Health Educ Res. 2008, 24 (4): 215-21.
McConigley R, Toye C, Goucke R, Kristjanson L: Developing recommendations for implementing the Australian Pain Society’s pain management strategies in residential aged care. Australas J Ageing. 2008, 27 (1): 45-9. 10.1111/j.1741-6612.2007.00266.x.
McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, et al: Fidelity outcomes in the national implementing evidence-based practices project. Psychiatr Serv. 2007, 58 (10): 1279-84. 10.1176/ps.2007.58.10.1279.
Zolotor AJ, Randolph GD, Johnson JK, Wegner S, Edwards L, Powell C, et al: Effectiveness of a practice-based, multimodal quality improvement intervention for gastroenteritis within a Medicaid managed care network. Pediatrics. 2007, 120 (3): e644-50. 10.1542/peds.2006-1749.
Heller LR, Gospodarowicz M, Jadad AR: eToolkits: improving pain management. J Pain Palliat Care Pharmacother. 2007, 21 (3): 67-70.
Roland CL: Partners for understanding pain: resources and toolkits to increase public awareness. J Pain Palliat Care Pharmacother. 2006, 20 (1): 21-4.
Study: Fewer heart deaths when QI efforts are made. Healthcare benchmarks and quality improvement 2006, 13(2):21-22.
Wright FC, Law CHL, Last LD, Klar N, Ryan DP, Smith AJ: A blended knowledge translation initiative to improve colorectal cancer staging. BMC Health Serv Res. 2006, 6: 4-10.1186/1472-6963-6-4.
Hannah KL, Schade CP, Cochran R, Brehm JG: Promoting influenza and pneumococcal immunization in older adults. Jt Comm J Qual Patient Saf. 2005, 31 (5): 286-93.
Levine SA, Brett B, Robinson BE, Stratos GA, Lascher SM, Granville L, et al: Practicing physician education in geriatrics: lessons learned from a train-the-trainer model. J Am Geriatr Soc. 2007, 55 (8): 1281-6. 10.1111/j.1532-5415.2007.01205.x.
Green CJ, Fortin P, Maclure M, Macgregor A, Robinson S: Informatics system support as a critical success factor for chronic disease management: necessary but not sufficient. Int J Med Inform. 2006, 75 (12): 818-28. 10.1016/j.ijmedinf.2006.05.042.
Glazier RH, Badley EM, Lineker SC, Wilkins AL, Bell MJ: Getting a grip on arthritis: an educational intervention for the diagnosis and treatment of arthritis in primary care. J Rheumatol. 2005, 32 (1): 137-42.
Health Literacy Toolkit: Entire Toolkit. 2006. http://www.csg.org/knowledgecenter/docs/ToolKit03HealthLiteracy.pdf,
Early Childhood Mental Health Consultation: An Evaluation Tool Kit. 2010. Georgetown University Center Child and Human Development (GUCCHD). http://gucchd.georgetown.edu/72407.html,
Mental Health Toolkit: Entire Toolkit. 2005. http://www.csg.org/knowledgecenter/docs/ToolKit05MentalHealth.pdf,
Children’s Hospital Boston: Developmental Screening Tool Kit for Primary Care Providers. National Academy for State Health Policy. http://www.nashp.org/node/2064,
PATH Vaccine Resource Library: Global Health Initiative Resource Tool Kit. 2011. http://www.path.org/vaccineresources/details.php?i=1909,
Introducing the 2008 Clinical Practice Guidelines Tool Kit. 2011. http://www.diabetes.ca/clinical-practice-education/professional-resources,
Primary Health Organisations: Service development toolkit for mental health services in primary health care. 2008. https://www.health.govt.nz/system/files/documents/publications/phoservicedevelopmenttoolkitmentalhealthservices.pdf,
This research was funded by a grant from Auto21 NCE (#ACA-301A) to Beth Bruce and Melanie Barwick. We would like to acknowledge the assistance of Jessica Darling, Information Specialist.
The authors declare that they have no competing interests.
SS was highly qualified personnel on Dr. Barwick’s sub-grant, initiated the study and conducted the search with the help of a librarian. RB and SS extracted relevant data, analyzed it and prepared the first draft of the manuscript. MB (senior responsible author) contributed to the analysis and manuscript preparation. BB has made important contributions to the interpretation of the results. All authors participated in discussions at all stages of the project and in the editing of the various drafts. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (https://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Barac, R., Stein, S., Bruce, B. et al. Scoping review of toolkits as a knowledge translation strategy in health. BMC Med Inform Decis Mak 14, 121 (2014). https://doi.org/10.1186/s12911-014-0121-7
- Knowledge translation
- Practice change