- Research article
- Open Access
Increasing utilization of Internet-based resources following efforts to promote evidence-based medicine: a national study in Taiwan
BMC Medical Informatics and Decision Making volume 13, Article number: 4 (2013)
Since the beginning of 2007, the National Health Research Institutes has been promoting the dissemination of evidence-based medicine (EBM). The current study examined longitudinal trends of behaviors in how hospital-based physicians and nurses have searched for medical information during the spread of EBM.
Cross-sectional postal questionnaire surveys were conducted in nationally representative regional hospitals of Taiwan thrice in 2007, 2009, and 2011. Demographic data were gathered concerning gender, age, working experience, teaching appointment, academic degree, and administrative position. Linear and logistic regression models were used to examine predictors and changes over time.
Data from physicians and nurses were collected in 2007 (n = 1156), 2009 (n = 2975), and 2011 (n = 3999). There were significant increases in the use of four Internet-based resources – Web portals, online databases, electronic journals, and electronic books – across the three survey years among physicians and nurses (p < 0.001). Access to textbooks and printed journals, however, did not change over the 4-year study period. In addition, there were significant relationships between the usage of Internet-based resources and users’ characteristics. Age and faculty position were important predictors in relation to the usage among physicians and nurses, while academic degree served as a critical factor among nurses only.
Physicians and nurses used a variety of sources to look for medical information. There was a steady increase in use of Internet-based resources during the diffusion period of EBM. The findings highlight the importance of the Internet as a prominent source of medical information for main healthcare professionals.
Physicians and nurses are main healthcare professionals in clinical service. They should periodically update their medical information to optimize patient care and outcome . In the past, knowledge was obtained from textbooks, colleagues, and journal articles [2, 3]. In recent years, development of medical informatics has provided an opportunity for easy access to up-to-date medical information. The Internet is widely used for locating medical knowledge now [4, 5].
There are a variety of Internet-based resources, such as Web portals, online databases, electronic journals, and electronic books [6–8]. Web Portals have been a good starting point to search medical information. A growing trend of using Web portals has been noticed around the world . However, challenges exist regarding how to easily find quality information from an overwhelmingly large number of Web-search results. In addition, the publication policy of traditional journals and textbooks has been changed with the advent of Internet . The number of medical journals and textbooks that offer electronic versions of their publications has grown tremendously . There is an increase of free available electronic journals and books. Furthermore, online databases offer access to retrieve the summarized evidence-based information. Mastery of access to online databases has become pertinent for improving the quality of clinical practice [11–15].
Policymakers of healthcare institutions have implemented information systems to support healthcare professionals’ use of evidence-based medicine (EBM), which involving free access to Internet-based EBM resources . They need to know whether these resources are regularly being employed to apply appropriate medical information on patient care. There are increasing studies investigating the information-searching behavior of health professionals [17, 18]. However, very few have tracked the trends in search of medical information over time.
Since the beginning of 2007, the National Health Research Institutes (NHRI) has overseen a collaborative project to provide information systems in support of clinical practice with evidence for regional hospitals of Taiwan [19, 20]. It is unclear exactly how such implementation is influencing the ways in which healthcare professionals obtain medical information. Thus, we examined responses to three questionnaire surveys administered between 2007 and 2011. The purpose of this was to demonstrate the longitudinal trends in behavior patterns among hospital-based physicians and nurses in searching medical information during a 4-year promotional period.
Initiatives of NHRI
The project launched by NHRI to promote the implementation of EBM was multifaceted, consisting of offering information resources and carrying out a series of activities. Overall, the program set up a Chinese EBM Web site, offered free usage of the Cochrane Library, and provided Chinese abstracts of the Cochrane Database of Systematic Reviews for health professionals to access. Furthermore, NHRI conducted monthly workshops with a total of 4714 attendee throughout Taiwan. In addition, this campaign established the Taiwan Evidence-Based Medicine Association (TEBMA) in 2007. In cooperation with TEBMA, NHRI organized annual conferences and competitions pertaining to EBM implementation.
NHRI developed a structured questionnaire using questions based on previous literature [3, 21–23]. The first survey was conducted at the beginning of 2007. The second and third surveys were conducted in 2009 and 2011, respectively.
The targets in this study were physicians and nurses working in Taiwan’s regional hospitals. A regional hospital is defined as a secondary care hospital, as appraised by Taiwan’s Joint Commission of Hospital Accreditation. Sixty-one regional hospitals, including a total of approximately 5000 physicians and 20000 nurses, were enrolled. Only the 4 regional hospitals that did not have a collaborative project with NHRI to promote the implementation of EBM were not enrolled. In the 2007 survey, the postal questionnaires were distributed at random to 30 members of the clinical staff (with the title of either medical doctor or nurse) of each regional hospital . In order to obtain more questionnaires from nurses a cluster sampling design was used in the 2009 survey, as described previously . Briefly, questionnaires were distributed to all staff at 13 hospitals selected at random, including 4 located in northern Taiwan, 3 in eastern Taiwan, 3 in southern Taiwan, and 3 in central Taiwan. Since we achieved a higher number of respondents using the cluster sampling method in the 2009 survey compared to the earlier survey, we continued the same cluster sampling method for our subsequent survey in 2011. All questionnaire returns were anonymous.
The survey included items for measuring the usage patterns of 7 informational resources: (1) Web portals (e.g., Google, Yahoo), (2) online databases, (3) electronic textbooks, (4) electronic journals, (5) textbooks, (6) printed journals, and (7) colleague consultations. The frequency was classified by Likert’s 4-point scale (always, often, seldom, and never).
The survey further explored the usage of access to 9 online databases: 2 databases in Chinese and 7 databases in English. Chinese databases included the Index to Chinese Periodical Literature (ICPL) and the Chinese Electronic Periodical Service (CEPS). English databases included the Cumulative Index to Nursing & Allied Health Literature (CINAHL), the Cochrane Library, MD Consult, MEDLINE/PubMed, ProQuest, UpToDate, and Micromedex. Utilization was defined as access at least once per month during the previous 6 months.
The background characteristics — including gender, age, faculty position, director position, working experience, EBM training, and academic level — were also examined. Age of the respondents was classified into four sections: 20–30, 31–40, 41–50, and more than 50 years old. EBM training was defined as participation in the EBM workshops or related curricula. In addition, academic level was divided into five categories: (1) those with a diploma from a technical school; (2) those who graduated from a junior college (2-year university); (3) those with a bachelor’s degree (The curriculum is 7 years for medical school, 6 years for dental school and 4 years for nursing school); (4) those with a master’s degree; and (5) those with a doctoral degree.
Validity and reliability
Content validity of the questionnaire was examined by 9 experts with more than 15 years each of professional clinical service. The internal consistency of all indexes was estimated using Cronbach’s coefficient alpha. The reliability and validity of the current survey were well supported by sufficient content validity index and Cronbach’s coefficient alpha .
The Ethical Review Board of NHRI approved the study protocol. The questionnaire was accompanied an introductory letter stating the purpose of this study and promising confidentiality. Returns were considered as indicating consent to participate in this study.
Likert’s 4-point scale for the use of informational resources was dichotomized for further analyses. A self-rating report of “always” or “often” was regarded as a favorable answer. The other two (“seldom” and “never”) were viewed as unfavorable answers. The statistical analyses were conducted using a commercially available program (SPSS 12.0 for Windows, SPSS Inc., Illinois, USA). Pearson’s chi-square test was used to compare the differences. Logistic regression was used to examine relationships among variables. Significance was defined as p < 0.05. All odds ratio (OR) and 95% confidence intervals (CI) were adjusted for the demographic characteristics and survey years.
Demographic data of respondents
This study enrolled 1156 participants in 2007 (605 physicians and 551 nurses), 2975 in 2009 (563 physicians and 2412 nurses), and 3999 in 2011 (645 physicians and 3354 nurses). The valid response rate was 63.2% in 2007, 57.8% in 2009, and 58.2% in 2011. Their demographic characteristics are shown in Table 1.
There were significant differences in the demographic characteristics between physicians and nurses. Males were more common among physicians; females were more common among nurses. Most physicians were between the ages of 30 and 50 years old, while most nurses were 20–40 years old. Most physicians had at least a bachelor’s degree; but most nurses didn’t. Physicians more often received EBM training than nurses did (p < 0.001). Administrative and faculty positions were more common among physicians than nurses.
Resources of medical information
The information-searching patterns of physicians and nurses are illustrated in Figure 1. The most common source of medical information in all survey years for physicians was Web portals (77.9%, 84.6%, and 95.1% in 2007, 2009, and 2011, respectively). As for nurses, the most common resource in 2007 was textbooks (73.2%) but was Web portals in 2009 (77.8%) and 2011 (94.2%). There were significant increases in the usage of four Internet-based resources — including Web portals, online databases, electronic journals, and electronic books – across the three survey years among physicians and nurses. In addition, there was no significant change in the utilization of textbooks and printed journals among physicians and nurses. Furthermore, there was a significant increase of colleague consultations in 2011 than in 2007 and 2009 among nurses, but not among physicians.
Demographic predictors of usage of Internet-based resources
Demographic data were incorporated into logistic regression analysis to determine the relationship of the usage of each Internet-based information source (Table 2). There were significant relationships between the usage of Internet-based resources and users’ characteristics, including age, academic degree, working experience, and director and faculty positions. The use of every Internet-based source significantly increased from 2007 to 2011 (p < 0.001). Male physicians were more likely to seek medical information through Web portals than female physicians (p < 0.01). In addition, young physicians (age ≤ 50) tended to access the online databases (p < 0.01) and electronic journals (p < 0.05) more than elder physicians (age > 50). Young nurses (age ≤ 30) were more likely to access every Internet-based resource than elder nurses (age > 50) (p < 0.05). Furthermore, physicians with a doctoral degree were more likely to use electronic journals than physicians without an advanced degree (p < 0.01). Nurses with a doctoral degree were more likely than those with a bachelor’s degree to use all 4 Internet-based resources (p < 0.05). Nurses with a master’s degree were more likely to access Web portals, online databases, and electronic journals than those with a bachelor’s degree (p < 0.05). In contrast, nurses with a degree from a technical school were less likely to use Web portals, online databases, and electronic journals than those with a bachelor’s degree (p < 0.001). Faculty physicians tended to access the online databases and electronic journals more than physicians without a faculty position. Faculty nurses were more likely to access Web portals, online databases, and electronic journals than nurses without a faculty position. Physicians with EBM training were more likely to access the online database than those without EBM training. Physicians with a working period less than 5 years were less likely to access Web portals than physicians with a working period greater than 10 years.
Trends in the use of 9 online databases
Participants were asked to report how often they accessed the online databases. Figure 2 displays the changes in the use of 9 online databases for the three survey years. Among physicians there were significant increases in the usage of 8 online databases — ICPL, CEPS, CINAHL, the Cochrane Library, MD Consult, MEDLINE/PubMed, Micromedex, and UpToDate — across the three survey years. In addition, there were significant increases in the usage of 8 online databases — CEPS, CINAHL, the Cochrane Library, MD Consult, MEDLINE/PubMed, Micromedex, ProQuest, and UpToDate — over the 4-year study period for nurses.
The current evaluation has illustrated how information-searching behaviors changed among main healthcare professionals. We characterized trends in the search of medical information across the nationwide regional hospitals of Taiwan, using data from 3 national surveys conducted with main healthcare professionals in 2007, 2009, and 2011. Coupled with these three surveys we showed a growth in the use of Internet-based resources. The data suggest that there has been a sustained effect following the dissemination of EBM implementation during the 4-year study period,
To our knowledge, this study is the first to examine national trends in sources of medical information. The results highlight that the Internet has become an increasingly important source of medical information, which is consistent with recent research findings indicating that the Internet has become the most common source of medical information [3, 24]. Internet-based resources can provide easy access to reduce barriers, such as physical distance from a library. In our study, medical information in which main healthcare professionals looked for has shifted to electronic media gradually.
Our study demonstrated a high level of usage with the Web portals. The use of Web portals increased steadily across the 3 survey years. Searching for medical information through Web portals has both advantages and disadvantages. Although Web portals are a convenient and easily accessible source of medical information, challenges regarding the quality and credibility of medical information exist. With the advent of Web portals (such as the development of Google Scholar), the quality of searching has been improving [25, 26]. Nevertheless, online databases are still more practical for retrieving evidence-based search results than Web portals . Familiarity with evidence-based sites is essential for healthcare providers in order to improve the quality of medical care. Our previous study showed increasing access of online evidence retrieval systems among physicians and nurses after a 2-year promotion of EBM . The current survey has further extended the inquiry by demonstrating a sustaining increase in the utilization of four Internet-based resources over 4-year period. We found that not only increased usage of Internet-based resources in general, but also the trend applied to all 8 popular online EBM databases. During our study period, NHRI offered free access to the Cochrane Library to promote EBM dissemination [11, 27]. Nevertheless, both physicians and nurses used a variety of databases to look for medical information rather than using the Cochrane Library alone. In addition, our study demonstrated that the increased usage of online database was proportional to that of Web portals. We found the majority of physicians and nurses used the online database for self-learning or clinical practice [28, 29]. However, we did not know why they accessed the Web portals. Further studies are needed to verify the difference in the motivation of usage between online database and Web portals.
In our study population, electronic journals have replaced printed journals as the most commonly used resource when looking for medical information. With the advance of medical informatics, the number of electronic books and journals is increasing exponentially. In addition, access to Web portals and online databases can enhance the utilization of electronic journals and electronic books. Folb et al. reported that healthcare professionals regarded convenience as the most important factor for accessing printed or electronic formats at times of need . These data suggest electronic resources have become more important for healthcare professionals when searching for medical information.
Differences in the backgrounds between physicians and nurses may lead to different behaviors of information searching through the Internet. In this study, both groups were increasingly using the Internet to find answers to clinical questions. However, their demographic predictors of access to Internet-based resources were somewhat different. First, gender and working period were significant factors in relation to the usage of Web portals among physicians only, while in nursing profession, it was not significant. Second, academic degree carried more significance in nurses than in physicians. Third, age and faculty position were important factors of usage for both physicians and nurses; this finding is in accordance with previous studies [20, 30]. And, fourth, electronic books were less associated with the personal characteristics which was similar to a report showing that use of electronic books was not related to users’ personal profiles . By distinguishing these characteristics, educational providers could devise better strategies to enhance their utilization. For example, physicians and nurses with senior age or without faculty position need more educational programs and interventions to prompt online utilization. In our study, the other personal characteristics – such as gender and working period – carried less significant influence in the usage. These results can provide stakeholders and promoters with valuable information to increase access to evidence-based information.
This study has several limitations common to survey research, such as potential biases from the incomplete response and self-reported method. We cannot be sure that these self-reported changes were fully translated into actual behaviors. In addition, some methodological issues should be considered in this study. First, we modified the sampling method in the 2009 and 2011 surveys to collect more questionnaires from nurses. Nevertheless, we believe the influence of this alteration was minimal because of high similarity of demographic backgrounds across the three questionnaire surveys. Second, our study did not take other factors which have enabled Internet health information searching into account. For instance, the influences of institutional characteristics on the participants, such as their own organizational change, were not determined. Third, this study did not examine the impact of such changes on clinical practice. Further studies are necessary to determine the usage of Internet-related resources in the improvement of patient outcome.
In conclusion, this study has provided nationally representative estimates for the preferences of main healthcare professionals in searching for medical information. We tracked trends in informational resources sought by main healthcare professionals over time. Our data indicated that the Internet was increasingly being used by physicians and nurses as main source of medical information in Taiwan. The results portray a tectonic shift in the ways in which main healthcare practitioners searched for the medical information, with increasing access of information through Internet-based resources among healthcare providers.
Steinbrook R: Searching for the right search —reaching the medical literature. N Engl J Med. 2006, 354 (1): 4-7. 10.1056/NEJMp058128.
Timpka T, Ekstrom M, Bjurulf P: Information needs and information seeking behaviour in primary health care. Scand J Prim Health Care. 1989, 7 (2): 105-109. 10.3109/02813438909088656.
Bennett NL, Casebeer LL, Kristofco R, Collins BC: Family physicians' information seeking behaviors: a survey comparison with other specialties. BMC Med Inform Decis Mak. 2005, 5 (1): 9-10.1186/1472-6947-5-9.
Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK: Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med. 2005, 165 (22): 2618-2624. 10.1001/archinte.165.22.2618.
Bennett NL, Casebeer LL, Kristofco RE, Strasser SM: Physicians' Internet information-seeking behaviors. J Contin Educ Health Prof. 2004, 24 (1): 31-38. 10.1002/chp.1340240106.
Groen F: Canada's national initiative to advance access to electronic journals. Health Libr Rev. 2000, 17 (4): 189-193. 10.1046/j.1365-2532.2000.00293.x.
Folb BL, Wessel CB, Czechowski LJ: Clinical and academic use of electronic and print books: the Health Sciences Library System e-book study at the University of Pittsburgh. J Med Libr Assoc. 2011, 99 (3): 218-228. 10.3163/1536-5050.99.3.009.
Eysenbach G: Challenges and changing roles for medical journals in the cyberspace age: electronic pre-prints and e-papers. J Med Internet Res. 1999, 1 (2): E9-10.2196/jmir.1.2.e9.
Morris S: The future of journals: where will electronic publishing take us?. Hosp Med. 2001, 62 (5): 301-304.
Pravikoff DS, Donaldson NE: Online journals: access and support for evidence-based practice. AACN Clin Issues. 2001, 12 (4): 588-596. 10.1097/00044067-200111000-00015.
Chiu YW, Weng YH, Lo HL, Hsu CC, Shih YH, Kuo KN: Impact of a nationwide outreach program on the diffusion of evidence-based practice in Taiwan. Int J Qual Health Care. 2010, 22 (5): 430-436. 10.1093/intqhc/mzq049.
Nicholson LJ, Warde CM, Boker JR: Faculty training in evidence-based medicine: improving evidence acquisition and critical appraisal. J Contin Educ Health Prof. 2007, 27 (1): 28-33. 10.1002/chp.90.
Dinkevich E, Markinson A, Ahsan S, Lawrence B: Effect of a brief intervention on evidence-based medicine skills of pediatric residents. BMC Med Educ. 2006, 6: 1-10.1186/1472-6920-6-1.
Smith CA, Ganschow PS, Reilly BM, Evans AT, McNutt RA, Osei A, Saquib M, Surabhi S, Yadav S: Teaching residents evidence-based medicine skills: a controlled trial of effectiveness and assessment of durability. J Gen Intern Med. 2000, 15 (10): 710-715. 10.1046/j.1525-1497.2000.91026.x.
McKibbon KA, Walker-Dilks CJ: The quality and impact of MEDLINE searches performed by end users. Health Libr Rev. 1995, 12 (3): 191-200. 10.1046/j.1365-2532.1995.1230191.x.
Isaac T, Zheng J, Jha A: Use of UpToDate and outcomes in US hospitals. J Hosp Med. 2012, 7 (2): 85-90. 10.1002/jhm.944.
Williams MG, Dittmer A: Textbooks on tap: using electronic books housed in handheld devices in nursing clinical courses. Nurs Educ Perspect. 2009, 30 (4): 220-225.
D'Alessandro DM, Kreiter CD, Peterson MW: An evaluation of information-seeking behaviors of general pediatricians. Pediatrics. 2004, 113 (1 Pt 1): 64-69.
Chiu YW, Weng YH, Lo HL, Hsu CC, Shih YH, Kuo KN: Comparison of evidence-based practice between physicians and nurses: a national survey of regional hospitals in Taiwan. J Contin Educ Health Prof. 2010, 30 (2): 132-138. 10.1002/chp.20070.
Chiu YW, Weng YH, Lo HL, Ting HW, Hsu CC, Shih YH, Kuo KN: Physicians' characteristics in the usage of online database: a representative nationwide survey of regional hospitals in Taiwan. Inform Health Soc Care. 2009, 34 (3): 127-135. 10.1080/17538150903102372.
Yu MK, Huang CY, Lai YJ: A survey of physicians' knowledge and usage of the medical database. Chin J Fam Med. 2001, 11: 128-138.
Wong K, Veness MJ: Internet access, awareness and utilization of web-based evidence: A survey of Australian, New Zealand and Singaporean radiation oncology registrars*. Asia-Pac J Clin Onco. 2005, 1 (4): 120-127. 10.1111/j.1743-7563.2005.00033.x.
Wilson P, Droogan J, Glanville J, Watt I, Hardman G: Access to the evidence base from general practice: a survey of general practice staff in Northern and Yorkshire Region. Qual Health Care. 2001, 10 (2): 83-89. 10.1136/qhc.10.2.83.
Gosling AS, Westbrook JI: Allied health professionals' use of online evidence: a survey of 790 staff working in the Australian public hospital system. Int J Med Inform. 2004, 73 (4): 391-401. 10.1016/j.ijmedinf.2003.12.017.
Younger P: Using Google Scholar to conduct a literature search. Nurs Stand. 2010, 24 (45): 40-46.
Anders ME, Evans DP: Comparison of PubMed and Google Scholar literature searches. Respir Care. 2010, 55 (5): 578-583.
Weng YH, Hsu CC, Shih YH, Lo HL, Chiu YW, Kuo KN: Dissemination of systematic reviews in hospital setting: a comparative survey for diffusing the Cochrane Library. Postgrad Med J. 2012, 88 (1043): 511-514. 10.1136/postgradmedj-2011-130652.
Chiu YW, Weng YH, Lo HL, Shih YH, Hsu CC, Kuo KN: Comparison of accessing of online databases between physicians and nurses in Taiwan. Inform Health Soc Care. 2012, 37 (4): 230-241. 10.3109/17538157.2012.654842.
Weng YH, Kuo KN, Yang CY, Lo HL, Shih YH, Chiu YW: Information-searching behaviors of main and allied health professionals: a nationwide survey in Taiwan. J Eval Clin Pract. 2012, 10.1111/j.1365-2753.2012.01871.x. [Epub ahead of print]
Tenopir C, King DW, Bush A: Medical faculty's use of print and electronic journals: changes over time and in comparison with scientists. J Med Libr Assoc. 2004, 92 (2): 233-241.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/13/4/prepub
We are grateful to the participants who gave their time in this study. This work was supported by research grants from the National Health Research Institutes, Taiwan [PH-099-SP02, PH-098-SP03, and 96AI-HDSP05-018].
The authors declare that they have no competing interests.
YHW and YWC conceived and developed the study and drafted the manuscript. KNK was responsible for organizing the team, obtaining funding and drafting the manuscript. CYY, HLL, YHS, and CC assisted with coordinating the study and drafting the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Weng, Y., Kuo, K.N., Yang, C. et al. Increasing utilization of Internet-based resources following efforts to promote evidence-based medicine: a national study in Taiwan. BMC Med Inform Decis Mak 13, 4 (2013). https://doi.org/10.1186/1472-6947-13-4
- Medical Information
- Regional Hospital
- Online Database
- Faculty Position
- Electronic Journal