This article has Open Peer Review reports available.
Ambulance officers' use of online clinical evidence
© Westbrook et al; licensee BioMed Central Ltd. 2006
Received: 29 May 2006
Accepted: 27 July 2006
Published: 27 July 2006
Hospital-based clinicians have been shown to use and attain benefits from online evidence systems. To our knowledge there have been no studies investigating whether and how ambulance officers use online evidence systems if provided. We surveyed ambulance officers to examine their knowledge and use of the Clinical Information Access Program (CIAP), an online evidence system providing 24-hour access to information to support evidence-based practice.
A questionnaire was completed by 278 ambulance officers in New South Wales, Australia. Comparisons were made between those who used CIAP and officers who had heard of, but not used CIAP.
Half the sample (48.6%) knew of, and 28.8% had used CIAP. Users were more likely to have heard of CIAP from a CIAP representative/presentation, non-users from written information. Compared to ambulance officers who had heard of but had not used CIAP, users were more likely to report better computer skills and that their supervisors regarded use of CIAP as a legitimate part of ambulance officers' clinical role. The main reasons for non-use were lack of access(49.0%) and training(31.4%). Of users, 51.3% rated their skills at finding information as good/very good, 67.5% found the information sought all/most of the time, 87.3% believed CIAP had the potential to improve patient care and 28.2% had directly experienced this. Most access to CIAP occurred at home. The databases frequently accessed were MIMS (A medicines information database) (73.8%) and MEDLINE(67.5%). The major journals accessed were Journal of Emergency Nursing(37.5%), American Journal of Medicine(30.0%) and JAMA(27.5%).
Over half of ambulance officers had not heard of CIAP. The proportion who knew about and used CIAP was also low. Reasons for this appear to be a work culture not convinced of CIAP's relevance to pre-hospital patient care and lack of access to CIAP at work. Ambulance officers who used CIAP accessed it primarily from home and valued it highly. Lack of access to CIAP at central work locations deprives ambulance officers of many of the benefits of an online evidence system.
Health care managers seeking to implement organisational and system-wide information system interventions to support improvements in clinical practice have a wide selection of possibilities from which to choose. However when selection criteria include evidence of intervention effectiveness the options rapidly fall away. We have undertaken a program of research focused upon examining the effectiveness of online evidence systems to support improved clinical decision-making and patient care. This research has centred on examination of the Clinical Information Access Program (CIAP) , an online evidence website implemented by the New South Wales Health Department in 1997. CIAP provides around 55,000 health practitioners in the state's public hospital sector with access to clinical databases and journals 24 hours a day, both at point-of-care and at home. It is the largest health evidence service undertaken in Australia and internationally.
Several studies of hospital clinicians' use of CIAP have demonstrated that this intervention is effective in supporting clinicians in their provision of clinical care , has influenced their information-seeking behaviours and increases the accuracy of clinical decision-making . A state-wide survey of 5,511 clinicians revealed significant differences in the experiences and use of CIAP reported by doctors, nurses and allied health professionals [5–7]. When administering this survey we took the opportunity to survey ambulance officers whose main work occurs outside hospitals but who have regular contact with hospitals. Ambulance officers all have passwords to access the online evidence service, but there was considerable variation in the extent to which ambulance stations had terminals available from which staff could access CIAP.
While there is an enormous literature relating to evidence-based practice in the medical, nursing and allied health professions, there is very limited data relating to the extent to which ambulance professionals have adopted, or been supported in their use of, evidence-based practice. To our knowledge no previous studies of pre-hospital professionals' use of online evidence systems have been published. Using a convenience sample of ambulance officers we aimed to examine their patterns utilisation of the CIAP, and to compare the characteristics of ambulance officers who used CIAP with those officers who knew of, but had not used CIAP.
The survey was conducted in 2001 in 65 randomly selected state hospitals where a survey coordinator ensured that 25% of the medical, nursing and allied health staff completed the paper-based questionnaire . Although the questionnaire was developed to investigate the experiences of hospital-based health professionals, the NSW Ambulance Service requested that ambulance officers be included in the survey as much of the questionnaire was considered pertinent to its workforce. The Service requested 2,000 questionnaires which were then sent by the Ambulance service to Ambulance Stations throughout the state for distribution to qualified ambulance officers. The researchers had no control over and were not involved in the processes of distribution or collection of the surveys which may have been less than optimal. Of these 2000 surveys 278 were completed, a return rate of 14%. Chi square analyses and a t-test were used to compare the characteristics of ambulance officers who were CIAP users with those officers who knew of, but had not used, CIAP. Some respondents (ranging from 1–5 in number) failed to answer a few questions. Percentages were calculated in terms of the number answering each item. Some questionnaire items were not relevant to ambulance officers working outside hospitals and were omitted from the analysis.
The convenience sample of 278 ambulance officers had a mean age of 38.0 years (SD = 9.1 years, range = 21–68 years); 72.6% were males and 27.4% were females. The gender distribution of the sample was similar to that of the state's ambulance staff . Respondents (N = 135) who had heard about CIAP were the primary focus of the study. They did not differ significantly in age from officers who had not heard of CIAP (Heard of CIAP: mean = 37.3 years, SD 9.39; Not heard: mean = 37.3 years, SD = 10.54, t = -0.008, df 267, p = 0.994). Those who had heard of CIAP did not differ in gender from officers who had not heard of CIAP (respectively 75.4% males compared to 69.8% of officers who had not heard of CIAP, χ2 = 1.034, df 1, p = 1.034).
Knowledge of CIAP and characteristics of users and non-users
Ambulance officers' knowledge and use of CIAP
Total sample of ambulance officers (n = 278)
Heard of CIAP
Ambulance officers who had heard of CIAP (n = 135)
Ambulance officers who used CIAP (n = 80)
Attended CIAP training course
Results of chi square analyses comparing CIAP users and non-users who had heard of CIAP
Questions and responses
Please rate your computer skills
χ2 = 15.80, df 2, p = 0.000
How did you first hear about CIAP?
χ2 = 19.76, df 5, p = 0.001
Indicate your support for: I think using CIAP is a legitimate part of my clinical role
χ2 = 16.85, df 2, p = 0.000
Indicate your support for: My direct supervisor thinks that using CIAP is a legitimate part of my clinical role
χ2 = 9.74, df 2, p = 0.008
Reasons for using or not using CIAP
Ambulance officers who used CIAP were asked to check as many of 17 reasons for using CIAP as applied to them. The reasons most frequently given were to: fill a knowledge gap (77.5%), undertake personal education eg for a specific course or project (71.3%), gain access to a standard reference (50%), assist research (48.8%), support the education of others (33.8%), improve patient outcomes (15.0%), confirm a clinical decision (12.5%), settle a dispute or controversy regarding diagnosis or treatment (11.3%), make a diagnosis (5.0%), develop a treatment plan (5.0%) and review policies or guidelines (5.0%).
When officers who knew of but had not used CIAP, were asked to check their reason(s) for non-use from a set of options their responses were: no access (49.0%), no training (31.4%), lack of time (17.6%), use other information sources (15.7%), don't need information from CIAP to do my job (9.8%), difficult to use (3.0%), too slow (2.0%) and other (19.6%).
Frequency of use of CIAP
CIAP users were asked, "In the last month, how often have you used CIAP?" The most frequent response was never (30.0%), followed by once (26.3%), 2–3 times (23.8%), 2–6 times per week (13.8%), once a week (6.3%), and daily 0.0%. When asked their intentions regarding their future use 1.3% said they did not plan to use CIAP, 17.7% planned to use it less than once a month, 45.6% checked 1–3 times per month, 16.5% said once per week, 17.7% said 2–6 times per week and 1.3% said daily. Thus users intended to increase their usage in the future; currently 20.1% of users were accessing CIAP once a week or more while in the future 35.5% planned to do so.
Ease of using CIAP
Users were asked to rate CIAP in terms of how easy it was to use. It was assessed as very easy by 5.1%, easy by 26.6%, neither easy nor difficult by 48.1%, difficult by 17.7% and very difficult by 2.5% of users. When requested to rate the speed of search time on CIAP 2.5% said excellent, 21.5% very good, 53.2% good, 21.5% fair and 1.3% poor. Users were asked to rate their access to technical help regarding CIAP. Only 1.3% said access to help was excellent, 8.0% checked very good, 45.3% good, 41.3% fair and 4.0% poor. Users rated their "skills in using CIAP to find information". No one said excellent, 11.3% checked very good, 40.0% good, 40.0% fair and 8.8% poor.
Location of access to CIAP
The questionnaire asked respondents the locations where they accessed CIAP. These questions were aimed at CIAP use by clinicians in hospitals rather than ambulance officers in the field. Thus only 2.5% of ambulance officers had access to CIAP "near to where you treat/manage patients", 6.3% had access in a university library, 3.8% in a hospital library, 30.0% in an "office away from the clinical area", 83.8% had access at home, and 11.3% in other places. Most ambulance officers (66.3%) only mentioned one place where they had access, 30.0% had access in two locations and 3.8% in three locations.
Respondents were asked to estimate the percentage of time they used CIAP in each of four locations; home, where you treat patients, other places in the hospital and other. The percentages were to total 100%. The most frequent place where ambulance officers used CIAP was at home; 65 of the 80 CIAP users said they did so and they estimated on average that 80.6% of their usage occurred there. Only 5 officers used CIAP near point-of-care and they estimated that an average of 51.0% of their usage occurred there. Fifteen officers used CIAP in "other places in the hospital" and on average 49.0% of their access time was spent there. Nineteen officers said they used CIAP in other places and on average 66.8% of their usage occurred there. Thus most users accessed CIAP at home and most of their use of CIAP occurred there.
At the time of the survey CIAP provided access to 15 databases. Ambulance officers who used CIAP were asked to indicate from a list the databases they used most frequently. These were MIMS (a medicines information database) (73.8%), MEDLINE (67.5%), Harrison's Book of Internal Medicine (33.8%), Interactive ECG Tutorials (28.8%), Cochrane (21.3%), CINAHL (17.5%), EMBASE (11.3%), Australian Medicines Handbook (10.0%), Therapeutic Guidelines (8.8%), Medweaver (7.5%), Micromedex (6.3%), Medix case-based learning (6.3%), the Medical Officer's Handbook (5.0%), PsychINFO (3.8%) and Interactive WWW internet tutorials (2.5%). The average number of databases checked was 3.0 (SD 1.7, range 0–8). Only two CIAP users did not frequently access any databases
There were 16 full-text medical, 14 nursing and 10 mental health journals available to CIAP users at the time of the survey. Respondents were asked to indicate from a list which they used most frequently. These were the Journal of Emergency Nursing (37.5%), American Journal of Medicine (30.0%), Journal of the American Medical Association (27.5%), British Medical Journal (22.5%), New England Journal of Medicine (20.0%), The Lancet (16.3%), Advances in Nursing Science (10.0%), Journal of Clinical Nursing (10.0%), Annals of Internal Medicine (8.8%), Journal of Clinical Investigation (7.5%), Advances in Nursing Science (7.5%), Nursing Standard (7.5%) and Pediatrics (7.5%). The average number of frequently referred to journals was 2.6 per user (SD 2.3, range 0–11). Thirty percent of CIAP users did not list any journal as frequently used.
Effectiveness of CIAP
Respondents were asked, "Considering the times you have used CIAP, how often did you find the information you wanted?" Thirteen percent said all of the time, 54.5% reported most of the time, 31.3% some of the time and no one checked never. Users were asked, "Do you believe CIAP has the potential to improve patient care?". Most (87.3%) answered yes, 11.4% said they did not know and 1.3% responded no. When asked "Do you have direct experience of CIAP resulting in improved patient care?" 28.2% of users said yes and 71.8% replied no.
Ambulance officers' questionnaire responses revealed lower levels of knowledge of CIAP (48.6%) than did other health professions surveyed at the same time; 58% of nurses , 72% of doctors  and 82% of allied health staff  had heard of CIAP. Despite indicators of ambulance officers' favourable endorsement of practice improvement initiatives such as CIAP  the proportion of ambulance officers who knew of and had also used CIAP (59.3%) was substantially less than for other health professions; 74% of nurses , 84% of doctors  and 76% of allied health  knew about and used CIAP. Of the total sample of ambulance officers 5.0% had heard of CIAP from a CIAP representative or at a presentation compared to 6.4% of all nurses and 6.1% of doctors. Thus lack of exposure to these informational inputs via a CIAP representative or presentation, does not appear to be a major factor in explaining ambulance officers' poorer knowledge of CIAP.
Ambulance officers' reasons for non-use differed from those of other professions. Nearly half of the ambulance officers who were non-users indicated that the reason for this was that they did not have access to CIAP compared to, for example, 22.3% of nurse non-users. While 63.5% of nurses cited lack of CIAP training as a reason for non-use, only 31.4% of ambulance officers did so. Most nurse users accessed CIAP near point-of-care (58.2% vs. 2.5% of ambulance officers), in the hospital library (31.9% vs. 3.8% of ambulance officers) or in an office away from the clinical area (29.7% vs. 30.0%). Most ambulance officers accessed CIAP at home (83.8%) compared to 41.7% of nurses . Thus lack of access to CIAP would seem to be a major contributing factor to lower use of CIAP by ambulance professionals. Few resources available on CIAP at the time of the survey were targeted at pre-hospital emergency care and this is also likely to have reduced the relevance of CIAP. This was similarly a problem for allied health professional groups and contributed to some groups' lower rates of CIAP use . Since the survey CIAP has substantially increased the resources available with several 100 fulltext journals available. However, as Tippett et al  point out there remains a significant shortage of research in pre-hospital emergency care limiting the availability of high quality evidence to guide evidence-based practice.
A further factor contributing to lower use among ambulance officers appears be the lack of support and encouragement to use CIAP in their work environments. Comparison of nurses' and ambulance officers' responses to statements about the legitimacy of using CIAP as part of one's clinical role showed stronger support for this proposition among nurses of whom 71.9% agreed/strongly agreed compared to 64.1% of ambulance officers. The difference between nurses and ambulance officers was much greater regarding agreement with the proposition that their direct supervisor saw using CIAP as a legitimate part of their role; 50.4% of nurses agreed/strongly agreed compared to 35.1% of ambulance officers. In response to the statement "People I work with use CIAP as a means of obtaining up-to-date information" only 22.5% of ambulance officers agreed/strongly agreed, while 53.8% of nurses did so. Thus the work cultures of nurses and other health professional groups were more supportive of the use of CIAP. This, as well as difficulties of access, may account for ambulance officers' much greater usage of CIAP at home.
Those ambulance officers who used CIAP were clearly committed to its use as they devoted out-of- work time to this activity. Not only did more ambulance professionals access CIAP at home than did any other professional group, but they also undertook most of their work with CIAP there. The smaller proportion of nurse users who used CIAP at home claimed that they performed on average 51.4% of their work with CIAP there, compared to ambulance officers who did 80.6% of their CIAP work there. Of ambulance officers who used CIAP, 67.5% reported finding information they sought on CIAP all or most of the time compared to 64.0% of nurses. Ambulance professionals' ratings of the speed of CIAP were similar to those of nurses, but their ratings of ease of use of CIAP and their skills in CIAP use were slightly lower than were nurses' ratings. Nurse users accessed CIAP more frequently (26.6% had used CIAP in the last week) than ambulance officers (20.1%). Most ambulance officers (87.3%) and nurses (84.2%) who used CIAP considered that CIAP had the potential to improve patient care, though fewer ambulance officers (28.2%) than nurses (35.1%) reported direct evidence of this. Lack of access to CIAP at point-of-care was a major barrier to ambulance professionals being able to utilise the full benefits of CIAP. The survey shows that compared to other health professions there was less cultural support for ambulance officers to use CIAP. Nevertheless a committed group of officers, who primarily used CIAP away from work, reported experiences of the benefits of CIAP similar to those of user groups in other professions.
A limitation of this study is that the sample represented a smaller and probably less representative proportion of ambulance professionals than was the case with the other health professions surveyed. We had limited control over the survey distribution and have no information about non-responders. It is possible that CIAP users were more likely to complete the survey than non-users. However the fact that over half of the respondents reported not having heard of CIAP provides some reassurance that this group was not significantly under-represented. Also the demographics of those who had heard of CIAP did not differ significantly from those who had not heard of CIAP. Given the lack of research undertaken about this group of health professionals we believe the results provide an important glimpse into the profession's responsiveness to adopting evidence-based practice measures where possible. Ambulance officers have been shown to hold highly favourable attitudes toward other health improvement initiatives such as the NSW Safety Improvement Program . Early assessments of the CIAP provided some evidence that ambulance officers were receptive to the benefits that CIAP might provide. In her report of an on-line survey of CIAP Ayres quoted the comments of an ambulance officer: "The CIAP results in greater clinical knowledge, so thanks for a great resource. This is absolutely fabulous for ambulance officers. We are often left out in terms of on-going education, and access to this site enables us to keep up with current trends, research and medications." (p181) This limited evidence all suggests that ambulance officers are open to initiatives aimed at improving care processes and the adoption of evidence-based practices.
We now have a growing body of research which demonstrates that access to online evidence systems improves professionals' accuracy of [4, 13, 14], and confidence in  answers to clinical questions, and positively impacts upon care delivery and patient outcomes [2, 16, 17]. The results of this survey suggest that ambulance officers as a professional group are also likely to reap these benefits when given easy access to, and encouragement by their work culture to use, such resources. Evidence-based health policy makers and organisational leaders should work towards increasing access to online evidence systems for health professionals both within and outside hospitals.
We are very grateful to the Ambulance Service of NSW for facilitating the distribution of surveys and particularly Mr Tony Donnelly. We also thank the ambulance officers who took the time to complete the survey. This research was funded by NSW Health.
- Ayres D, Wensley M: The clinical information access project. Med J Aus. 1999, 171: 544-546.Google Scholar
- Westbrook J, Gosling AS, Coiera E: Do clinicians use online evidence to support patient care? A study of 55,000 clinicians. Journal of American Medical Informatics Association. 2004, 11 (2): 113-120. 10.1197/jamia.M1385.View ArticleGoogle Scholar
- Westbrook J, Coiera E, Gosling AS, Braithwaite J: Critical incidents and journey mapping as techniques to evaluate the impact of online evidence retrieval systems on health care delivery and patient outcomes. Int J Med Inform. 2006,Google Scholar
- Westbrook J, Coiera E, Gosling AS: Do online information retrieval systems help experienced clinicians answer clinical questions?. J Am Med Inform Assoc. 2005, 12: 315-321. 10.1197/jamia.M1717.View ArticlePubMedPubMed CentralGoogle Scholar
- Gosling A, Westbrook J, Spencer R: Nurses' use of online clinical evidence. J Ad Nurs. 2004, 47 (2): 201-211. 10.1111/j.1365-2648.2004.03079.x.View ArticleGoogle Scholar
- Gosling A, Westbrook J: Allied health professionals' use of online evidence: a survey of 790 staff working in the Australian public hospital system. Int J Med Inform. 2004, 73: 391-401.View ArticlePubMedGoogle Scholar
- Westbrook J, Gosling A, Westbrook M: Use of point-of-care online clinical evidence retrieval systems by junior and senior doctors in NSW public hospitals. Internal Medical Journal. 2005, 35 (7): 399-404. 10.1111/j.1445-5994.2005.00836.x.View ArticleGoogle Scholar
- Lindberg D, Siegal E: On assessing the impact of medical information: does Medline make a difference?. Meth Infom Med. 1991, 30 (4): 239-240.Google Scholar
- Ambulance Service of New South Wales: Annual Report 2001/02. Sydney. 2002Google Scholar
- Gosling AS, Westbrook J: Allied health professionals' use of online evidence: a survey of 790 staff working in the Australian public hospital system. Int J Med Inform. 2004, 73: 391-401.View ArticlePubMedGoogle Scholar
- Tippett V, Woods S, FitzGerald G, Clark M: Towards a national research agenda for the ambulance and pre-hospital sector in Australia. Journal of Emergency Primary Health Care. 2003, 1 (1–2): [http://www.jephc.com/full_article.cfm?content_id=38]Google Scholar
- Westbrook M: Evaluation of the Safety Improvement Program in New South Wales: Report on the NSW Health questionnaire evaluation of the SIP course. Sydney: Centre for Clinical Governance Research in Health, University of New South Wales. 2005Google Scholar
- Hersh W: Information retrieval. A health and biomedical perspective. 2003, New York: Springer-Verlag, 2Google Scholar
- Hersh W, Crabtree K, Hickam D, Friedman CP, Tidmarsh P, Mosbaek C, Kraemer D: Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. 2002, 9 (3): 283-293.Google Scholar
- Westbrook J, Gosling AS, Coiera E: The impact of an online evidence system on confidence in decision making in a controlled setting. Med Dec Mak. 2005, 25: 178-185. 10.1177/0272989X05275155.View ArticleGoogle Scholar
- Magrabi F, Coiera E, Westbrook J, Gosling AS: Clinician use of online evidence in primary care consultations. Int J Med Inform. 2005, 74 (1): 1-12. 10.1016/j.ijmedinf.2004.10.003.View ArticlePubMedGoogle Scholar
- Maviglia S, Yoon C, Bates DW, Kuperman GJ: KnowledgeLink: Impact of context-sensitive information retrieval on clinicians information needs. Journal of American Medical Informatics Association. 2006, 13 (1): 67-73. 10.1197/jamia.M1861.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/6/31/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.