Improving antibiotic prescribing for adults with community acquired pneumonia: Does a computerised decision support system achieve more than academic detailing alone? – a time series analysis

Design

A two stage pre and post intervention cohort study, and a time series analysis

Setting

This study was performed at the Royal Melbourne Hospital, an urban adult tertiary teaching hospital with 350 beds including 14 intensive care unit (ICU) beds. The emergency department assesses 50,000 patients per year, leading to 16,000 admissions to hospital. This hospital did not have an electronic medical record or a computerised order entry system. Over 30 different doctors were working in the ED at any point in time over the study periods, and the allocation of doctors to patients was not structured. A computerised antibiotic approval system restricting access to ceftriaxone was also in operation over all three time periods of this study. Its implementation pre dated the commencement of this study. It approved ceftriaxone use for all patients with severe pneumonia, and its content agreed with the CAP guideline content.

Participants

This study described the prescribing behaviour of doctors (both senior and junior medical staff) managing patients in the ED. Specifically, the study focused on antibiotic prescribing for all patients who were initially diagnosed with CAP by the treating clinician in the ED.

Intervention

The study extended over three distinct time periods. The first, (or 'baseline') period was from April 2003–March 2004. The second (or 'academic detailing') period (AD) was February 2005–October 2005 and the third (or 'computerised decision support') period (CDSS) was from April 2006–September 2006.

During the first ('baseline') time period, electronic and paper copies of national antibiotic prescribing guidelines were available to staff in the ED [13] but no particular additional efforts were made to encourage uptake of the guideline.

At the start of the second ('academic detailing') time period, a program of academic detailing was initiated at the hospital. This involved training two senior ED clinicians, a pharmacist and a nurse to provide academic detailing to their colleagues. They spent one on one time educating colleagues (doctors and pharmacists) about antibiotic prescribing recommendations. These activities were opportunistic and occurred during the usual rostered hours. Interactions were not scheduled and no formal documentation of AD encounters was made. Posters and laminated cards with information about severity assessments and appropriate antibiotic choices for patients with CAP were distributed and actively promoted throughout the ED during the academic detailing period. These personnel and advertising material remained available throughout the following ('computerised decision support') time period, but were not specifically promoted.

At the commencement of the computerised decision support period, the guideline for the management of patients with CAP was deployed on an existing decision support tool. This tool is a web-based transferable system that was designed at the hospital using a .NET framework and implemented in January 2005.

The CAP algorithm used the Pneumonia Severity Index (PSI) to guide site of management decisions (inpatient vs. outpatient care) and the modified British Thoracic Society severity score (CURB) to highlight patients with severe pneumonia who were likely to need review by the intensive care unit (ICU) staff. [17, 18] The program was integrated with hospital databases containing patient demographics and pathology results to facilitate rapid calculation of scores required for these prediction rules. Use of these scores was not, however mandated. Users could choose to skip the score to obtain antibiotic advice alone. Antibiotic allergy reminders were included. If a user had previously registered an allergy for a patient this was presented, otherwise a reminder was given to check with the patient. Detailed information was included about unusual pathogens to consider, the most appropriate choice of empiric antibiotics, the duration of therapy, and the timing of change from intravenous to oral antibiotic therapy. Users had access to medical literature via the Internet, along with local interpretation of this literature within the CDSS. Users could browse the CDSS content without logging a patient in, so it could be used as an educational tool as well as providing patient specific advice. There was general agreement between the empiric antibiotic recommendations made in the national guideline, the AD directives and the content of the CDSS.

The CDSS was available hospital wide and its use was entirely voluntary. All hospital clinicians could access it via a shortcut on the desktop of any hospital computer. No specific incentives were provided to encourage its use. It was not triggered by any other computer systems. It resided alongside other electronic hospital guidelines. An introductory demonstration was provided to the ED staff and to all staff at a hospital grand round. Thereafter, infectious diseases registrars or pharmacists provided demonstrations informally.

Data collection

All patient presentations to the ED were available for inclusion in the study. Patients were prospectively identified from a database in the ED where the treating doctor already routinely recorded the patient's diagnosis. All patients with a diagnosis of pneumonia, chest infection, lower respiratory tract infection, pleuritic chest pain, cough, shortness of breath, and/or aspiration were identified. Patients were included in the study if they had a new respiratory symptom, a new chest x-ray infiltrate consistent with pneumonia, and if the initial assessment made by the treating doctor was that the patient had pneumonia.

Exclusion criteria included: Age <18 years, immunocompromised patients (corticosteroids ≥ 15 mg prednisolone/day for ≥ 2 weeks, HIV positive with CD4 <200 umol/L, transplant recipients on immunosuppressive therapy), suspected or known severe acute respiratory syndrome (SARS), nosocomial pneumonia (discharged from hospital in the previous 2 weeks, after an admission longer than 48 hours), and/or known suppurative lung diseases such as cystic fibrosis or bronchiectasis.

Data were prospectively collected from the medical history by a single trained research nurse, according to a set of specified rules. A single clinician was assigned to make judgements about any difficult issues, and a random sample of these cases was cross checked with a second infectious diseases physician. This group comprised 5% of the total patient cohort (40 patients). Specific clinical and pathological and radiological data available within the first 24 hours were sought to allow calculation of severity scores. [17, 18] Clinicians' comments about suspicion of aspiration, and documentation of known antibiotic allergies were recorded. The time to antibiotic therapy was calculated using the time of presentation, documented electronically by the ED triage nurse, and the time of antibiotic administration as documented on the medication chart by the nurse in ED or on the ward.

Information regarding ongoing antibiotic use was collected. Any antibiotics that were clearly being used to treat a separate infection (as described in the patient's medical record) were not included. Where the duration of treatment after discharge was not recorded, it was assumed to be 5 days. Antibiotic costs were calculated using pharmacy purchasing data. No actual changes in the cost of drugs commonly prescribed for pneumonia occurred over the study period. The admission criteria for ICU were based entirely upon the treating clinician's assessment in all time periods. No protocols or guidelines were enforced. Clinicians were not aware that the study was being conducted. The researchers had no clinical role in the ED over the study period. There were no major changes in the number or composition of staff in the ED, or their responsibilities over the study period. This study was approved by the ethics committee of Melbourne Health. Individual consent from the clinicians or the patients involved was not required.

Outcome measures

The primary outcome assessed was the prescription of empiric antibiotic therapy that adequately covered the likely pathogens (both typical and atypical) and was concordant with recommendations. This included the combination of a recommended beta lactam (amoxicillin, ampicillin, benzylpenicillin, ceftriaxone, cefotaxime or cefuroxime) plus either a macrolide (erythromycin, roxithromycin, clarithromycin or azithromycin) or doxycycline. The use of moxifloxacin alone was also classed as appropriate. Patients who received additional antibiotics were still classed as appropriate, so long as their antibiotic regimen included the recommended drugs (reflecting that the patients at least received appropriate cover). The possibility that antibiotics were required for other concordant problems was appreciated, and without detailed clinical information, it was not possible to determine if this additional antibiotic use was unnecessary.

A number of secondary outcomes were also examined. For patients who required ICU intervention at any time during their admission, the proportion that were admitted directly from the ED to the ICU was evaluated as a marker of early recognition of severe disease. Similarly, the proportion of patients requiring ICU management at any time during their admission who were initially prescribed the recommended empiric broad spectrum antibiotics for severe pneumonia in the ED was compared. Appropriate therapy for this group was defined as ceftriaxone (or benzylpenicillin plus gentamicin), in combination with either intravenous azithromycin or erythromycin. The use of moxifloxacin alone was also deemed appropriate.

The number of patients prescribed an antibiotic to which they had a documented allergy was examined. The overall pattern of antibiotics prescribed, and the average cost of antibiotics per patient, were assessed in each time period. Finally, the time between presentation to the ED and the administration of antibiotics was recorded.

Statistical methods

Baseline characteristics of subjects were compared between the three periods using a chi-squared test of homogeneity for categorical variables and analysis of variance for continuous variables. An a priori level of statistical significance of 0.05 was assumed.

The baseline period extended over one year to give an indication of the baseline pattern of change in the rate of concordant prescribing over time, in the absence of any intervention. The academic detailing period included enough patients to detect an improvement in mean concordance from 65% to 75% (188 patients, power = 0.8 and p = 0.05). The computerised decision support period included enough patients to detect an expected further improvement in concordance from 75% to 85% (120 patients, power = 0.8, p = 0.05).

Multivariable logistic models were used to compare the mean proportions of concordance across the three periods, while adjusting for disease severity, age, and suspected aspiration. Secondary outcome measures were assessed in the same way. Specifically, among the patients who required ICU admission, the proportion directly admitted from ED to the ICU, and the proportion administered appropriate broad-spectrum empiric antibiotic therapy, were compared. This was specifically recorded as a measure of the degree of recognition of markers of severe illness, which were a key focus of the guideline content. The proportion of patients with a known antibiotic allergy who received that antibiotic was also compared. Time to antibiotic administration was recorded as a measure of whether the CDSS delayed decision making to any extent.

A time series analysis was performed to evaluate changes in concordance of prescribing over time, covering all three time periods. The rate of concordant prescribing was expected to improve over time. Change in concordance over time was assessed with a binary logistic model, incorporating month of treatment as a continuous variable. The 'expected' proportion of concordant treatment at any given time then plausibly corresponds to a regression line fitted through the data. We hypothesized that the rate of concordant prescribing after the intervention (in the third time period) would be greater than that expected given the observed trend before the intervention (the first and second time periods). Statistical analysis was performed using Stata version 9.0. [19]