Skip to main content

Advertisement

We’d like to understand how you use our websites in order to improve them. Register your interest.

Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration’s electronic medical record

Abstract

Background

Antimicrobial prophylaxis is an evidence-proven strategy for reducing procedure-related infections; however, measuring this key quality metric typically requires manual review, due to the way antimicrobial prophylaxis is documented in the electronic medical record (EMR). Our objective was to electronically measure compliance with antimicrobial prophylaxis using both structured and unstructured data from the Veterans Health Administration (VA) EMR. We developed this methodology for cardiac device implantation procedures.

Methods

With clinician input and review of clinical guidelines, we developed a list of antimicrobial names recommended for the prevention of cardiac device infection. We trained the algorithm using existing fiscal year (FY) 2008–15 data from the VA Clinical Assessment Reporting and Tracking-Electrophysiology (CART-EP), which contains manually determined information about antimicrobial prophylaxis. We merged CART-EP data with EMR data and programmed statistical software to flag an antimicrobial orders or drug fills from structured data fields in the EMR and hits on text string searches of antimicrobial names documented in clinician’s notes. We iteratively tested combinations of these data elements to optimize an algorithm to accurately classify antimicrobial use. The final algorithm was validated in a national cohort of VA cardiac device procedures from FY2016–2017. Discordant cases underwent expert manual review to identify reasons for algorithm misclassification.

Results

The CART-EP dataset included 2102 procedures at 38 VA facilities with manually identified antimicrobial prophylaxis in 2056 cases (97.8%). The final algorithm combining structured EMR fields and text note search results correctly classified 2048 of the CART-EP cases (97.4%). In the validation sample, the algorithm measured compliance with antimicrobial prophylaxis in 16,606 of 18,903 cardiac device procedures (87.8%). Misclassification was due to EMR documentation issues, such as antimicrobial prophylaxis documented only in hand-written clinician notes in a format that cannot be electronically searched.

Conclusions

We developed a methodology with high accuracy to measure guideline concordant use of antimicrobial prophylaxis before cardiac device procedures using data fields present in modern EMRs. This method can replace manual review in quality measurement in the VA and other healthcare systems with EMRs; further, this method could be adapted to measure compliance in other procedural areas where antimicrobial prophylaxis is recommended.

Peer Review reports

Background

Post-surgical and post-procedural infections, including cardiac device infections following electrophysiology procedures, are highly morbid [1, 2]. These severe complications can be prevented with appropriate pre-incisional antimicrobial prophylaxis, which can reduce rates of infection by approximately 80% and are strongly recommended in multi-society guidelines [3,4,5,6]. Given the high morbidity of these infections and their preventability, pre-incisional antimicrobial compliance is an important quality metric used in most U.S. hospitals, including the Veterans Health Administration (VA). However, measurement of this key quality metric represents a significant challenge as, unlike most medications, antimicrobials administered in the operating room generally do not require that an order be placed. Thus, measurement systems that are effective for monitoring most medication orders cannot be applied to assess pre-incision prophylaxis. Previously, a time-consuming and expensive manual review process was undertaken to determine compliance with pre-incisional antimicrobial prophylaxis in select surgical procedures [7, 8]; however, there is currently no infrastructure for tracking compliance in non-surgical procedural areas (i.e., outside the operating room and by an interventionist as opposed to a surgeon) where prophylaxis is also recommended.

Identifying whether antimicrobial prophylaxis was administered has historically relied on manual chart review because an electronic method has several challenges. Some institutions designed their electronic medical record (EMR) to include structured data fields for ongoing antimicrobial stewardship [9, 10]; however this approach may be unfeasible in large healthcare systems with existing EMR systems and is often reliant on structured data about antimicrobial dispensing. Within most procedural and operating room settings, pre-incisional antimicrobial prophylaxis can be directly dispensed without documentation in the EMR of medication administration. Documentation of prophylaxis regimens may be present only in pre-procedural order sets without EMR documentation of administration or may only be present in clinician notes, such as the operative note or anesthesia records. This documentation style creates an opportunity for a novel clinical informatics approach combining structured (e.g., pre-procedural antimicrobial orders) and unstructured data (e.g., documentation of prophylaxis in clinical notes) to measure compliance with pre-incisional prophylaxis. Further, this method has the potential to support quality monitoring of non-surgical procedures, including cardiac device implantations, where little is known about compliance with guideline-concordant antimicrobial prophylaxis.

The goal of this study was to develop a methodology integrating structured and unstructured data elements found in modern EMRs to replace manual record review for quality measurement with an electronic tool to measure pre-incisional prophylaxis. We demonstrated the performance of this method using cardiac device implantation procedures in the VA. The VA, a national healthcare system with a robust EMR, is an ideal setting to conduct this work. In particular, VA’s Corporate Data Warehouse (CDW) includes extensive unstructured text data from clinical notes in the EMR, allowing us to access a large dataset to develop and validate our tool [11]. Methods combining structured and unstructured data for electronic measurement of pre-incision prophylaxis could be adapted for other settings, including for traditional surgical procedures.

Methods

We established a method to accurately classify cardiovascular implantable electronic device (CIED) procedures with pre-incisional antimicrobial prophylaxis by 1) developing an algorithm in SAS and SQL software that combined structured and text data from the VA EMR, and 2) testing and validating the algorithm using manual chart review of fiscal year (FY) 2008–2015 cardiac device procedures from the VA Clinical Assessment Reporting and Tracking-Electrophysiology (CART-EP) program and from a national dataset reviewed by members of the study team. The VA Boston Institutional Review Board approved this research prior to data collection and analysis. The study was also approved by the VA CART-EP program.

Data sources

The VA CART-EP program is a national quality program supporting all VA cardiac catheterization labs, where invasive cardiac procedures are performed. The program’s mission is to monitor and enhance the quality and safety of invasive cardiac procedures for Veterans through clinical analytics and information technology [12, 13]. Although reporting to the CART program is now mandatory for all invasive cardiac procedures, cardiac device procedures are voluntarily reported to the CART-EP application. During the study period, approximately 20% of cardiac device procedures performed across the national VA healthcare system were part of CART-EP. CART-EP does not collect information about pre- or post-procedural antibiotic use; however, these data were available as part of an earlier study that compared the effectiveness of different infection prevention strategies and used the gold-standard of manual review to extract data about pre-incision prophylaxis [14].

The data used in our algorithm were derived from the VA CDW, a single national data repository which is where structured data and clinical notes within the VA are stored [14, 15]. Data extracted from the VA CDW include antimicrobial orders and documentation of antimicrobial dispensing/administration as well as unstructured data, which include electronic clinical notes entered into the VA EMR. While notes have been available since FY2000, one VA facility does not push clinical note data from their EMR into the CDW. In addition, clinical notes that are not electronic (e.g., scanned documents in the EMR such as hand-written notes and PDF files of outside hospital records) are not available as searchable text data in CDW. We validated our algorithm using independent manual record review in the VA’s research EMR, VistAWeb [16].

Potential sources of antimicrobial prophylaxis data

Within the VA EMR, four potential sources of antimicrobial prophylaxis documentation were identified. The first was in the computerized order entry system in the form of antimicrobial orders. These orders could be entered either in the outpatient setting prior to a planned procedure, or in the inpatient settings prior to an urgent or emergent procedure. The second was antimicrobial administration, as documented in the VA bar coding system for inpatient orders, and outpatient prescriptions orders in the outpatient setting. The third was in clinical notes entered as electronic text notes into the VA EMR; these could be any typed note signed within the system within the 7-day window prior to the procedure and including the procedure date. The fourth was clinical notes written on paper and scanned before being entered as an attachment into the EMR; these included hand-written anesthesia records. All of the first three data sources were included for potential inclusion in the electronic algorithm and all four sources were used to establish a gold standard for whether antimicrobial prophylaxis was appropriately administered.

Study sample

There were 6497 cardiac device procedures, including implantations and revisions of permanent pacemakers, entered into the CART-EP application between FY2008–15. After the full cohort was created using the CART-EP program data, a sample of the procedures (n = 2102) underwent manual review; study team members extracted information about various infection prevention strategies, including type and administration of pre-operative antimicrobial prophylaxis. The comprehensive review strategy included: 1) review of clinical notes entered directly into the VA EMR, 2) review of orders and medication administration, and 3) review of scanned-in paper records, which include hand-written anesthesia and procedural notes. Details of the sampling procedure are previously described [14, 15]. Duplicate records and records without a verified procedure in the EMR corresponding to the date entered into CART-EP were excluded from the analysis.

Algorithm development and refinement process

We developed the algorithm to detect previously identified cases with and without antimicrobial prophylaxis in our gold standard dataset over several stages. First, we combined an initial set of data elements available from the VA EMR. Then, we added more data elements, removed elements with weaker performance, and iteratively arrived at the combination of data elements that best identified whether antimicrobial prophylaxis was given. The final version became our electronic measurement algorithm. Specifically, the optimization process was as follows. We began by including all the available data elements, e.g., electronic clinician text note extraction only, orders only, administration only, then added timing of the searches (e.g., including or excluding the procedure date), and then considered the types of antimicrobials, including an evaluation of only intravenous antimicrobials versus inclusion of oral medications.

Three clinicians (AA, PM, WBE) developed a comprehensive list of antimicrobials potentially administered for pre-incisional prophylaxis prior to cardiac device placement. The list of antimicrobials was based on 1) recommendations from surgical prophylaxis guidelines, which includes suggestions for appropriate prophylaxis prior to cardiac device procedures, and 2) clinician review and expertise [17]. A list of antimicrobial names, including both generic and brand-names and variations to account for spelling errors, was then created and this list was used to search clinical notes for documentation of administration. (See Additional file 1 for a listing of the text search terms used.) The list was also mapped to structured data in the VA EMR and orders and administration of relevant medications were extracted. For structured variables, a 7-day window prior to the procedure date was used, based on common clinical practice patterns, which may include entry of the pre-procedural antimicrobial at a pre-procedural cardiology visit.

Algorithm performance was assessed as sensitivity (how many true positive cases were identified as positive by the algorithm) and specificity (how many true negative cases were identified as negative by the algorithm). Manually reviewed CART-EP data was used as the gold standard for algorithm development. To finalize the algorithm, all discordant cases (e.g., algorithm flagged positive/CART-EP manual review was negative or algorithm flagged negative/CART-EP manual review was positive) underwent a second round of manual review by a different reviewer (EK or WBE) to identify the reasons for the discordant flag and to classify the reasons for discordance through expert analysis and discussion among clinicians on the study team. We used these findings to adjust the algorithm to improve accuracy.

Algorithm validation

We validated the algorithm by applying the relevant structured and unstructured data extracts to all cardiac device procedures performed in facilities with procedure volume ≥ 50 total cases within the VA healthcare system from FY 2016–17. A sample of these procedures underwent manual review (EK) to confirm whether the algorithm successfully identified whether antimicrobial prophylaxis was administered. Reasons for discordance were again assessed through expert assessment and discussion. We also conducted an analysis stratified by VA facility, to determine if facility-level effects were driving algorithm performance and accuracy.

Results

Algorithm development

The manually reviewed CART-EP dataset included 2102 procedures at 38 VA facilities with appropriately administered antimicrobial prophylaxis documented in 2056 cases (97.8%). We applied the various data elements described above (see Appendix for detailed descriptions of these elements) and found 1954 (93.0%) of the CART-EP cases had a positive text note search, 1899 (90.3%) had an antimicrobial order identified, and 150 (7.14%) had documentation of medication administration (Table 1). When text note searches were combined with antimicrobial orders, 2048 cases were flagged (97.4%); there were very few flags in the medication administration data (150, 7.2%) and adding this variable did not change either sensitivity or specificity as in all cases there was also an antimicrobial order present.

Table 1 Iterative performance of antimicrobial prophylaxis identification algorithm in development stage with gold standard CART-EP Program Data (manual review; n = 2102 procedures in 38 facilities)a

Some facilities routinely placed a drug order for the pre-operative antimicrobial during a pre-procedure visit into the VA computerized ordering system and then did not document type of antimicrobial in the clinician note about the procedure; however, there was documentation that “prophylaxis” or “antibiotics/antimicrobials” were administered prior to incision. Due to this documentation style, adjusting the algorithm to include orders entered into the computerized order entry system for an appropriate pre-operative antimicrobial substantially improved sensitivity, although specificity was slightly reduced (Table 1). Specificity of drug orders was limited primarily because antimicrobials could be prescribed for another reason (i.e., prescribed to treat an unrelated infection, such as bronchitis) and/or were administered after the procedure but none-the-less ordered during the pre-operative period.

We found 39 false negative and 27 false positive cases (66 procedures misclassified, 3.14%, Table 2). The most common reason for false-negatives was lack of documentation of a drug name and lack of an order in the clinical notes, but notation that pre-incisional prophylaxis was administered (e.g., procedure note stating, “antibiotics administered prior to incision” without specifying type, 27 cases; all of these were at the same facility). False negatives also included lack of documentation of a specific drug in the procedure and anesthesia records and order placed > 7 days prior to the procedure (4 cases). In these cases, the orders were placed weeks to months prior to the procedure and some cases may have been rescheduled during the intervening period, thus causing an unusually long lag time between the antimicrobial order and performance of the cardiac device procedure. Three cases were missed because the procedure note documenting administration of prophylaxis was entered after the procedure date; thus the algorithm measured the antimicrobial administration as post-procedural. One case was due to documentation of prophylaxis in paper records only, one case was due to incorrect procedure date entered into the VA EMR, and one case was due to incorrect procedure type entered (cardiac catheterization, not cardiac device procedure). We found no antimicrobial documentation in any notes in two cases, while in another case, no relevant clinical or pharmacy documentation was identified in the CDW.

Table 2 Reason for misclassification between antimicrobial prophylaxis algorithm and manual reviewa

Among the 27 false positive flags, 21 had antimicrobials documented in the clinician procedure note but administered after the procedure (e.g., “give cefazolin x 3 doses post-procedure, then Keflex x 5 days” documented at the end of the note, with no pre-incision doses administered). Some cases were incorrectly flagged by the algorithm for several reasons, including documentation that prophylaxis was given but not which type (2 cases), patient on antimicrobials for unrelated reason (e.g., treatment of chronic obstructive pulmonary disorder or bronchitis) at the time of the procedure (3 cases), antimicrobials used as part of a flush or wash during the procedure, but not given systemically (5 cases), and documentation of antimicrobial allergy (e.g., cefazolin allergy) in the clinician procedure note, but no administration of the drug.

Based on the initial findings, several updates to the algorithm were made (Table 1, second stage). All duplicate entries were removed and cases with incorrect non-cardiac device procedures incorrectly coded were excluded. Then, several variations to optimize algorithm performance were tested. First, as many of the false-positive flags were due to oral antimicrobials administered post-procedure but documented and recommended in the clinician note about the procedure, we excluded antimicrobials rarely used for prophylaxis and agents that are only available in oral formulation from all data sources (text searches, orders, administration). Second, penicillin, which is rarely used for prophylaxis but commonly documented as an allergy, was also excluded from the text note searches. Third, antimicrobial orders were limited to those lasting < 24 h, to reduce the number of false-positives that arose from patients receiving antimicrobials for treatment of unrelated infections, which are generally prescribed for longer periods of time. Lastly, cases with antimicrobial orders placed on the day of the procedure but after the time of the procedure were excluded; post-procedure antimicrobial administration is both ineffective and against clinical guidelines and should not be used to assess whether a patient has received appropriate pre-procedural antimicrobial prophylaxis. The optimal algorithm, which included a limited set of antimicrobial orders and excluded post-incisional antimicrobials, had a sensitivity of 98.1% and a specificity of 41.3%.

Algorithm validation

The final algorithm was applied to the validation cohort of VA patients who underwent cardiac device procedures from FY2016–17 (n = 18,903). Unlike procedures identified in CART-EP, which included 38 facilities, this cohort included all 65 VA facilities that performed at least 50 cardiac device procedures during the study period. The final algorithm identified pre-procedural antimicrobials in 16,606 cases (87.8%).

A random sample of 50 cases with a positive flag and 50 cases with a negative flag then underwent manual review; 48 positively flagged cases were true positives (sensitivity, 96%) whereas 10 of the negatively flagged cases were true negatives (specificity, 20%). The two false-positive cases were both due to documentation of antimicrobial administration in the clinical procedure notes but administered after the procedure (Table 2). The false negative cases (40) were primarily due to EMR documentation of antimicrobial administration only in scanned paper records that were not searchable by the algorithm (70%); this limitation was clustered by facility. Other false negative cases had mention of antimicrobial administration in the typed record, but the antibiotic name was only mentioned in scanned notes that were not available for electronic text searches (27.5%). One false negative was due to an incorrect procedure date (2.5%).

Facility-level variation

We found facility variation in both the development (Fig. 1) and validation data (Fig. 2). In comparison with CART-EP findings, in 25 of 38 facilities (66%), the algorithm and manual review results were identical, and compliance was 100%. In 8 facilities, the algorithm identified higher levels of compliance than the levels identified in the CART-EP review, whereas in 5 facilities the algorithm found lower compliance than the CART-EP manual review. In the validation data set, there was substantial variation on the facility level. Among 65 distinct VA facilities, 55 representing 15,450 (81.7%) of cases had prophylaxis identified in the algorithm in > 80% of cardiac device cases (Fig. 1); 5 facilities had prophylaxis compliance rates between 60 and 80% suggesting the algorithm may have accurately identified poor adherence to clinical guidelines, and 5 facilities had prophylaxis identified in < 60% of cases, suggesting that the algorithm may have low performance potentially due to clinical documentation practices that may limit the ability of an automated system to accurately measure pre-incisional prophylaxis, potentially due to ongoing reliance on hand-written notes in these facilities.

Fig. 1
figure1

Algorithm Development: Facility Variation in Detection of Pre-Incisional Prophylaxis Among CART-EP Cardiac Device Procedures (n = 2102 procedures in 38 facilities)

Fig. 2
figure2

Algorithm Validation: Facility Variation in Detection of Pre-Incisional Prophylaxis Among Patients Undergoing Cardiac Device Procedures (n = 18,903 procedures in 65 facilities)

Discussion

Appropriate pre-incisional antimicrobial prophylaxis is a cornerstone of prevention of procedure-related infections, including cardiac device infections. Prior work demonstrates that pre-incisional antimicrobial prophylaxis can reduce the incidence of infections by up to 80% [3, 18]. However, resources available for monitoring this key quality metric have been limited. To address this need, we created a novel clinical informatics tool to measure appropriate, guideline-concordant pre-incision antimicrobial prophylaxis for cardiac device procedures. Methodology developed can be adapted for application in other procedural settings, including for traditional surgical procedures to automate and expand quality monitoring activities. We found our electronic identification algorithm was highly accurate in most facilities; however, where EMR documentation included scanned hand-written notes, the algorithm found compliance rates of 60% or less and thus might not accurately measure quality in these settings. As EMR use increases in the VA, and hand-written notes are increasingly not used for clinical documentation, implementation of this tool could be used to expand quality measurement and reduce the burden of manual review to identify appropriate care, particularly in light of the transition to a Cerner EMR [19, 20].

We identified several potential barriers to disseminating this automated, clinical informatics tool throughout the VA – these concerns are likely to also affect non-VA facilities with modern EMRs. First, there was wide variety in how pre-incisional antimicrobial prophylaxis was documented. In one facility, no orders were placed and no specific antimicrobial was documented in the EMR; however, there was text in the clinical note that prophylaxis not otherwise specified had been administered. Thus, methods aimed to measure specific antimicrobial type will not be effective for capturing whether appropriate prophylaxis was given in these procedures. A potential way to adjust the algorithm is to include “prophylaxis” as a text search, however, this would also collect “no prophylaxis given” and “post-incision prophylaxis” as false-positives. Due to the potential for high rates of false positives with this term, and the limited value it demonstrated for improving algorithm performance, we elected not to use it in our tool.

Another reason for false-negative results was documentation only in scanned-in, hand-written paper records that did not include electronic text search functionality. Hand-written paper records may have been more common during the time-period of the development cohort (FY2005–15); thus, as EMRs and electronic anesthesia documentation is increasingly adopted, this may be less of an implementation challenge in the future as there are emerging technologies for converting text in scanned files and hand-written documentation is increasingly de-implemented [21]. Two cases were missed because the clinician procedure note was entered after the procedure date. Due to the high rate of post-procedural antimicrobials in this population, approaching 50% of cases [22, 23], we elected not to extend the timeframe of the text note searches to include the period after the procedure, as doing so would inappropriately flag guideline discordant post-procedural administration as guideline-concordant and would greatly increase the number of false positive cases.

The last substantial reason for false-negatives was an order placed far in advance of the procedure, potentially because the procedure was rescheduled in the interim. Increasing the timeframe for the searches to a period greater than 7 days prior to the intervention greatly increases the false positive rate, as the probability of an intervening antimicrobial prescription for another reason, unrelated to prophylaxis, greatly increases. Thus, altering the tool to capture these additional prophylaxis orders was not pursued. However, if the goal is to maximize sensitivity, then this would be a potential strategy to enhance case ascertainment.

False positive flags were attributed to several different factors. As noted above, inappropriate and ineffective post-procedural antimicrobial use is common in the cardiac device population [22, 24]. These post-incisional antimicrobials were often documented in the clinician note about the procedure and thus were identified during text searches of specific antimicrobial types. Limiting the false-positives that arise from post-incisional antimicrobials is challenging. The clinician note about the procedure is the place where prophylaxis is most commonly documented in the EMR and excluding it would create many more false-negative cases. Prior to adjustment of the antimicrobial list, several cases had a false-positive flag due to documentation of an antimicrobial allergy, typically a penicillin allergy, which is the most commonly documented antimicrobial allergy [25], in the clinician note. However, these antimicrobials are rarely used for prophylaxis, and after we removed them from the EMR text note searches, we found fewer false positives arising from allergy documentation.

The algorithm could be improved with better adherence to documenting antimicrobial orders and administration in existing structured data fields. The opportunity to create structured procedure or prophylaxis notes within the VA EMR may arise as the VA transitions to the Cerner EMR. Another way to improve algorithm performance may be through deep machine learning approaches. These methods create algorithms using a corpus of previously reviewed cases to train the computer to detect patterns of interest, and have been useful in isolating information like templated questions and answers documented in VA clinical notes [26]. .Application of these complex algorithms many not be practical given the facility variation in EMR documentation we discovered and the upcoming VA EMR transition [27]. This is a promising opportunity for future research.

Our study was limited in several ways. First, our initial algorithm was developed using the data from the VA CART-EP program, which had voluntary participation and only a subset of VA facilities and procedures were entered into the database. We attempted to address this limitation by validating the algorithm using a national dataset including all facilities and all cardiac device procedures. When the algorithm was applied to all VA facilities, we found substantial interfacility variation, which may cause implementation challenges if EMR documentation practices do not evolve. Applications developed for use within the VA EMR may not be directly applicable to other EMR systems; however, the programming used in our algorithm is relatively straightforward, and the most commonly used systems outside of the VA, including Cerner and EPIC, have text note searching options [19, 28]. Structured variables, such as antimicrobial orders, are generally easily extracted from any electronic order entry system [10].

Conclusions

We developed a methodology based on structured and unstructured data elements from the EMR that could be applied for real-time quality measurement within the VA healthcare system. This method could be adapted for other procedural areas where antimicrobial prophylaxis is recommended but comprehensive measurement has been limited to resource-intense manual review.

Availability of data and materials

The data that support the findings of this study are not publicly available due to federal data policies; however, the authors can provide limited de-identified data upon reasonable request and with permission of the VA Boston Healthcare System Data Security and Privacy office.

Abbreviations

Abx:

Antimicrobial

CART-EP:

VA Clinical Assessment Reporting and Tracking-Electrophysiology

CDW:

Corporate Data Warehouse

EMR:

Electronic medical record

NPV:

negative predictive value

PPV:

positive predictive value

VA:

Veterans Health Administration

References

  1. 1.

    Baddour LM, Epstein AE, Erickson CC, Knight BP, Levison ME, Lockhart PB, Masoudi FA, Okum EJ, Wilson WR, Beerman LB, et al. Update on cardiovascular implantable electronic device infections and their management: a scientific statement from the American Heart Association. Circulation. 2010;121(3):458–77.

  2. 2.

    Margey R, McCann H, Blake G, Keelan E, Galvin J, Lynch M, Mahon N, Sugrue D, O'Neill J. Contemporary management of and outcomes from cardiac device related infections. Europace. 2010;12(1):64–70.

  3. 3.

    Berríos-Torres SI, Umscheid CA, Bratzler DW, Leas B, Stone EC, Kelz RR, Reinke CE, Morgan S, Solomkin JS, Mazuski JE, et al. Centers for Disease Control and Prevention guideline for the prevention of surgical site infection, 2017CDC guideline for the prevention of surgical site infection, 2017CDC guideline for the prevention of surgical site infection, 2017. JAMA Surg. 2017;152(8):784–91.

  4. 4.

    Anderson DJ, Podgorny K, Berrios-Torres SI, Bratzler DW, Dellinger EP, Greene L, Nyquist AC, Saiman L, Yokoe DS, Maragakis LL, et al. Strategies to prevent surgical site infections in acute care hospitals: 2014 update. Infect Control Hosp Epidemiol. 2014;35(6):605–27.

  5. 5.

    Bratzler DW, Dellinger EP, Olsen KM, Perl TM, Auwaerter PG, Bolon MK, Fish DN, Napolitano LM, Sawyer RG, Slain D, et al. Clinical practice guidelines for antimicrobial prophylaxis in surgery. Am J Health Syst Pharm. 2013;70(3):195–283.

  6. 6.

    Bratzler DW, Houck PM, Richards C, Steele L, Dellinger EP, Fry DE, Wright C, Ma A, Carr K, Red L. Use of antimicrobial prophylaxis for major surgery: baseline results from the National Surgical Infection Prevention Project. Arch Surg. 2005;140(2):174–82.

  7. 7.

    Rosenberger LH, Politano AD, Sawyer RG. The surgical care improvement project and prevention of post-operative infection, including surgical site infection. Surg Infect. 2011;12(3):163–8.

  8. 8.

    Specifications Manual for Joint Commission National Quality Core Measures (2010A1): Measure Information Form SCIP-Inf-3 [https://manual.jointcommission.org/releases/archive/TJC2010B/MIF0112.html]. Accessed 3 Aug 2018.

  9. 9.

    Kuper KM, Nagel JL, Kile JW, May LS, Lee FM. The role of electronic health record and “add-on” clinical decision support systems to enhance antimicrobial stewardship programs. Infect Control Hosp Epidemiol. 2019;40(5):501–11.

  10. 10.

    Wisniewski MF, Kieszkowski P, Zagorski BM, Trick WE, Sommers M, et al. Development of a clinical data warehouse for hospital infection control. J Am Med Inform Assoc. 2003;10(5):454–62.

  11. 11.

    US Department of Veterans Affairs: 172VA10P2: VHA Corporate Data Warehouse - VA. 79 FR 4377. Updated September 30, 2017. Accessed 1 Oct 2018.

  12. 12.

    Byrd JB, Vigen R, Plomondon ME, Rumsfeld JS, Box TL, Fihn SD, Maddox TM. Data quality of an electronic health record tool to support VA cardiac catheterization laboratory quality improvement: the VA clinical assessment, reporting, and tracking system for Cath labs (CART) program. Am Heart J. 2013;165(3):434–40.

  13. 13.

    Tsai TT, Box TL, Gethoffer H, Noonan G, Varosy PD, Maddox TM, Fihn SD, Gross TP, Jesse RL, Rumsfeld JS. Feasibility of proactive medical device surveillance: the VA clinical assessment reporting and tracking (CART) program. Med Care. 2013;51(3 Suppl 1):S57–61.

  14. 14.

    Asundi A, Stanislawski M, Mehta P, Baron AE, Mull HJ, Ho PM, Zimetbaum PJ, Gupta K, Branch-Elliman W. Real-world effectiveness of infection prevention interventions for reducing procedure-related cardiac device infections: insights from the veterans affairs clinical assessment reporting and tracking program. Infect Control Hosp Epidemiol. 2019;40(8):855–62.

  15. 15.

    Asundi A, Stanislawski M, Mehta P, Baron AE, Gold H, Mull H, Ho PM, Gupta K, Branch-Elliman W. Prolonged antimicrobial prophylaxis following cardiac device procedures increases preventable harm: insights from the VA CART program. Infect Control Hosp Epidemiol. 2018;39(9):1030–6.

  16. 16.

    VA Office of Information and Technology (August 2016). “VistAWeb Version 16.1.8.2 User Manual.” Retrieved 1/23/20, from https://www.va.gov/vdl/documents/Clinical/VistaWeb/vistawebusermanual.pdf.

  17. 17.

    Bratzler DW, Dellinger EP, Olsen KM, Perl TM, Auwaerter PG, Bolon MK, Fish DN, Napolitano LM, Sawyer RG, Slain D, et al. Clinical practice guidelines for antimicrobial prophylaxis in surgery. Surg Infect. 2013;14(1):73–156.

  18. 18.

    de Oliveira JC, Martinelli M, Nishioka SA, Varejao T, Uipe D, Pedrosa AA, Costa R, D'Avila A, Danik SB. Efficacy of antibiotic prophylaxis before the implantation of pacemakers and cardioverter-defibrillators: results of a large, prospective, randomized, double-blinded, placebo-controlled trial. Circ Arrhythm Electrophysiol. 2009;2(1):29–34.

  19. 19.

    Cerner EHR Clinical Solutions [https://www.cerner.com/solutions/clinical-solutions]. Accessed 22 May 2019.

  20. 20.

    EHR Modernization Homepage [https://www.ehrm.va.gov/]. Accessed 9 Aug 2019.

  21. 21.

    Bui DD, Del Fiol G, Jonnalagadda S. PDF text classification to leverage information extraction from publication reports. J Biomed Inform. 2016;61:141–8.

  22. 22.

    Branch-Elliman W, Stanislawski M, Strymish J, Baron AE, Gupta K, Varosy PD, Gold HS, Ho PM. Cardiac electrophysiology laboratories: a potential target for antimicrobial stewardship and quality improvement? Infect Control Hosp Epidemiol. 2016;37(9):1005–11.

  23. 23.

    Mehrotra P, Gupta K, Strymish J, Kramer DB, Lambert-Kerzner A, Ho PM, Branch-Elliman W. Implementation of infection prevention and antimicrobial stewardship in cardiac electrophysiology laboratories: results from the SHEA research network. Infect Control Hosp Epidemiol. 2017;38(4):496–8.

  24. 24.

    Basil A, Lubitz SA, Noseworthy PA, Reynolds MR, Gold H, Yassa D, Kramer D. Periprocedural antibiotic prophylaxis for cardiac implantable electrical device procedures: results from a Heart Rhythm Society survey. JACC Clin Electrophysiol. 2017;3(6):632–4.

  25. 25.

    Macy E. Penicillin and beta-lactam allergy: epidemiology and diagnosis. Curr Allergy Asthma Rep. 2014;14(11):476.

  26. 26.

    Divita G, Shen S, Carter ME, Redd A, Forbush T, Palmer M, Samore MH, Gundlapalli AV. Recognizing questions and answers in EMR templates using natural language processing. Stud Health Technol Inform. 2014;202:149–52.

  27. 27.

    Branch-Elliman W, Strymish J, Kudesia V, Rosen AK, Gupta K. Natural language processing for real-time catheter-associated urinary tract infection surveillance: results of a pilot implementation trial. Infect Control Hosp Epidemiol. 2015;36(9):1004–10.

  28. 28.

    Heidemann L, Law J, Fontana RJ. A text searching tool to identify patients with idiosyncratic drug-induced liver injury. Dig Dis Sci. 2017;62(3):615–25.

Download references

Acknowledgements

We would like to thank the VA Clinical Assessment Reporting and Tracking Program for their support.

Funding

This work was supported by VA HSR&D grant PO 18–031 (PI WBE). WBE is supported by NIH NHLBI 1K12HL138049–01.

Author information

Affiliations

Authors

Contributions

WBE had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. She attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. WBE affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained. Study concept and design: WBE, HJM, KS, MSL. Acquisition of data: WBE, KS, AA, PM, EK, MSH. Analysis and interpretation of data: WBE, HJM, KS, MS. Drafting of the manuscript: HJM, WBE, KS. Critical revision of the manuscript for important intellectual content: HJM, WBE, KS, EK, MHS, MSL, AA, PM, MS. Final Approval of the version to be published: WBE, HJM. Statistical analysis: KS, HJM. Administrative, technical or material support: WBE. Study supervision: WBE. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hillary J. Mull.

Ethics declarations

Ethics approval and consent to participate

VA Boston Healthcare System Intuitional Review Board approved this research with a waiver of documented informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Appendix

Appendix

Table 3 Algorithm components for measurement of pre-procedural antimicrobial use in cardiac device procedures

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mull, H.J., Stolzmann, K., Kalver, E. et al. Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration’s electronic medical record. BMC Med Inform Decis Mak 20, 15 (2020). https://doi.org/10.1186/s12911-020-1031-5

Download citation

Keywords

  • Antibiotic prophylaxis
  • Algorithm
  • Veterans health administration
  • Cardiac device procedure
  • Quality measurement