This article has Open Peer Review reports available.
Enhanced health event detection and influenza surveillance using a joint Veterans Affairs and Department of Defense biosurveillance application
© Lucero et al; licensee BioMed Central Ltd. 2011
Received: 10 February 2011
Accepted: 19 September 2011
Published: 19 September 2011
The establishment of robust biosurveillance capabilities is an important component of the U.S. strategy for identifying disease outbreaks, environmental exposures and bioterrorism events. Currently, U.S. Departments of Defense (DoD) and Veterans Affairs (VA) perform biosurveillance independently. This article describes a joint VA/DoD biosurveillance project at North Chicago-VA Medical Center (NC-VAMC). The Naval Health Clinics-Great Lakes facility physically merged with NC-VAMC beginning in 2006 with the full merger completed in October 2010 at which time all DoD care and medical personnel had relocated to the expanded and remodeled NC-VAMC campus and the combined facility was renamed the Lovell Federal Health Care Center (FHCC). The goal of this study was to evaluate disease surveillance using a biosurveillance application which combined data from both populations.
A retrospective analysis of NC-VAMC/Lovell FHCC and other Chicago-area VAMC data was performed using the ESSENCE biosurveillance system, including one infectious disease outbreak (Salmonella/Taste of Chicago-July 2007) and one weather event (Heat Wave-July 2006). Influenza-like-illness (ILI) data from these same facilities was compared with CDC/Illinois Sentinel Provider and Cook County ESSENCE data for 2007-2008.
Following consolidation of VA and DoD facilities in North Chicago, median number of visits more than doubled, median patient age dropped and proportion of females rose significantly in comparison with the pre-merger NC-VAMC facility. A high-level gastrointestinal alert was detected in July 2007, but only low-level alerts at other Chicago-area VAMCs. Heat-injury alerts were triggered for the merged facility in June 2006, but not at the other facilities. There was also limited evidence in these events that surveillance of the combined population provided utility above and beyond the VA-only and DoD-only components. Recorded ILI activity for NC-VAMC/Lovell FHCC was more pronounced in the DoD component, likely due to pediatric data in this population. NC-VAMC/Lovell FHCC had two weeks of ILI activity exceeding both the Illinois State and East North Central Regional baselines, whereas Hines VAMC had one and Jesse Brown VAMC had zero.
Biosurveillance in a joint VA/DoD facility showed potential utility as a tool to improve surveillance and situational awareness in an area with Veteran, active duty and beneficiary populations. Based in part on the results of this pilot demonstration, both agencies have agreed to support the creation of a combined VA/DoD ESSENCE biosurveillance system which is now under development.
The United States faces many potential threats to human health, including disease outbreaks, accidental environmental exposures, and acts of terrorism. Effective biosurveillance at a government agency level is dependent upon representative and timely data collection, proper analysis and interpretation of data at the local and regional levels, and incorporation of the national perspective and visualization techniques to provide a broad, accurate picture across regions or jurisdictions. Achieving these steps is critical in order to obtain early warning of health risks, early detection of health events and overall situational awareness of disease or potential bioterrorist activity.
Developing robust biosurveillance capabilities within government agencies and having the ability to share information across agencies has become increasingly important in recent times . Linkage of federal agency biosurveillance capabilities and health data exchange is an important component of multiple laws, initiatives and directives including Public Law 110-53 (Implementing Recommendations of the 9/11 Commission Act of 2007) , the National Biosurveillance Integration Center (NBIC) code , Homeland Security Presidential Directive (HSPD)-10 (Biodefense for the 21st Century) , HPSD-21 (Public Health and Medical Preparedness) , Public Law 109-417 (Pandemic All-Hazards and Preparedness Act) , the Federal Health Architecture initiative , and the National Biosurveillance Strategy for Human Health . The Departments of Defense (DoD) and Veterans Affairs (VA) are among the largest healthcare systems providing care in the United States. In 2010, the VA healthcare system had 8.3 million Veterans enrolled and over 75 million out-patient visits and 679,600 inpatient admissions . In 2010, the Military Health System (TRICARE) had 9.6 million eligible beneficiaries and approximately 42 million direct care out-patient visits and 265,000 inpatient admissions . Persons who served in the active military and were discharged or released under conditions other than dishonorable may qualify for VA healthcare benefits. The VA and DoD healthcare systems operate independently and maintain separate electronic health record (EHR) systems. In 2004 there was a recognized need to exchange information between the two EHRs in order to provide a more seamless transition of care from DoD to VA. In response, the Federal Health and Bidirectional Health Information Exchange (FHIE/BHIE) interagency program was launched to permit secure, real-time exchange of medical record data between VA and DoD. However, the information exchange is currently incomplete and there is no comprehensive system in place that allows for streamlined transition of health care records between the two agencies. In 2009, President Obama, VA Secretary Shinseki and DoD Secretary Gates announced that DoD and VA had taken the first steps in creating a unified Virtual Lifetime Electronic Record (VLER) . The program is currently in a pilot phase, testing exchange of information over the Nationwide Health Information Network (NHIN). In the future, this system will provide connectivity that has never before been accomplished and will greatly improve access to health information within the federal government as well as private sector partners nationwide, providing a continuous health record from time of enlistment to death.
Both Departments are committed to developing and maintaining state-of-the-art biosurveillance systems for monitoring disease activity and emerging health threats. Despite the fact that many DoD and VA facilities exist in common geographic areas, biosurveillance is currently being performed independently by each Department using a system developed by the Johns Hopkins University Applied Physics Laboratory (JHUAPL) and Walter Reed Army Institute of Research known as ESSENCE (Electronic Surveillance System for the Early Notification of Community-Based Epidemics) [12–14]. ESSENCE is also the primary biosurveillance system in place at many state and local health departments, however these systems do not contain any VA or DoD hospital data in their applications [15–18]. The DoD and VA ESSENCE systems utilize complex algorithms to evaluate International Classification of Diseases, 9 th Revision (ICD-9) diagnosis codes from outpatient and emergency department visits. Codes are clustered into broad syndrome groups which are designed to capture both naturally occurring disease outbreaks and potential bioterrorist events. Although the two systems provide very similar functionality, they are completely independent and there is no formal sharing of biosurveillance data between systems or across agencies.
VA and DoD recognize that performing joint health surveillance could provide the agencies' public health personnel with additional indicators that may not be detectable in their respective individual surveillance populations. As an example, influenza outbreaks are often first seen in the pediatric population and may spread more rapidly among the very young given the extent of their social interactions. Therefore, early influenza activity and trends may be missed by VA surveillance which has little to no pediatric data. On the other hand, older patients and those with multiple co-morbid conditions often suffer much higher morbidity and mortality due to influenza. As a result, the severity of an influenza season or pandemic may be underestimated by the DoD system, which is overrepresented by younger and otherwise healthy individuals. In addition, combining surveillance populations provides a larger statistical sample of total population in the region for earlier recognition of disease events and monitoring the spread of a communicable disease through a community. Prior evaluations of biosurveillance systems with multiple data streams found that data from different sources was useful for detecting and monitoring disease outbreaks and provide a substantial alerting-timeliness advantage [19–21]. ESSENCE has previously been linked with disparate systems for real-time data exchange and communication . In that project, data sharing was simplified though a service-oriented architecture and the accurate and timely information exchange that was achieved displayed potential to enhance effective collaboration and improve patient care. A joint surveillance approach would likely also provide additional power for the detection of rare disease events, emerging health threats, and bioterrorist attacks. The clinical application of an enhanced, joint surveillance system would be beneficial to healthcare providers and public health practitioners in both Departments.
Although the VA and DoD ESSENCE systems are not currently integrated, an opportunity to evaluate the benefits of combined surveillance exists in North Chicago, IL where VA and DoD have partnered to create the first fully integrated federal healthcare facility. The Great Lakes Naval Training Center is the home to the U.S. Navy's only Recruit Training Command and is the central processing location for all Naval recruits. The majority of trainees are in the 18-25 year age group. Starting in June 2006, the Naval Health Clinics-Great Lakes inpatient, surgical, emergency and some limited outpatient services for active duty personnel and their dependents were relocated to North Chicago Veterans Affairs Medical Center (NC-VAMC). This consolidation includes a project to increase interoperability between VA and DoD electronic health records by allowing bidirectional flow of data. Full integration of all inpatient and outpatient services occurred October 1, 2010 when all DoD care and medical personnel had relocated to the expanded and remodeled NC-VAMC campus and at that time the facility was officially renamed Captain James A. Lovell Federal Health Care Center (Lovell FHCC). Both agencies have agreed to maintain and support each others' main mission, highest quality patient care for Veterans and operational readiness of military personnel. The merger of these VA and DoD facilities provided an opportunity to evaluate data in a combined VA/DoD patient population without the establishment of a formal biosurveillance sharing agreement or data exchange. Biosurveillance data from active duty military personnel and their dependents that have been seen and treated at NC-VAMC/Lovell FHCC starting June 1, 2006, is available in the VA ESSENCE system for analysis.
The objectives of this study were: 1) Describe changes in facility and patient demographics resulting from the VA/DoD integration in North Chicago, 2) Determine if health event detection and influenza surveillance in the joint facility was enhanced compared with two other VA-only facilities in the Chicago-metro area (Jesse Brown VAMC and Hines VAMC), 3) Examine the combined surveillance data in relation to the VA-only and DoD-only components within ESSENCE.
The study was a retrospective analysis of VA ESSENCE surveillance data for the three Chicago-area VAMCs (NC-VAMC/Lovell FHCC, Jesse Brown VAMC and Hines VAMC). The NC-VAMC/Lovell FHCC is located in Lake County, IL, 33 miles north of downtown Chicago and provides care for active duty military, beneficiaries and Veterans throughout Northern Illinois and Southern Wisconsin. The Jesse Brown VAMC is located in adjacent Cook County, IL and provides care to enrolled Veterans in the City of Chicago; Cook County, IL; and in six counties in northwestern Indiana. The Hines VAMC is located 12 miles west of downtown Chicago also in Cook County, IL and provides care to Veterans primarily from Cook, DuPage, La Salle and Will counties. The Stanford University institutional review board approved this study. Analysis was conducted using VA ESSENCE Version 1.4. The system collects information from all VA Medical Centers and community based outpatient clinics (CBOCs) in all 50 states, plus Guam, American Samoa, Puerto Rico, U.S. Virgin Islands and the Philippines. VA ESSENCE pulls ICD-9 diagnosis codes and demographic data from all outpatient and emergency department visits. A limited number of inpatient visits and consultations are also captured. Visits are grouped into a VA ESSENCE syndrome category if one or more of the assigned ICD-9 code(s) for the visit matches a defined code for the syndrome. Syndrome categories include: Botulism-like, Febrile Disease, Fever, Gastrointestinal (GI), Hemorrhagic Illness, Influenza-like Illness (ILI), Miscellaneous/Asthma, Neurological, Rash, Respiratory and Shock/Coma. ICD-9 codes can be analyzed individually or at the syndrome level.
Data are analyzed by the system's aberrancy-detection algorithms to identify when observed counts are significantly above predicted values. The alerting algorithm methodology for ESSENCE syndromic surveillance data has been described previously [23, 24]. VA ESSENCE determines an expected count by employing regression modeling based on historical data, day-of-the week effects, seasonal trends and effects due to other factors. After determining the expected count, the system applies statistics and runs significance tests for each syndrome or ICD-9 code to determine whether the observed counts are reasonably close to what is expected from the model predictions. When the test for reasonable agreement fails, VA ESSENCE produces flags (red or yellow alerts) to indicate a count, which is significantly above the predicted level. Alerts are designed to warn the system user of a possible outbreak or cluster. The tests for reasonable agreement employ confidence intervals (CIs). When the observed count falls between the 95% and 99% CI a low-level (yellow) alert is generated. If the count exceeds the 99% CI, a high-level (red) alert is triggered.
We compared aggregate patient and facility demographic data in VA ESSENCE prior to (August 2005-May 2006) and after (June 2006-March 2009) integration began at the VA and DoD North Chicago facilities. Data elements evaluated included: total number of visits captured per month, patient age and gender, syndrome groups and ICD-9 diagnosis codes. Data were exported from VA ESSENCE to Microsoft Excel for analysis. For visits with an ICD-9 code appearing in multiple syndrome categories, as well as those visits assigned multiple different ICD-9 codes, and in cases where the patient had more than one outpatient visit on a single date, duplicates were identified and removed in Microsoft Excel to ensure that each visit was counted only once in the analysis. Because the live DoD ESSENCE system only maintains the most recent 18 months of data, DoD data from the 2005-2006 pre-integration time period was not readily available through the active DoD ESSENCE application. However, archived DoD ESSENCE emergency department (ED) data was obtained from electronic storage for comparison, as it was this department which initially transferred to the VA facility at the start of the merger. Descriptive statistics and univariate analysis for this study were performed using SAS 9.1 (SAS Institute, Cary, NC).
We searched the Chicago Department of Public Health online archives for disease outbreaks and major health events occurring between June 2006 and March 2009. The archive consisted primarily of press releases and health warnings issued by the health department. We attempted to identify events that were large in magnitude as well as ones with the potential to affect the entire metro-Chicago area in order to maximize the chances of activity within the Chicago-area VA system. The majority of events were not suitable for this analysis as they involved only a small number of cases or were limited to a single non-VA hospital or other facility. However, we identified and selected one infectious disease outbreak (Salmonella, Taste of Chicago Festival, July 2007) and one weather-related health event (Heat Wave, multiple days, July 2006) in the Chicago-metro area for evaluation. These reports were the two identified which represented large regional events that were most likely to have affected the cities of Chicago and North Chicago. The Taste of Chicago is one of the world's largest annual food festivals, beginning the Friday before Independence Day and ending the Sunday after. During the 2007 event (June 29-July 8, 2007), a Salmonella outbreak occurred in which approximately 800 people became ill and nearly 40 were hospitalized. The outbreak was eventually traced to hummus served by a single vendor [25, 26]. The second event was the 2006 North America heat wave, which spread through most of the United States and Canada, producing extreme temperatures, which led to the death of at least 225 people. Chicago was among the U.S. cities affected during July and August of 2006. Extreme temperature advisories were issued by the Chicago City Health Department on July 14 and 31, 2006 [27, 28]. National Weather Service climate summary for Chicago 2006, reported that the first wave occurred from July 13-17th with the warmest day on July 15th when the high reached 97 degrees. The second wave was approximately July 29-August 4th with the warmest days on July 31st and August 1st with high temperatures of 99 degrees both days . Neither the GI nor the heat wave event was reviewed in our systems prior to selection for analysis in this project. For this analysis, we reviewed VA ESSENCE gastrointestinal syndrome and heat-injury data (ICD-9: 992) during these two time periods at each of the three Chicago VAMCs. Regression/Exponentially Weighted Moving Average (EWMA) was used as the VA ESSENCE alerting algorithm.
Finally, we used available VA ESSENCE data to determine the weekly percentage of visits for ILI at each Chicago-area facility for the 2007-2008 influenza season and compared it with the Centers for Disease Control and Prevention (CDC) regional baseline for the East North Central region (Illinois, Indiana, Michigan, Ohio and Wisconsin) and the Illinois State baseline. We then plotted VA ESSENCE ILI data against the CDC/Illinois Sentinel Provider Network (ILINet) and Chicago's Cook County ESSENCE emergency department visits for ILI during the same influenza season. Data for the CDC/Illinois Sentinel Provider Network was obtained from the Illinois Department of Public Health Influenza Surveillance 2007-2008 Flu Activity Report . Providers report number of patients seen and the number with ILI, defined as fever ≥ 100°F and a cough and/or sore throat. Data from the Cook County ESSENCE system was obtained from the ESSENCE Surveillance Data for Influenza-Like Illness from the Cook County Department of Public Health Sentinel Influenza Surveillance Report . Twenty-five hospital EDs in the county participate in Cook County ESSENCE, however there is no VA or DoD data in their system. The Cook County ESSENCE ILI syndrome is defined as a symptom complex of fever and cough or sore throat. ILI data from the DoD ESSENCE system has previously been found to have a very strong correlation with CDC ILINet data . For the VA ESSENCE system we included visits with an ICD-9 diagnosis code of influenza, fever, cough or sore throat to approximate the ILI definition used by ILINet and Cook County ESSENCE. We also limited the analysis to VA ED, urgent care and primary care clinics in order to more closely match the outpatient settings contained in these other systems.
The purpose of our evaluation was to investigate the potential advantages of a consolidated biosurveillance application combining the distinct and unique patient populations of our two agencies. This study represents the first example of merging and analysis of biosurveillance data between VA and DoD. Our study, which was conducted in a single integrated VA/DoD facility in North Chicago, highlights the improvements achieved in sample size, along with age and gender diversity in a combined VA/DoD patient pool. Although these changes were an expected outcome of the facility merger, it represents a unique patient mix, which does not exist in any other VA facility and has not been described previously. Median patient age in this facility fell by more than 15 years following the merger. Prior to integration, there were no visits in the pediatric age groups in the VA data, however post-integration they rose to more than 10% of the visits. The proportion of visits in the 65 and over age group fell from 47% to 26%. As a comparison, the U.S. Census Bureau ACS reported that 24% of the North Chicago population was under 18 years of age and just 6% was age 65 years and over . Although the facility is not necessarily a representative sample of the community population, the patient mix in the merged facility now more closely resembles the population characteristics of North Chicago.
The analysis also revealed potential improvements in temporal alerting and flu trending demonstrated by the review of alerts generated for two large public health events and seasonal ILI trends when compared with two Chicago-area VA-only medical centers which do not include any pediatric data. For syndromes such as ILI where one population routinely demonstrates a higher concentration of disease or earlier peak in disease frequency, our results suggest that it may be beneficial to perform sub-group analyses of the individual VA and DoD populations, in addition to the combined population analysis, in order to better characterize disease trends. The benefits of joint VA/DoD biosurveillance were also suggested based on comparisons between the combined population with the VA-only and DoD-only component sub-analysis performed at the NC-VAMC/Lovell FHCC. In situations where health threats exist, accurate and readily available disease surveillance data is critically important, as it not only improves the execution of emergency management procedures, but also the appropriate and timely diversion of resources. Furthermore, early detection of disease trends could alert facilities with large numbers of active duty personnel and recruits (such as Great Lakes Naval Training Center) to take appropriate public health actions to avoid compromise of operational readiness. This analysis provided additional impetus for the submission of a Joint Incentive Fund (JIF) proposal in support of the development of a combined VA/DoD ESSENCE Biosurveillance System. Both agencies annually contribute funds to the DoD-VA Health Care Sharing Incentive Fund which provides seed money through the JIF project process for creative sharing initiatives designed to facilitate coordination, use, or exchange of health care resources with the goal of improving health care provided to beneficiaries of both departments.
In August 2009, the VA/DoD National Biosurveillance JIF project proposal was approved and $4.8 million was appropriated for the initial phases of the project. We believe this project supports exploration in innovative surveillance methods and performance measures which have long been advocated [34–38]. The overarching goal of this effort is to position two well established biosurveillance programs to collaborate on the analysis of health surveillance data, improve information and knowledge sharing between our agencies and enhance biosurveillance capabilities. Coding evaluations and system development work have been performed independently by each agency over the years [32, 39–42]. Because each system has been functioning independently and both versions have made unique upgrades and enhancements over the years, the best features of each system can be incorporated into the joint system, which would ultimately provide more functionality than either system independently. For example, DoD ESSENCE has successfully incorporated patient disposition and some vital sign data into their system which would be a valuable addition to current VA functionality. VA ESSENCE is upgrading graphing and mapping functionality which would be an improvement over what the DoD system currently has available. A bidirectional VA/DoD surveillance system advances the goals and priorities outlined in the Homeland Security Presidential Directive-21 and the National Biosurveillance Strategy for Human Health 2008-2013. These documents have prioritized improving existing biosurveillance infrastructure, fostering collaboration between public health professionals responsible for biosurveillance within different agencies and coordination of health-related information sharing across all levels of government to improve the effectiveness of disease monitoring and response.
The system created through the JIF project will contain data for all VA and DoD facilities. This pilot analysis represented a unique situation where a VA and DoD facility physically merged which provided the opportunity, in advance of the launch of the JIF project, to evaluate data from VA and DoD populations in a single surveillance application (the VA ESSENCE system). However, the joint system will differ in that VA and DoD facilities will remain distinct entities although there will be opportunities with this system to look at surveillance data for cities and regions where both VA and DoD facilities exist in close proximity to each other or where sharing agreements exist. In a way, the combined system will more closely mimic ESSENCE systems that have been established in state and local health departments, which incorporate data from distinct healthcare systems/hospitals from the civilian sector in close geographic proximity. In the future, there may also be opportunities for sharing ESSENCE data with state or local ESSENCE systems for even greater situational awareness in communities containing both Federal and civilian healthcare facilities.
This analysis was limited to a single facility and was a retrospective review of surveillance data. Although a number of VA facilities have DoD patient sharing agreements, North Chicago was chosen for this initial demonstration project due to the large volume of DoD patients seen at the facility and the proximity of two other VA Medical Centers in the same metropolitan area for comparison. In the future, real-time analysis of surveillance data across multiple facilities during actual health events will be important in determining the usefulness of a combined population for situation awareness. The JIF joint surveillance project will be a much larger data integration effort, which will provide additional information to analyze the utility of integrated VA and DoD surveillance data for event detection and situational awareness. Secondly, since the surveillance system relies solely on ICD-9 diagnosis codes, coding errors may impact the quality of the data. Thirdly, there were relatively few large-scale public health events identified for this time period through the Chicago Health Department online archives for inclusion in this analysis. A review of county, state or other resources might be helpful in identifying additional historical events that could be analyzed in the future. Finally, since we did not review patients' clinical records we cannot determine whether cases with GI syndrome codes or heat injury codes were related to the specific events identified during the time periods in question.
Analysis of biosurveillance data in this joint VA/DoD facility suggested the potential for improved ILI trend monitoring and temporal alerting in a combined biosurveillance application. The combined population in this first Federal Health Care facility represents a larger statistical sample of the total population (including pediatric) in the region, which may result in improved recognition of disease events and ILI trends compared to the VA population alone. Joint VA/DoD biosurveillance may be beneficial in other areas where VA and DoD facilities are in close proximity or where facility sharing agreements exist.
The authors would like to acknowledge the following individuals for their support and contributions: Mark Lesko, Mike Tessier and Jeff Young, Captain James A. Lovell Federal Health Care Center; Mark Fiorenza, Anosh Mostaghimi, Payam Etminani, Jessica Lopez, Jude Lopez and Patricia Schirmer, VA Office of Public Health Surveillance and Research; Wayne Loschen and Howard Burkom, Johns Hopkins University Applied Physics Laboratory.
Disclaimer: The views expressed in this paper are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, Department of Defense or the United States government.
- Broome C: Federal role in early detection preparedness systems. MMWR Morbidity and Mortality Weekly Report. 2005, 54 (Suppl): 7-9.Google Scholar
- Public Law 110-53. Implementing Recomendations of the 9/11 Commission Act of 2007. [http://intelligence.senate.gov/laws/pl11053.pdf]
- 6 U.S.C. Sec. 195b. National Biosurveillance Integration Center. [http://www.gpo.gov/fdsys/pkg/USCODE-2010-title6/pdf/USCODE-2010-title6-chap1-subchapIII-sec195b.pdf]
- Homeland Security Presidential Directive/HSPD10. Biodefense for the 21st Century. [http://www.fas.org/irp/offdocs/nspd/hspd-10.html]
- Homeland Security Presidential Directive/HSPD-21. Public Health and Medical Preparedness. 2007Google Scholar
- Public Law 109-417. 109th Congress. Pandemic and All-Hazards Preparedness Act. [http://www.fas.org/programs/bio/resource/documents/pl%20109-417.pdf]
- Department of Health and Human Services. Office of the National Coordinator for Health Information Technology. Federal Health Architecture (FHA). [http://healthit.hhs.gov/portal/server.pt/community/healthit_hhs_gov__federal_health_architecture/1181]
- U.S. Department of Health and Human Services. Centers for Disease Control and Prevention. Office of Public Health Preparedness and Response. Biosurveillance Coordination Unit. National Biosurveillance Strategy for Human Health. [http://www.cdc.gov/osels/ph_surveillance/bc.html]
- National Center for Veterans Analysis and Statistics. Selected Veterans Health Administration Characteristics: FY 2003 to FY 2010. [http://www.va.gov/vetdata/Utilization.asp]
- 2011 Military Health System Stakeholder's Report. The Quadruple Aim: Working Together, Acheiving Success. [http://www.health.mil/Libraries/Documents_Word_PDF_PPT_etc/2011_MHS_Stakeholders_Report.pdf]
- The White House, Office of the Press Secretary. President Obama Announces the Creation of a Joint Virtual Lifetime Electronic Record. [http://www.whitehouse.gov/the_press_office/President-Obama-Announces-the-Creation-of-a-Joint-Virtual-Lifetime-Electronic-Reco/]
- Brown K, Pavlin J, Mansfield J, Elbert E, Foster V, Kelley P: Identification and Investigation of Disease Outbreaks by ESSENCE. Journal of Urban Health. 2003, 80 (Suppl 1): i119-View ArticlePubMed CentralGoogle Scholar
- Lewis M, Pavlin J, Mansfield J, O'Brien S, Boomsma L, Elbert Y, Kelley P: Disease outbreak detection system using syndromic data in the greater Washington DC area. American journal of preventive medicine. 2002, 23 (3): 180-186. 10.1016/S0749-3797(02)00490-7.View ArticlePubMedGoogle Scholar
- Lombardo J, Burkom H, Elbert E, Magruder S, Lewis S, Loschen W, Sari J, Sniegoski C, Wojcik R, Pavlin J: A systems overview of the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE II). Journal of urban health. 2003, 80 (2 Suppl 1): i32-i42.PubMedPubMed CentralGoogle Scholar
- Grannis S, Wade M, Gibson J, Overhage JM: The Indiana Public Health Emergency Surveillance System: ongoing progress, early findings, and future directions. AMIA Annual Symposium proceedings. 2006, 304-308.Google Scholar
- Lombardo J, Burkom H, Pavlin J: ESSENCE II and the framework for evaluating syndromic surveillance systems. MMWR Morbidity and Mortality Weekly Report. 2004, 159-165. 53 SupplGoogle Scholar
- Resch Z, Bullock C, Winslow A, Kelsey A: Missouri's Syndromic Surveillance Experience. Advances in Disease Surveillance. 2007, 187-4Google Scholar
- Zhang G, O'Connell E, Londono L: Amplification of Syndromic Surveillance's Role in Miami-Dade County. Advances in Disease Surveillance. 2008, 199-5Google Scholar
- Burkom H, Elbert Y, Feldman A, Lin J: Role of data aggregation in biosurveillance detection strategies with applications from ESSENCE. MMWR Morbidity and Mortality Weekly Report. 2004, 53 (Suppl): 67-73.Google Scholar
- Burkom H, Murphy S, Coberly J, Hurt-Mullen K: Public health monitoring tools for multiple data streams. MMWR Morbidity and Mortality Weekly Report. 2005, 54 (Suppl): 55-62.Google Scholar
- Green M, Cohen D, Kaufman Z, Peled T, Aharonowitz G, Bromberg M: Use of Simulated Bioterrorist Attacks to Evaluate Syndromic Surveillance Systems Based on Multiple Data Sources. Journal of Urban Health. 2003, 80 (Suppl 1): i136-View ArticlePubMed CentralGoogle Scholar
- Hauenstein L, Gao T, Sze T, Crawford D, Alm A, White D: A cross-functional service-oriented architecture to support real-time information exchange in emergency medical response. IEEE Engineering in medicine and biology society conference proceedings. 2006, 6478-6481. SupplGoogle Scholar
- Burkhom H, Elbert E, Magruder S, Najimi A, Peter W, Thompson M: Developments in the Roles, Features, and Evaluation of Alerting Algorithms for Disease Outbreak Monitoring. Johns Hopkins APL Technical Digest. 2008, 27 (4): 313-331.Google Scholar
- Elbert E, Burkom H: Temporal alerting algorithm methodology for ESSENCE syndromic surveillance data. National Syndromic Surveillance Conference 2002. 2002, New York Academy of Medicine, New York, NYGoogle Scholar
- City Health Dept. Continues Investigation of Food Borne Illness. [http://egov.cityofchicago.org/city/webportal/jsp/content/showDynamicContentItem.jsp?BV_SessionID=@@@@1118252208.1253733487@@@@&BV_EngineID=ccccadeigjmkgjdcefecelldffhdfho.0&print=true&topChannelName=Dept&contentOID=536958100&contenTypeName=1006]
- Centers for Disease Control and Prevention. Chicago Responds to Salmonella Outbreak at a Citywide Food FestivalChicago. [http://www.bt.cdc.gov/publications/feb08phprep/section2/Chicago.asp]
- City Prepared for Warmest Temperatures of Summer. [http://egov.cityofchicago.org/city/webportal/portalContentItemAction.do?BV_SessionID=@@@@1118252208.1253733487@@@@&BV_EngineID=ccccadeigjmkgjdcefecelldffhdfho.0&contentOID=536942180&contenTypeName=COC_EDITORIAL&topChannelName=Dept&blockName=Health%2F2006%2FI+Want+To&context=dept&channelId=0&programId=0&entityName=Health&deptMainCategoryOID=-536887965]
- City Officials Repeat Warnings Relating to Extreme Heat and Humidity. [http://egov.cityofchicago.org/city/webportal/portalContentItemAction.do?BV_SessionID=@@@@1118252208.1253733487@@@@&BV_EngineID=ccccadeigjmkgjdcefecelldffhdfho.0&contentOID=536942766&contenTypeName=COC_EDITORIAL&topChannelName=Dept&blockName=Health%2F2006%2FI+Want+To&context=dept&channelId=0&programId=0&entityName=Health&deptMainCategoryOID=-536887965]
- National Weather Service. 2006 Climate Summary for Chicago. [http://www.crh.noaa.gov/lot/?n=2006_ord_summary]
- 2007-2008 Flu Activity Report. [http://www.idph.state.il.us/flu/fluupdate07-08.htm]
- Cook County Department of Public Health Influenza Surveillance Report. [http://www.cookcountypublichealth.org/files/pdf/FluReport2009Wk10.pdf]
- Marsden-Haug N, Foster V, Gould P, Elbert E, Wang H, Pavlin J: Code-based syndromic surveillance for influenzalike illness by International Classification of Diseases, Ninth Revision. Emerging infectious diseases. 2007, 13 (2): 207-216. 10.3201/eid1302.060557.View ArticlePubMedPubMed CentralGoogle Scholar
- U.S. Census Bureau. American Community Survey. North Chicago city, Illinois Population and Housing Narrative Profile: 2005-2007. [http://factfinder.census.gov/servlet/NPTable?_bm=y&-qr_name=ACS_2007_3YR_G00_NP01&-geo_id=16000US1753559&-gc_url=&-ds_name=&-_lang=en]
- Centers for Disease Control and Prevention. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC Working Group. Morbidity and mortality weekly report Recommendations and reports. 2004, 53 (RR-5): 1-11.Google Scholar
- Hope K, Durrheim D, d'Espaignet E, Dalton C: Syndromic Surveillance: is it a useful tool for local outbreak detection?. Journal of Epidemiology and Community Health. 2006, 60 (5): 374-375. 10.1136/jech.2005.035337.View ArticlePubMedPubMed CentralGoogle Scholar
- Sosin D: Syndromic surveillance: the case for skillful investment. Biosecurity and bioterrorism. 2003, 1 (4): 247-253. 10.1089/153871303771861441.View ArticlePubMedGoogle Scholar
- Sosin D: Draft framework for evaluating syndromic surveillance systems. Journal of urban health. 2003, 80 (2 Suppl 1): i8-13.PubMedPubMed CentralGoogle Scholar
- Stoto M, Schonlau M, Mariano L: Syndromic Surveillance: Is It Worth the Effort?. Chance. 2004, 17 (1): 19-24.View ArticleGoogle Scholar
- Betancourt J, Hakre S, Polyak C, Pavlin J: Evaluation of ICD-9 codes for syndromic surveillance in the electronic surveillance system for the early notification of community-based epidemics. Military Medicine. 2007, 172 (4): 346-352.View ArticlePubMedGoogle Scholar
- Oda G, Hung L, Lombardo J, Wojcik R, Coberly J, Sniegoski C, Holodniy M: Diagnosis Coding Anomalies Resulting in Hemorrahagic Illness Alerts in Veterans Health Administration Outpatient Clinics. Advances in Disease Surveillance. 2007, 260-4Google Scholar
- Pattie D, Atherton M, Cox K: Electronic influenza monitoring: evaluation of body temperature to classify influenza-like illness in a syndromic surveillance system. Quality management in health care. 2009, 18 (2): 91-102. 10.1136/qshc.2009.032177.View ArticlePubMedGoogle Scholar
- Schirmer P, Lucero C, Oda G, Lombardo J, Wojcik R, Holodniy M: Trends During 2008-2009 Influenza Season in Veterans Affairs Medical Facilities Using the VA Essence Biosurveillance System. International Society for Disease Surveillance Annual Conference. 2009, Miami, FloridaGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/11/56/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.