RT-NISS is a real-time NI surveillance system that is useful for the wider hospital. Using our system, it takes 2 hours for one ICP to deal with 70 new suspicious NI alerts across 3,500 inpatients. ICPs only need several minutes per day to make decisions about suspicious outbreaks; the system also supports ICPs to have comprehensive knowledge about NIs risks across an entire hospital. ICPs can immediately undertake targeted professional measures to control NIs—just like shooting an arrow at a target. Furthermore, RT-NISS can help ICPs to target surveillance with greater efficiency, without the need for manually collecting large amounts of data. Most importantly, the daily surveillance data helped us greatly to improve our prevention and control measures. Long-term data might be used to further investigate the risk factors of NIs, which would benefit and inform targeted prevention and control programs.
Recently, many hospitals in the US, and approximately one third of California’s hospitals, have begun to use various automated surveillance systems. As a result, these hospitals may achieve more depth and breadth with implementing evidenced-based infection control practices in this part of the U.S .
Most studies [5–9] in this field are concerned with limited targeted NI surveillance, and real-time hospital-wide surveillance has been seldom reported. This is because there are two problems that are difficult to overcome. First, a computer algorithm for all NIs has been difficult to develop with the required sensitivity and specificity; it has to consider many screening conditions, and be able to deal not only with general wards, but also special wards. Second, even though there are computer algorithms with higher sensitivity and specificity, they have only been used for cross-sectional studies or retrospective reviews. So to realize a daily real-time prewarning capability, the system must retain all NI information and be able to undertake complicated time-serial alignments to identify and recognize new suspicious NIs on a daily basis.
Many studies in the past [14, 15] have applied different NI screening computer algorithms (such as positive microbiology reports or instances of antibiotic administration) for retrospectively or prospectively detecting NIs. Unfortunately, the use of antibiotic administration has been shown to be sensitive but not very specific, because there were many cases of excessive antibiotic usage, which can lead to false positive findings, e.g. the prophylactic or empirical use of antibiotics. The criterion of a positive microbiology report has been shown to be more specific but easier to loose NIs, because the physicians do not always order microbiology cultures for NIs. Despite this, the above studies have provided plenty of basic data. Recently, some studies have attempted to combine different NI criteria to develop processes for automatic identification. Brossette SE et al.  reported a successful hospital-wide prospective system with good accuracy in 2006, which combined clinical microbiology (including serological and molecular testing) and patient status data; the sensitivity was 0.86 and specificity was 0.98. We suspect that a possible reason for the low sensitivity was that the computer algorithm did not consider antibiotic usage.
Learning from the merits of previous computer systems, our computer screening algorithm included positive microbiological examination, antibiotic administration, serological and molecular testing (e.g. C-reactive protein, calcitonin), imaging reports, fever, invasive device use, inpatients transfer data, and other variables. Additionally, some of the screening strategies were designed for special wards areas. Compared with the “gold standard” (manual survey of NIs), RT-NISS showed good sensitivity (98.8%) and specificity (93.0%). The sensitivity and specificity was calculated using all of the screening algorithms. Theoretically, it is better to obtain higher sensitivity during NI screening, and good specificity during NI confirmation. Algorithms need to improve specificity, but at the same time be balanced with good sensitivity. It is inevitable that any electronic screening surveillance system will report some false-positive results with relatively low specificity. A strength of our study was that it used RT-NISS to screen for suspicious NIs alerts, followed by the ICPs scrutiny of the alerts, which help to obtain the excellent levels of sensitivity and specificity. Using this method, the specificity increased from 93.0% to 99.3%, with a favorable kappa (0.93). However, when using the ICP’s diagnosis, the sensitivity decreased from 98.8% (84/85) to 94.1% (80/85) because four alerts that were confirmed via manual survey were excluded by the ICPs because the infection information provided by RT-NISS was insufficient for confirmation.
When RT-NISS was implemented in a hospital for the first time, or used in a cross-sectional study or retrospective review, the prewarning NI rate was at 10% for all inpatients. The NIs alerts number should vary with differing conditions in relation to the NI computer screening algorithm, such as the different rates of microbiological testing. In our study, the prewarning NI rate was 15.0% (146/974) by RT-NISS in the survey in March 2013, but it was 8.8% (948/10,765) in July 2011. The main reason for the different percentages was the different microbiological testing rate (32.5% in March 2013 vs 10.4% in July 2011, Figure 5). A positive microbiological testing result was an important criterion in the computer screening algorithm, so a higher rate should bring about more NI alerts.
The other strength of our study was the criterion development and evaluation of daily real-time prewarning surveillance. Our innovation was that RT-NISS recorded all NI information and undertook time-serial alignments, so as to identify and prewarn new suspicious cases each day. Consequently, the daily rate of NI alerts was about 2%, which was less than the total rates in the two studies (15.0% vs 8.8%). This dramatically reduced the daily work burden of the ICPs. Because of this, it was possible for the ICPs to confirm all of the alerts during their routine work, which resulted in real-time surveillance. Furthermore, the ICPs were able to focus on other important issues, such as providing training to healthcare workers (e.g. about adequate hand hygiene), promoting evidence-based interventions about rational antibiotic administration, preventing or controlling clusters or outbreaks, and supervising and helping healthcare workers to improve their knowledge about transmission-based precautions. With the help of RT-NISS, this has led to an improved understanding and enhanced cooperation between healthcare workers and ICPs for preventing NIs.
Our study has several limitations. First, medical records were not included in the computer screening algorithm because many of the words in the records were based on the doctors’ personal preferences, and did not follow formal terminologies. Second, the comparison between RT-NISS and the manual survey was not conducted on all 3,500 inpatients in the study hospital because there were insufficient ICPs. Third, in order to balance good sensitivity with specificity, the RT-NISS could not prewarn and identify all of the NI inpatients. However, the NIs that were lost were rare, and usually associated with mild infections. Finally, the RT-NISS could only monitor NIs when the patient was in hospital, many NIs probably occurred when the inpatients were discharged and were not under surveillance. Only a small number of NIs were prewarned by RT-NISS when the patients were readmitted to the same hospital, and the NI had happened after discharge.