Abstract
Objective
This analysis was undertaken to determine how the data completeness, consistency, and other attributes of our local syndromic surveillance program compared to the National Syndromic Surveillance Platform.
Introduction
In 2005, the Cook County Department of Public Health (CCDPH) began using the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE) as an emergency department (ED)-based local syndromic surveillance program (LSSP); 23 (100%) of 23 hospitals in suburban Cook County report to the LSSP. Data are transmitted in delimited ASCII text files (i.e., flat files) and contain a unique patient identifier, visit date and time, zip code, age, sex, and chief complaint. Discharge diagnosis and disposition are optional data elements. Prior to 2017, the Illinois Department of Public Health placed facilities participating in the Cook LSSP in a holding queue to transform their flat file submissions into a HL7 compliant message; however as of 2017, eligible hospitals must submit HL7 formatted production data to IDPH to fulfill Meaningful Use. The primary syndromic surveillance system for Illinois is the National Syndromic Surveillance Program (NSSP), which transitioned to an ESSENCE interface in 2016. As of December 2016, 20 (87%) of 23 hospitals reporting to the LSSP also reported to IDPH and the NSSP. As both syndromic surveillance systems aim to collect the same data, and now can be analyzed with the same interface, CCDPH sought to compare the LSSP and NSSP for data completeness, consistency, and other attributes.
Methods
Our comparison of NSSP to the LSSP focused on data completeness for key demographic and medical variables and consistency in total visit counts. Analysis of completeness utilized data from December 2016 for 20 hospitals contributing HL7 production data to IDPH at that time. Total visit counts in both systems were compared for the same 20 hospitals from February 5th-11th 2017, a randomly chosen time period. A target threshold of less than 3% difference in total visit counts was set by the CCDPH system users. Analysis was completed in Microsoft Excel 2010. Other attributes of the surveillance systems were qualitatively assessed by the primary system users at CCDPH.
Results
All variables required by the LSSP had 98-100% completeness in both the LSSP and NSSP (unique patient identifier, age, sex, zip code, visit time and date, and chief complaint). However, the LSSP optional data elements, discharge diagnosis and discharge disposition, were less complete in the LSSP, compared to the NSSP (Diagnosis: 56% versus 83%, Disposition: 66% versus 80%). Among variables required for NSSP reporting but not reported to the LSSP, completeness ranged from 100% (race, ethnicity) to 82% (county). Optional data elements within NSSP ranged in completeness from 73% (initial pulse oximetry) to 0% (initial blood pressure, insurance coverage). Of the 20 hospitals evaluated for visit counts, only one hospital had <3% difference in visit counts in the LSSP and NSSP for all 7 days assessed. Ten hospitals had >3% difference in visit counts on all seven days. Average seven day differences for hospitals ranged from 0% to 54%. Eighteen (90%) of 20 hospitals were reporting larger numbers of visits to NSSP than to the LSSP.
Conclusions
Overall completeness of data was similar between the national and our local ESSENCE systems with most required variables having over 98% completeness. NSSP had higher completeness over the LSSP for discharge diagnosis and disposition. Additional data elements required by NSSP, but unavailable in the LSSP, had similarly high completeness but optional NSSP variables of interest showed greater variability in reporting. Differences in visit counts were higher than expected. An ongoing exploration of these differences has shown they are multifaceted and require hospital-specific interventions. There are strengths and limitations to both the NSSP and LSSP. CCDPH has direct control over data sharing between jurisdictions in the LSSP and there has historically been less system “down time” in the LSSP compared to the NSSP; however, the use of flat files instead of HL7, as well as having fewer incentives for hospital participation (e.g. Meaningful Use) after 2016, results in limited data collection and stagnant growth compared to the NSSP. Jurisdictions using their own LSSPs should consider analyzing their data completeness, consistency, and quality compared to the NSSP.