Abstract
Objective:
The aim of this project was to assess the face validity of surveillance case definitions for heroin overdose in emergency medical services (EMS) and emergency department syndromic surveillance (SyS) data systems by comparing case counts to those found in a statewide emergency department (ED) hospital administrative billing data system.
Introduction:
In 2016, the Centers for Disease Control and Prevention funded 12 states, under the Enhanced State Opioid Overdose Surveillance (ESOOS) program, to utilize state Emergency Medical Services (EMS) and emergency department syndromic surveillance (SyS) data systems to increase timeliness of state data on drug overdose events. An important component of the ESOOS program is the development and validation of case definitions for drug overdoses for EMS and ED SyS data systems with a focus on small area anomaly detection. In fiscal year one of the grant Kentucky collaborated with CDC to develop case definitions for heroin and opioid overdoses for both SyS and EMS data. These drug overdose case definitions are compared between these two rapid surveillance systems, and further compared to emergency department (ED) hospital administrative claims billing data, to assess their face validity.
Methods:
The most recent available data were pulled from multiple hospitals in a large healthcare system serving an urban region of Kentucky. Definitions for acute heroin overdose were applied to all three sources. For SyS and ED data, definitions were queried against the same hospitals within this geographic region and aggregated to week-level totals. SyS and ED data are similar with the exception of additional textual information available in SyS (such as chief complaint). Our EMS definition of heroin overdose was loosely based on a draft definition that was produced by the Massachusetts Department of Public Health, and relies more on textual analysis versus ICD10 codes used in SyS and ED data systems. While SyS and ED used the same hospitals as the frame of selection, EMS used incidents that occurred in the approximate catchment area served by those hospitals. Weekly totals from all three data sources were plotted in R studio with LOESS-smoothed trend lines. Unsmoothed times series plots also demonstrate highly correlated trends, but the smoothed trend lines are less cluttered and easier to interpret.
Results:
Visual interpretation of the LOESS-smoothed trend lines shows very similar trajectories among all three sources [Fig 1]. The resultant graph demonstrates that individually, the time courses described by SyS and EMS data track closely with the one observed in ED data. The absolute counts between the three sources showed some differences, as expected. The EMS system captures a slightly different cohort that may include people that do not go to the ED (observation patients, refused transport, etc.) and SyS/ED have slightly different definitions (as ED does not include a free-text chief complaint. These types of limitations are better explored through data linkage that may or may not include medical record review to establish ground truth.
Conclusions:
Public health surveillance of drug overdoses has traditionally relied on ED billing data. In most states, however, there is a lag of at least several months before this data becomes available for analysis. In some jurisdictions the delay may be considerably longer. Rapid surveillance data sources may allow for more timely identification of changes in overdose patterns at the local level. In addition, SyS/EMS can be used together to confirm that a spike seen in one rapid system is confirmed within the other, with relative ease.
Though the comparison is a rather simple or crude visual analysis of three data systems at a common geographic level, there is still appears to be a common pattern among the three systems. While this does not carry the validity of cross-data matched analysis, it does provide some of the utility of looking at these system collective without match; and therefore may be of use to surveillance users that may be limited by de-identified data.