Abstract
Objective
The aims of this project were 1) to assess the validity of a surveillance case definition for identifying heroin overdoses (HOD) in a NEMSIS 3–compliant, state ambulance reporting system; and 2) to develop an approach that can be applied to assess the validity of case definitions for other types of drug overdose events in similar data state data systems.
Introduction
In 2016, the Centers for Disease Control and Prevention funded 12 states, under the Enhanced State Opioid Overdose Surveillance (ESOOS) program, to utilize state Emergency Medical Services (EMS) and emergency department (ED) syndromic surveillance (SyS) data systems to increase timeliness of state data on drug overdoses. A key aspect of the ESOOS program is the development and validation of case definitions for drug overdoses for EMS and ED SyS data systems. Kentucky’s ESOOS team conducted a pilot validation study of a candidate EMS case definition for HOD, using data from the Kentucky State Ambulance Reporting System (KStARS). We examined internal, face validity of the EMS HOD case definition by reviewing pertinent information captured in KStARS data elements; and we examined external agreement with HOD cases identified Kentucky’s statewide hospital billing database.
Methods
From KStARS, we extracted EMS emergent transports by any ambulance service to hospitals in a single, large health care system in Kentucky. We included responses with dispatch dates between January 1, 2017 and March 31, 2017. From Kentucky’s statewide hospital claims data system, we extracted inpatient discharges, ED visits and observational stays at the destination hospitals, with admit dates in the same range. We classified EMS cases as HOD based on specific combinations of the following criteria for EMS data elements: primary or secondary provider impression of heroin poisoning (T40.1X4), heroin-related keywords in the patient care narrative or chief complaint, and patient’s response to naloxone as indicated in the medications list1. We used standard drug overdose case definitions for ICD-10-CM-coded hospital billing data2 to classify hospital records from the destination facilities to the same categories. We produced descriptive analyses of the heroin overdose cases detected in both data sources, EMS and hospital. To assess the degree of overlap in the HOD cases identified by the two data systems, we matched the identified EMS HOD cases against the entire set of UKHC hospital cases. Finally, we assessed the validity of the classification of EMS cases as heroin overdoses by reviewing the EMS patient care narratives and related EMS data elements, as well as the ICD-10-CM hospital diagnostic codes for cases that matched to a hospital record.
Results
We identified 5,517 emergent EMS transports to the destination hospitals in the first quarter of 2017. Of these, 94 (17/1,000) were identified by our case definition as a HOD. We identified 29,631 unduplicated, emergent encounters at the destination hospitals (including inpatient discharges, ED visits, and observational stays; and excluding elective and newborn encounters). Of these, 105 (3.5/1,000) included a diagnostic code for HOD. Linkage of EMS and hospital cases indicated that 141 unique HOD cases were identified in the two files combined. Of these, 58 (41%) were identified as HOD in both systems. 23 HOD cases identified in EMS were matched to a hospital record that had no mention of a HOD; and 13 could not be matched to a hospital record. Additionally, 47 HOD cases identified in the destination hospitals were not matched to an EMS transport to those destination facilities. Overall, 76 out of the 94 (81%) EMS cases identified as heroin overdoses were judged likely to be true heroin overdoses, as indicated by either 1) positive response to naloxone and patient admission of recent heroin use, or 2) hospital diagnosis of heroin overdose, or both. For 2% of identified cases, there was evidence of a false positive finding. The remaining 17% of identified heroin cases were inconclusive: there was information suggestive of opioid overdose, but no clear evidence to suggest, nor to rule out, that the opioid was heroin. Generally, inconclusive cases were identified as heroin overdoses due to positive response to naloxone, combined with mention of the word “heroin” in the narrative that did not indicate an HOD. Examples of the latter include negations (patient denies heroin use) or a bystander who stated that the patient had a history of heroin use.
Conclusions
We assessed the performance of a straightforward case definition for heroin overdose for EMS data. Face validity of 81% of identified heroin overdoses was supported by clerical review of EMS records and/or hospital ICD-10-CM diagnostic codes. Some proportion of the other 19% of cases that were identified as heroin overdoses may have been overdoses involving opioids other than heroin, but we could not quantify that proportion based on the available information.
Future work will consider sensitivity (true heroin overdoses that may fail to be captured by this case definition) and refinements to the basic definition that may yield improved results. Lessons learned from this pilot project will inform subsequent, larger-scale validation studies for EMS drug overdose case definitions.
References
1. Rhode Island Enhanced State Opioid Overdose Surveillance (ESOOS). Case Definition for Emergency Medical Services. Aug 2017.
2. Injury Surveillance Workgroup 7. Consensus Recommendations for National and State Poisoning Surveillance. The Safe States Alliance. Atlanta, GA. April 2012.