Abstract
ObjectiveOur objective was to determine if the detection performance ofcurrent surveillance algorithms to detect call clusters is improved bystratifying by exposure category.IntroductionThe Centers for Disease Control and Prevention (CDC) uses theNational Poison Data System (NPDS) to conduct surveillance ofcalls to United States poison centers (PCs) to identify clusters ofreports of hazardous exposures and illnesses. NPDS stores basicinformation from PC calls including call type (information requestonly or call reporting a possible chemical exposure), exposure agent,demographics, clinical, and other variables.CDC looks for anomalies in PC data by using automated algorithmsto analyze call and clinical effect volume, and by identifying callsreporting exposures to pre-specified high priority agents. Algorithmsanalyzing call and clinical effect volume identify anomalies when thenumber of calls exceeds a threshold using the historical limits method(HLM). Clinical toxicologists and epidemiologists at the AmericanAssociation of Poison Control Centers and CDC apply standardizedcriteria to determine if the anomaly is a potential incident of publichealth significance (IPHS) and then notify the respective healthdepartments and PCs as needed. Discussions with surveillance systemusers and analysis of past IPHS determined that call volume-basedsurveillance results in a high proportion of false positive anomalies.A study assessing the positive predictive value (PPV) of thisapproach determined that fewer than four percent of anomalies over afive-year period were IPHS.1A low PPV can cause an unnecessarywaste of staff time and resources. We hypothesized that first stratifyingcall volume by exposure category would reduce the number of falsepositives. With the help of medical toxicologists, we created 20toxicologically-relevant exposure categories to test this hypothesis.MethodsTo compare cluster detection performance between the twoapproaches, we used a historical testbed of hourly exposure callcounts with and without initial stratification by exposure categoryfrom 10 selected PCs from Jan 1, 2006 - Jul 31, 2015. We ran theHLM for both non-stratified and stratified testbeds to estimate themonthly number of anomalies triggered (i.e., alert burden). Our targetsignals to assess detection performance consisted of call samples fromthree large public health events: the 2009 Salmonella food poisoningevent from contaminated peanut butter, the 2012 Hurricane Sandy-associated carbon monoxide poisonings in New Jersey, and the 2014Elk River contaminated water spill in West Virginia (WV). Foreach event, we chose 30 random calls one thousand times to obtain1000 random sets of inject clusters. Each inject cluster was iterativelyadded into the testbed with and without initial stratification byexposure category. We then applied the HLM for each iteration to seeif the algorithm identified the inject cluster. The sensitivity for eachapproach for each PC was calculated as the proportion of iterationswhere the algorithm identified the inject cluster. We reported mediansensitivities from the ten PCs for each of the time windows of 1, 2,4, 8, and 24 hours.ResultsFigure 1 summarizes results for the WV event with markersshowing anomaly burden (x-axis) and sensitivity (y-axis) using thestratified (Δ) and the non-stratified (o) approach by different timewindows (hrs). The results from the other two events are not shownbut established similar patterns. Anomaly burden is shown as theestimated monthly anomaly count for each approach. For example,markers linked by the arrow show that with a 4-hour time window,the stratified approach achieves nearly perfect sensitivity with ~10anomalies as the monthly anomaly burden while sensitivity of thenon-stratified approach is below 20% with ~40 monthly anomalies.The stratified approach gave improved overall sensitivity across alltime windows, and reduced anomaly burden for 1-, 2-, and 4-hourtime windows.ConclusionsWe found a consistent detection advantage (higher sensitivityand lower anomaly burden) for the stratified vs traditional non-stratified approach for 1-, 2-, and 4-hour time windows. Furtherresearch should focus on refining the stratified approach and thespecific surveillance parameters (such as time windows) that increasealgorithm performance.Figure 1: Detection performance comparison: stratified vs non-stratifiedapproach; 2014 Elk River contaminated water spill in West Virginia scenario