Abstract
Objective
Standardize selection of indicator data streams and corresponding alerting algorithms for syndromic, reportable disease, and confirmed diagnostic categories derived from veterinary laboratory test order data for bovines.
Introduction
The Johns Hopkins University Applied Physics Laboratory is collaborating with epidemiologists of the US Dept. of Agriculture''s Animal and Plant Health Inspection Service (APHIS) Center for Epidemiology and Animal Health (CEAH) to increase animal health surveillance capacity. CEAH monitors selected syndromic animal health indicators for stakeholder reporting. This project’s goal was to extend this capacity to bovine veterinary laboratory test accession data.
Methods
Indicators for weekly monitoring were derived from bovine test records from the Colorado State University Veterinary Diagnostic Laboratory System from 27 Jun 2010 - 29 May 2016. Selected indicator types were syndromic test orders, disease-specific orders, and disease-specific positive results. Indicators were adopted if APHIS epidemiologists considered them worth monitoring and if they were represented by at least 100 lab accessions.
Ten syndromes were chosen for routine monitoring based on body systems, bovine-specific concerns (e.g. mastitis), and concepts to capture novel threats. Reportable diseases were chosen from the list published by the Colorado Dept. of Agriculture [1]. Based on APHIS concerns and test order frequencies, 4 diseases were chosen for weekly monitoring: Bluetongue, Brucellosis, Epizootic Hemorrhagic Disease, and Paratuberculosis. To monitor positives, we considered both the number and the ratio of herds with at least one positive result for each disease. For included tests (excluding results quantified with antibody levels), we counted an accession as “positive” if the result field contained strings “positive”, “suspect”, or “detect” without negation terms. For weekly counts, we added the number of herds with any positives after deduplication. Diseases adopted for monitoring of positive results were Bovine Viral Diarrhea, Trichomoniasis, and Paratuberculosis.
From experience and literature, we compared variants of 4 algorithm types, including: the C2 method of the CDC Early Aberration Reporting System, a CuSUM control chart with a sliding baseline, the temporal scan statistic Gscan applied to hospital infection counts, and the CDC Historical Limits method.
We adapted a semisynthetic simulation approach for algorithm comparison in which authentic disease count data are used as baseline, and simulated signals are added to the background as detection targets. In discussions about specific diseases and veterinary testing practice, CEAH required sensitivity to one-week data spikes as well as effects of health threats with multi-week incubation periods and more gradual test ordering. For such gradual signals, we chose the lognormal signal model of Sartwell applied to incidence data for many diseases. Incubation periods vary widely by disease, and for this project, we chose lognormal parameters such that 90% of reported cases would occur within 6 weeks. We conducted separate algorithm detection trials for spike and gradual signals.
Calculations of sensitivity, alert rate, and timeliness were derived with sets of 1000 repeated trials for each combination of algorithm and syndrome or disease. We applied minimum performance requirements of 95% sensitivity, ≤1 alert per 8 weeks, and mean detection delays of <2 weeks. The rule adopted for recommending an alerting method was to seek the method with the lowest alert rate that satisfied the sensitivity, alert rate, and delay criteria.
Results
The Table shows the syndromes with chosen algorithms and thresholds for detection of the gradual signals. The scan statistic Gscan and the historical limits method HistLim achieved consistently higher sensitivities with acceptable alert rates than the other methods applied . The presentation will extend the results to reportable disease and clinical positive indicators and to the spike signals for all indicators.
Conclusions
Among results for both signal types, the results yielded a few preferred methods covering all chosen indicator streams. Monitored indicators with median weekly counts = 0 remain a challenge requiring more background data and veterinarian judgment. From analysis of orders from the few available laboratories, manual review will be required to achieve accurate syndromic categorization for each lab. Monitoring of test positives will require combined analysis of positive herd counts and percentages (of all tested herds) due to routine variation in laboratory submissions.
References
[1] Colorado Department of Agriculture, Livestock Health: Reportable Diseases in Colorado, https://www.colorado.gov/pacific/aganimals/livestock-health, last accessed Aug. 23, 2017.