The New York City (NYC) syndromic surveillance system has been monitoring syndromes from city emergency department (ED) visits since 2001. We conducted an evaluation of statistical aberration detection methods currently in use in our system as well as alternative methods, applying six temporal and four spatio-temporal aberration detection methods to two years of ED visits in NYC spiked with synthetic outbreaks. We found performance varied between the methods in regard to sensitivity, specificity, and timeliness, and implementation of these methods will depend on needs, frequency of signals, and technical skill.