Abstract
Objective
To compare prevalence estimates obtained by the ADDM cerebral palsy surveillance method to other administrative or diagnostic indications of cerebral palsy.
Introduction
Cerebral Palsy (CP) is the most common cause of motor disability in children. CP registries often rely on administrative data such as CP diagnoses or International Classification of Diseases (ICD) codes indicative of CP. However, little is known about the validity of these indicators. We calculated sensitivity, specificity, positive and negative predictive values of CP ICD-9 codes and CP diagnoses compared to a “gold standard” CP classification based on detailed medical and education record review.
Methods
This sample includes 50,332 8-year-olds living in four US sites (32 counties in Alabama, 5 counties in Georgia, 10 counties in Wisconsin, and 5 counties in Missouri) in 2006, 2008, and 2010. The Autism and Developmental Disabilities Monitoring (ADDM) Network reviewed medical and education records for these children as part of the US Centers for Disease Control and Prevention population-based surveillance of developmental disabilities. All of these children received special education services or were assigned one or more ICD-9 codes associated with a variety of developmental disabilities by community medical providers. Medical and education records were reviewed by trained staff; if the records contained CP diagnoses or motor findings indicative of CP, detailed clinical information was abstracted for additional review by trained clinicians who determined whether the child met the CP case definition based on all information available. Abstracted records were also reviewed for evidence of known motor disorders or genetic conditions that disqualified a child from being a CP case, such as inborn error of metabolism or muscular dystrophy. Trained clinicians reviewed and excluded children with confirmed disqualifying conditions.
We calculated CP prevalence, sensitivity, specificity, and positive and negative predictive values for three different methods used to identify cases, using the ADDM surveillance case identification as the gold standard. These methods include: 1) ICD-9 codes for CP (342–344); 2) a CP diagnosis written in the medical or education records, excluding children with disqualifying conditions, and 3) both ICD-9 codes (342–344) and a CP diagnosis written in the medical or education records, excluding children with disqualifying conditions. In an attempt to avoid requiring record review for method 1, we considered using ICD-9 codes for disqualifying conditions. However, we found that ICD codes for these conditions did not correlate well with disqualifying conditions identified in medical record reviews; therefore disqualifying conditions were not considered for method 1. Methods 2 and 3 did require review of medical records for disqualifying conditions and for a written CP diagnosis, but overall were less extensive than traditional ADDM surveillance methods.
In order to determine the impact of different classification criteria on how and which children are captured by surveillance methods, we compared demographic and other characteristics of all children who met the ADDM surveillance case definition. We compared children who would and would not be classified as CP cases using method 3.
Results
Out of the total 50,332 children, 1294 met the ADDM surveillance case definition, 2201 had CP ICD codes (method 1), 1502 had a written CP diagnosis and no disqualifying conditions (method 2), and 1345 had both CP ICD codes and a written diagnosis and no disqualifying conditions (method 3). Each study year, between 32—48% of abstracted children were excluded due to disqualifying conditions found in medical records. The ADDM network gold standard CP prevalence was 3.3 per 1000 in 2006, 3.1 per 1000 in 2008, and 2.9 per 1000 in 2010.
For method 1, sensitivity was 90.0%, specificity was 97.4%, positive predictive value was 51.6% and negative predictive value was 99.7%. Method 1 prevalence estimates were 5.3 per 1000 in 2006, 4.6 per 1000 in 2008, and 4.6 per 1000 in 2010. For method 2, sensitivity was 98.1%, specificity was 88.4%, PPV was 84.5% and NPV was 98.4% compared to the ADDM Network definition. Method 2 estimated prevalence was 3.9 per 1000 for 2006, 3.6 per 1000 for 2008, and 3.2 per 1000 for 2010. For method 3, sensitivity was 89.6%, specificity was 99.5%, PPV was 84.3% and NPV was 99.7%. Method 3 estimated prevalence was 3.5 per 1000 for 2006, 3.2 per 1000 for 2008, and 2.8 per 1000 for 2010.
Using Pearson’s Chi-Square tests, we compared demographic and other characteristics of ADDM Network CP case children who also met method 3 case definition (n = 1134) and children who met the ADDM Network CP definition but not method 3 case definition (n = 160). Demographic information was not different between these children. ADDM Network CP case children who did not meet method 3 criteria were significantly less likely to require a wheelchair for mobility than children who met method 3 criteria (4.4% versus 27.4%, p < .05).
Conclusions
Relying on ICD-9 codes without excluding disqualifying conditions to identify CP cases (method 1) resulted in high sensitivity (90%), but low positive predictive value as well as an overestimated CP prevalence when compared with the ADDM Network method. Use of a written diagnosis and excluding disqualifying conditions (method 2) resulted in very high sensitivity (98%), with fewer false positives but overestimated CP prevalence compared to the ADDM estimate. In contrast, using both CP ICD codes and a written CP diagnosis and excluding disqualifying conditions (method 3) yielded prevalence estimates similar to ADDM Network CP estimates; this approach also had high sensitivity, specificity, and PPV. Methods 2 and 3 still require manual record review, unlike method 1. For method 2, reviewers would need to review all records for CP and disqualifying conditions. Method 3 only requires review of records with CP ICD codes, comprising 4% of all records currently reviewed. Method 3 would fail to capture children without both a written diagnosis and ICD codes; and this approach may be less sensitive for detecting CP among children with less severe motor impairment than using the gold standard.
Using ICD codes and written CP diagnoses contained in medical and education records combined with a limited medical record review to identify disqualifying conditions could lower operational costs of CP surveillance while preserving accurate prevalence estimates compared with the more labor-intensive processes currently used. Further evaluation is needed to determine if improvements in efficiency are worth potential trade-offs in the data collected by the system. Of particular importance is whether the approach could capture all the necessary indicators that are important to stakeholders. Additional analyses would also need to evaluate whether the surveillance methods affect other findings, such as previously observed disparities, co-occurring conditions, or CP severity.