P2.156: Identifying Pediatric Emergency Visits for Aggression: Validation of Billing Diagnosis Codes

Background: Mental health crises are a prominent cause of emergency department (ED) visits among children and adolescents. To address the needs of these patients, research is needed to improve detection and management of agitation and aggression. However, conducting research on this population has been limited by difficulties in identifying ED visits for agitation or aggression using data that is readily available in the electronic health record. In a 2002 study, Peleggi et al. determined that nine billing diagnosis codes had excellent sensitivity (90%) and very good specificity (82%) for identifying pediatric ED visits for agitation/aggression at a single center, however, these results require replication, as billing practices vary by site. Our objective was to externally validate these diagnosis codes by determining their sensitivity and specificity for identifying pediatric ED visits for agitation/aggression, when compared against manual review of ED notes, at a different center.

Methods: We performed a retrospective cross-sectional single-test diagnostic accuracy study of mental health encounters in the pediatric ED of an academic urban children’s hospital, utilizing electronic health record (EHR) data by children 6-18 years old from April 12, 2023 to December 30, 2023. Mental health encounters were defined as those with any billing diagnosis code in the Child and Adolescent Mental Health Disorders Classification System (CAMHD-CS). The index test was the presence of any of 9 aggression diagnosis codes, while the reference test consisted of structured criteria to determine if ED note documentation confirmed that agitation/aggression was a reason for the visit. We described demographic and clinical characteristics of all pediatric behavioral health ED encounters, as well as those of index test-positive, index test-negative, aggression-confirmed and no-aggression-confirmed encounters. Sensitivity, specificity, positive predictive value and negative predictive value were calculated with 95% confidence intervals.

Results: We identified 855 pediatric behavioral health encounters (63% 13-18 years old, 59% female) (Table 1). Of these, the diagnosis code set identified agitation or aggression in 62 encounters (7%), compared to 118 encounters (14%) confirmed to have agitation or aggression based on ED note documentation (Table 2). The 9 diagnosis codes utilized yielded a sensitivity of 40% (95% CI: 31%, 49%) and specificity of 98% (95% CI: 97%, 99%) in the identification of visits for agitation/aggression when compared to manual note review. The positive predictive value was 76% (63%, 85%) and the negative predictive value was 91% (89%, 93%).

Conclusion: Upon external validation, a set of 9 billing diagnosis codes had poor sensitivity and excellent specificity for the identification of pediatric ED visits for agitation/aggression. In light of these test characteristics, novel methods such as natural language processing may be needed to accurately identify pediatric ED visits for aggression/agitation.