P1.228: Decoding the NICU: Readability study using LLM-powered NICU progress summaries for families

Background: Due to the 21st Century Cures Act, families now gain immediate access to lab results, imaging reports, and daily progress notes. This information can be difficult to understand, particularly in the neonatal intensive care unit (NICU) where daily notes include complex medical jargon. This may result in a communication barrier between parents and healthcare providers, potentially contributing to healthcare disparities.
Our objective in this study is to determine the feasibility of utilizing large language models (LLMs) to assist in summarizing NICU progress notes and provide summaries at an acceptable reading level per the American Medical Association guidelines.

Methods: We created 100 simulated daily progress notes documenting common neonatal complications in infants born between 22-41 weeks of gestation using large language models (LLMs) including Claude (3.7 Sonnet, 3.5 Haiku) and ChatGPT (4.5). Medical accuracy was verified through validation of 20% of the notes by two board-certified neonatologists. These validated notes were then processed through the same LLMs to generate Plain Language Accessible Note Summaries (PLANS). The resulting summaries were evaluated using a standardized readability assessment, the Flesch-Kinkaid Grading level, to ensure accessibility.

Results: All LLM-PLANS achieved the target 8th-grade readability level (average 7.6), representing a notable improvement from the original technical documentation (average 8.43). The LLMs consistently ensured a reading level of 6th to 8th grade, while the progress notes represented a range of 6.3 to 12.5. Neonatologist review confirmed that the summaries maintained clinical accuracy while effectively translating medical terminology into accessible language. Preliminary physician feedback indicated that implementing PLANS could reduce time spent on parent education while improving communication efficiency.

Conclusion: This study demonstrates that LLMs can effectively translate complex NICU documentation into parent-friendly summaries without compromising medical accuracy. By providing equitable access to understandable medical information, LLM-PLANS have the potential to improve parental engagement, reduce physician documentation burden, and deliver higher-value care to all families regardless of health literacy level. Further research is needed to assess the impact of LLM-PLANS implementation on clinical outcomes and family satisfaction in real-world NICU settings.