P1.234: From Passive Learning to AI-Powered Training: Congenital Heart Disease Education for Neonatal-Perinatal Medicine Fellows Using Generative Artificial Intelligence

Background: Despite evidence supportive active, personalized training, Neonatal-Perinatal Medicine (NPM) education often relies on passive didactics. Large language models (LLMs) could transform medical education by delivery adaptive, case-based training. However, evidence on LLMs’ impact on NPM fellowship training is limited. This study evaluates the feasibility and effectiveness of the NeoCard AI Tutor, an LLM-based educational tool designed to enhance knowledge acquisition, confidence, and engagement in congenital heart disease (CHD) education for NPM trainees.

Methods: We developed NeoCardAI.com, a web-based platform aligned American Board of Pediatrics CHD specifications. Using OpenAI’s ChatGPT-4o, trainees engaged in simulated case-based learning, access summaries, and practice questions. Over six weeks between September and October 2024, participants completed 12 sessions with the AI tutor. Pre- and post-intervention assessments and three-month follow-up, including a 15-item multiple-choice test and surveys on confidence and attitudes toward AI, were analyzed. Semi-structured focused group interview was evaluated using inductive thematic analysis. The project received support from the AAP Marshall Klaus Research grant.

Results: Twelve NPM fellows enrolled in the study. Confidence in clinical management significantly improved in four areas after course completion: cyanotic neonates (p=0.015), left-sided lesions (p=0.003), right-sided lesions (p=0.006), and mixing lesions (p=0.026) (Table 1). Positive shifts in attitudes toward LLMs included reliability as a medical knowledge source (p=0.017) and information retention aid (p=0.007). Knowledge acquisition and retention showed positive trends in median test score compared to baseline, but were not statistically significant (p=0.685, p=0.270, respectively) (Figure 1). Survey showed the AI tutor was helpful for simulated scenarios, and the platform was easy to use, high quality, and adaptable to study routines. Favorability was positive, with the majority recommending the AI tutor. Fellows engaged in a median of 3 conversation threads [IQR 2, 5], with 70% of the interactions lasting ≤ 10 minutes, 12% 10-20 minutes, and 18% >20 minutes. The focus group highlighted strong perceived educational benefit, but persistent challenges regarding time constraints, fellowship curriculum integration, and clinical realism of case simulations limited sustained engagement.

Conclusion: NeoCard AI Tutor is a feasible and favorable tool for CHD education, showing significant gains in confidence and AI receptivity. However, limited utilization may hinder content mastery outcomes. Further studies should explore utilization barriers, education platform user experience, curriculum integration, LLM evaluation, and assess long-term efficacy of optimized deployment.