P1.231: Beyond Graduation: Long-term Outcomes of an Active, Project-based Neonatal Fellowship Quality Improvement Curriculum

Background: In 2013, the Accreditation Council for Graduate Medical Education (ACGME) mandated active trainee involvement in quality improvement (QI) as part of its implementation of quality and safety milestones introduced in 2009. Despite this requirement, the long-term impact of fellowship QI curricula on knowledge, skill retention, and real-world application remains insufficiently documented. This study evaluates the long-term effectiveness of a Neonatal Fellowship Quality Curriculum using the Kirkpatrick model of educational outcomes.

Methods: Two cohorts of fellowship graduates were compared: pre-curriculum (2013–2015) and post-curriculum (2016–2023). All eligible participants were surveyed in March 2025 using REDCap. The curriculum, launched in 2015, included structured didactics and active components such as a fellow-led QI project and a QI journal club (added in 2021). Outcomes were assessed across the four Kirkpatrick levels, with Level 4 further evaluated using the Institute for Healthcare Improvement (IHI) Project Assessment Scale. Descriptive statistics were used, with Mann-Whitney U and Fisher’s exact tests applied for comparisons.

Results: Of 28 eligible fellow graduates, 23 responded (82% response rate). All reported exposure to QI modules or didactics; the post-curriculum cohort reported increased participation in active QI fellowship experiences. Table 1 describes evaluated outcomes.

Kirkpatrick Level 1: Ability to improve patient care increased significantly from 60% in the pre-curriculum cohort to 91% in the post-curriculum cohort, reflecting a 31-percentage point increase in perceived skill development.

Kirkpatrick Level 2: Self-reported QI knowledge and ability to independently lead, perform, or teach QI significantly increased post-curriculum, with higher confidence and competency reported across all domains (Figure 1).

Kirkpatrick Level 3: Post-curriculum graduates were more likely to remain engaged in QI after fellowship, including leading projects, pursuing additional QI training, and holding quality-related leadership roles.

Kirkpatrick Level 4: The proportion contributing to a post-fellowship QI project achieving an IHI Scale score ≥5 (indicating modest improvement or higher) was 50% pre-curriculum compared to 60% post-curriculum, with only post-curriculum respondents reporting sustainable improvement. In the 2016–2020 cohort, 88% reported involvement in high-impact projects—the highest rate across all groups.

Qualitative

Assessment: Post-curriculum respondents were more likely to use QI-specific terminology when describing their involvement and identified themselves as "leaders and teachers" in QI.

Conclusion: Post-curriculum respondents who participated in active QI experiences during fellowship reported a higher level of QI knowledge, skills, and confidence in leading quality improvement. Those respondents were likelier to remain engaged in QI work after graduation and more frequently described contributions to higher-level, sustained improvement efforts. These observed differences in perception, behavior, and outcomes occurred despite similar exposure to QI didactics across both cohorts, suggesting that hands-on, experiential learning may be key to lasting impact. Although limited by a small sample, findings indicate that integrating active QI participation into curricula promotes sustained educational outcomes and real-world clinical improvement.