Khan Academy’s Journey to Experimentation
How Khan Academy scaled experimentation and now uses A/B testing to improve Khanmigo, its generative AI tutor.
Experimentation rarely starts as a mature system. Most organizations begin with a few A/B tests and gradually build a real culture of evidence.
In this session, Dr. Kelli Hill shares how Khan Academy evolved experimentation into a cross-functional discipline spanning research, analytics, and engineering.
Learn how her team uses AI-driven evaluation and A/B testing to improve Khanmigo, their AI tutor, going beyond typical product metrics to focus on learning quality, student outcomes, and responsible AI behavior.
You’ll learn how their team approaches:
In this session, Dr. Kelli Hill shares how Khan Academy evolved experimentation into a cross-functional discipline spanning research, analytics, and engineering.
Learn how her team uses AI-driven evaluation and A/B testing to improve Khanmigo, their AI tutor, going beyond typical product metrics to focus on learning quality, student outcomes, and responsible AI behavior.
You’ll learn how their team approaches:
- Designing experiments for AI systems, prompts, and model behavior
- Evaluating learning quality, not just engagement metrics
- Combining automated AI evaluation with production A/B tests
- Scaling experimentation across product, engineering, and data teams
Khan Academy
Kelli Hill, PhD
Senior Director of Data Insights
Khan Academy
Senior Director of Data Insights
Khan Academy
GrowthBook
Luke Sonnet
Head of Experimentation
GrowthBook
Head of Experimentation
GrowthBook
Register Today
Webinar: Thursday, April 16 at 9am PT / 12pm ET
We do not sell your information for any purpose. Please see our Privacy Notice for details.
Trusted by 2,700+ companies worldwide
"The fact that we could retain ownership of our data was very, very important. Almost
no solutions out there allow you to do that. Most of them you're passing the user data to a third-party service and that's something we really wanted to avoid."
no solutions out there allow you to do that. Most of them you're passing the user data to a third-party service and that's something we really wanted to avoid."
JOHN RESIG, Chief Software Architect, Khan Academy
