The Moment We’ve Been Waiting For: Evidence on AI Tutors and Real Student Outcomes
When Khan Academy’s Khanmigo first rolled out to classrooms, we were all holding our breath a little. Here was this sophisticated AI tutor, powered by GPT-4 technology, ready to provide personalized support to students who needed it most. The promise was tantalizing: an infinitely patient tutor available to every student, regardless of zip code or family income. But promise and reality are two different animals in education. Two years in, with over 2 million students in school settings now using Khanmigo and the tool expanded to all U.S. teachers for free in 2024, we finally have rigorous data on whether this actually works. The answer is more complicated and more interesting than a simple yes or no.
The WestEd longitudinal study released in late 2025 gives us our clearest picture yet. Students who engaged with Khanmigo for at least 30 minutes per week showed a 0.23 standard deviation improvement in math achievement over the course of an academic year. Let me translate that number into something meaningful. That’s not transformational. It’s not the kind of leap that makes you reconsider everything about math instruction. But it’s also not trivial. It’s a solid, measurable gain. Think of it this way: if you had two students with similar starting abilities, and one used Khanmigo regularly while the other didn’t, the student with the AI tutor would likely move from the 50th percentile to somewhere around the 59th percentile. That matters.
Where AI Tutors Actually Shine: Language Learners and the Real Possibility of Closing Gaps
Here’s where the data gets genuinely compelling. The same WestEd research found something remarkable when they disaggregated the results by student population. English Language Learners using Khanmigo showed a 0.31 standard deviation improvement in math achievement. I want to sit with that number for a second, because it’s not just better than the overall group. It’s substantially better. We’re talking about students who often struggle not with the math itself but with the language barriers that prevent them from accessing math instruction, finally having a tool that can explain concepts in multiple ways, slow down as needed, and provide patient repetition without judgment.
This is the kind of finding that keeps me up at night in the best possible way, because it suggests something fundamental about how AI tutors work. They’re not replacing teachers. They can’t. But they might be particularly powerful for students whose learning barriers are wrapped up in language and pacing. When a student can ask a question in their own way and get an explanation that adjusts to their specific misunderstanding, something shifts. The gap doesn’t just narrow because the student is getting more practice. It narrows because the student is finally getting instruction that meets them where they actually are.
The Engagement Problem: Why Tools Sit Unused and What It Means for Your Classroom
But here’s where I need to be honest about the shadows in this picture. A contrasting study from Stanford’s Center for Education Policy Analysis in 2025 found something that should make every educator pause. Without structured teacher facilitation, student engagement with AI tutors dropped by 60% after the first three weeks of use. Sixty percent. That means most students who were using Khanmigo regularly in September weren’t using it by late October. That’s not a failure of the tool. That’s a failure of how we implemented it.
This finding tells us something crucial about learning science that we sometimes forget: technology is never the intervention. The intervention is what teachers do with the technology. An AI tutor sitting in a student’s account, unused, helps no one. But an AI tutor that a teacher has woven into the actual workflow of instruction, that students have been trained to use, that teachers are monitoring and responding to, that works differently entirely. I think of my colleagues who’ve had success with Khanmigo. They’re the ones who use it during class, who show students how to ask it the right questions, who check in on which problems students are struggling with, who build it into the fabric of their teaching rather than treating it as an optional add-on.
If you’re considering Khanmigo for your classroom or your own learning, this matters enormously. You can visit the Khan Academy Khanmigo educator overview to understand how to integrate it intentionally. The research is clear: passive access isn’t enough. Active, structured integration is what generates the actual learning gains we’re seeing in the data.
The Privacy Question We Still Haven’t Fully Answered
There’s another dimension to this two-year moment that we need to discuss, even though it’s less exciting than learning gains. The U.S. Department of Education released AI in Education guidance in 2025 that flagged a serious concern: 78% of educational AI tools reviewed did not fully comply with FERPA’s 2023 updated digital provisions. That’s the Family Educational Rights and Privacy Act, the law that’s supposed to protect student data. And we’re talking about tools that are collecting information about how students learn, what they struggle with, how they think through problems. That’s intimate data.
I’m not suggesting that Khanmigo is one of those non-compliant tools. But the fact that this is even a landscape we’re operating in should make us thoughtful. Before you adopt any AI tutoring tool in your classroom or recommend it to students, check the compliance status. Visit the U.S. Department of Education AI in Education guidance 2025 and understand what questions to ask your district technology leadership. Student privacy isn’t a checkbox. It’s foundational to trust, and trust is foundational to learning.
What This Means for You: Moving Forward with Eyes Open
Two years of data on Khanmigo tells a story worth paying attention to. AI tutors can help. They particularly help students who are battling language barriers alongside content barriers. But they only help when they’re actively integrated into intentional teaching, and they only help when we’re protecting student data. That’s not a disappointing conclusion. It’s a hopeful one, because it means the outcome depends largely on us. On teacher judgment about how to use a tool, when to use it, and whether to use it at all.
If you’re a teacher exploring this for your classroom, think small and structured. Start with one unit, one skill, one group of students. Monitor engagement carefully. Check in on what students are actually learning, not just whether they’re completing assignments. If you’re a student or parent thinking about using Khanmigo for additional support, understand that it works best as a supplement, not a replacement, for actual instruction. Use it when you have a specific question, when you want to see something explained differently, when you need practice at three in the afternoon and a human tutor isn’t available.
The question isn’t whether AI tutors will revolutionize education. The question is how we use them wisely, ethically, and in service of actual learning. That’s a harder question, and honestly a more important one. What’s your experience been with AI tools in learning, either as a student or educator? I’d love to hear what you’re seeing in your own context.
