Study: AI Tutors Help Kids Learn Math More When They Force a Redo
A Tennessee experiment with 6,000+ middle schoolers found AI tutoring only boosted math scores when paired with forcing students to prove they'd mastered a skill.
What happened: Researchers from the University of Toronto and Wharton tested four ways of practicing fractions with more than 6,000 Tennessee middle schoolers: plain computerized instruction, that same software plus an AI tutor, and each of those with a mastery rule requiring three correct answers in a row after any mistake. Students used the software, similar to Khan Academy, for just 50 minutes, then took a short test a week later. Only the group that got both the AI tutor and the mastery rule scored meaningfully higher, about 3 percentage points, than the plain instruction group.
Why it matters: There is growing concern that AI tools let students skip the hard part of learning by handing over answers instead of understanding. This study, still an early working paper, offers one of the first controlled signs that AI tutoring can actually help, but only when it is built to slow students down and make them demonstrate a skill, not just deliver a faster answer. That is a real design lesson as schools rush to adopt AI tutoring products.
How it works, plainly: Without AI, a student who got a problem wrong could see step-by-step solution examples, but often just skimmed past them without understanding the error. The AI tutor instead responded directly to the student's own work, walked through the specific mistake, and then required three correct answers on that skill before letting them move on. Students in this group spent more time per question and were more likely to get the very next problem right, a sign they were actually engaging rather than guessing.
What's still unproven: The gain was small, and it only showed up on the easiest questions closely resembling what students had practiced; it did not carry over to harder, transfer-style problems. The whole test was one 50-minute session with a single follow-up quiz a week later, so it is unclear if the effect holds over a semester or a school year. The paper has not been peer-reviewed yet, and even the lead researcher says it is far from proof AI is a classroom game changer.