Before paying for AI tutoring, decide what improvement would mean
AI tutoring can improve learning in particular settings, but a faster or more polished piece of homework is not sufficient evidence. Families need to understand what assistance the student receives and how the provider checks progress beyond the assisted task. The distinction is supported by research on Tutor CoPilot and a separate experiment comparing unrestricted and guarded AI assistance. Tutor CoPilot; Bastani and colleagues.
The buying decision should begin with the difficulty the student needs help resolving. If the problem is understanding a mathematical method, a service that makes submission easier may be solving the wrong problem very efficiently.
The promising result is more specific than the product label
Tutor CoPilot's trial found a 4% -point increase in topic mastery, with a 9-point gain for students of lower-rated tutors. The intervention assisted tutors during mathematics teaching. It did not test every product sold as an AI tutor, and topic mastery is not interchangeable with a rise in school grades or examination scores. [Original paper].
A provider cannot establish its own effectiveness merely by citing a positive study about another system. Ask what is the same about its approach, and what has actually been tested.
Ask how the research relates to the actual service. Does the support reach the educator or the student? Does it help diagnose a difficulty, provide hints or supply solutions? What is the teacher or tutor expected to do with it?
Those questions should lead to a demonstration, not just a statement that a person remains involved. Human involvement has value when it changes the response to the learner.
Better practice performance can be the wrong reassurance
In Bastani and colleagues' high-school experiment, the unrestricted GPT Base group performed 17% worse than the control group on an unassisted exam despite better practice performance. A guarded version mitigated the loss. This is a relative difference in that study, not a forecast for every student using today's AI tools. [].
