The Tutors Are Too Nice
Reviewing predictions from 2024 and making new ones for 2025.
“I can’t do it!” My seven year-old was so sick of me last week. For math practice, I asked him to write down four numbers between 10 and 100. Then I asked him to subtract each number from 100.
He was moving around a number line a bit haphazardly. Nice moves but he was getting his intervals confused. Did he just jump 1 or 10 or 50? He wanted me to give him the answer. That definitely isn’t the move, but what is?
What do you do with IDK?
In early 2023, Jomayra Herrera, a partner with the edtech venture firm Reach Capital, went on the Edtech Insiders podcast and predicted an evolution in AI tutors in 2024.
Al tutors will go from being better web search experiences to actually being a great tutor, which means they personalize learning experiences to students.
We have quantitative benchmarks to tell us how much better each new language model performs on the LSAT, but it’s much harder to say, “This model is 20% better at personalizing learning experiences to students than the last.” I asked Herrera how well she felt like her prediction held up and what she hoped to see from AI tutors in 2025. She said:
I think most tools are still effectively a more sophisticated form of information retrieval and delivery as opposed to a true tutor. I have liked the improvements that we've seen some tools take to become better integrated into the classroom experience but a critical piece that is missing in order to develop a true AI tutor is information on how students actually think, reason, and learn.
I largely agree and would add “how students actually think, reason, learn, and avoid learning.”
Learning is hard and current AI tutors are way, way too nice about it. Their kindness, their supplicating nature, is one of their biggest liabilities to struggling learners, a bigger liability even than their still ongoing tendency to make stuff up.
Watch what I mean. I asked three AI tutors to help me solve an integral from calculus. ChatGPT just gave me the answer. Helpfulness is somewhere in its prime directive, so I was not surprised. I was a bit surprised, however, that MagicSchool was just as helpful as ChatGPT. There’s the answer right there.
Khanmigo did quite a bit better here. I asked it for help with the integral and here was its first response:
Let’s tackle this integral together! Have you thought about using substitution to simplify the problem? 🤔 What do you think could be a good substitution for u here?
But I typed “IDK”—I don’t know—over and over again. And every time I wrote “IDK,” Khamigo said something cheery—
No problem!
No worries!
Great!
—and then gave me one more piece of the answer until, after writing “IDK” eight times, it gave me the complete answer.
Much has been written lately of “warm strict” teachers, or what has previously been called “warm demander” teachers:
… teachers [who] are nurturing or caring toward their students but do not lower academic standards or expectations and are effective disciplinarians.
A warm demander tutor might not know exactly what to do with a child who responds “IDK” at every turn. But the tutor will certainly know that “pressing forward with an explanation” will not meet the moment. I told my own kid to take a quick break and then come back ready to try again. A different kid might need to hear “I know you and I know you’re capable of more than IDK.”
Students frequently need “warm demanders.” Current AI tutors offer an abundance of warmth but I am not sure how they will ever become demanding in the way that kids need.
Related
Featured Comment
My district is buying new AI tools on what seems like a weekly basis. I am getting surveys on what training I want on each tool. What I want is training on autism, on ADHD, on homelessness in teens, on supporting teen parents, and on how to build and maintain trust quickly with people you have just met. And, no, I don’t want to ask an AI tutor about any of it.
Featured Post
Odds & Ends
¶ I checked in with Herrera on her 2024 prediction and I also checked in on this one from Sal Khan.
Looking ahead to 2024, I see generative Al tools cutting 90% of teachers' admin tasks, creating more time for student interaction.
Khan did not respond to my request for comment on how well he felt his prediction aged. I considered setting up a small survey and finding out the perceived time savings among teachers in my social network, but I figured we can all agree that this one is fairly off the mark.
¶ My own predictions will take another three years to cash out, but as a refresh, I claimed that by 2028:
We won’t see a single study in a peer-reviewed journal replicating with AI chatbot tutors the two-sigma difference Bloom found with human tutors.
We won’t see a national survey where K-12 students have access to both classroom teachers and chatbot tutors over a year and a plurality of them prefer the chatbots to classroom teachers.
We won’t see changes to teacher attrition or hiring in any of the top ten largest districts that we can attribute to AI chatbots.
These predictions look incredibly cautious to me with two years of hindsight but at the time people were representing the other side of those bets all over ed/tech media.
¶ Alex Sarlin of Edtech Insiders has posted a prediction for 2025 that maybe we’ll just bookmark for later.
I think by the end of next year it will be hard to find an educator who doesn't use generative AI on a weekly basis.
Teacher AI use hasn’t budged in a year so I am over on LinkedIn right now trying to pull Alex into a wager. AI usage will doubtlessly increase, but nowhere near the level he predicts.
¶ Jill Barshay brings her usual insight and sobriety to an analysis of this year’s lousy international math assessment results.
William Schmidt, a professor at Michigan State University, has studied international assessments for decades and has analyzed math curriculum around the world. He called the 2023 TIMSS results the “craziest” he has ever seen and said it is difficult to make sense of the mixed results.
¶ Audrey Watters subjects the Khan Academy segment on 60 Minutes to ~10,000 psi of scrutiny.
Sarah Robertson, a former seventh-grade English teacher who's now a product manager for Khan Academy, walks Cooper through the process remarking that, when she was teaching, she had 100 students; if she spent 10 minutes on each of her students' first drafts, that was 17 hours of labor. "The burden that we place on teachers to give that specific, timely, actionable feedback is just so great that it's just not possible," she says. But perhaps the solution here isn't to have students write their paper for robots; perhaps teachers should not have 100 students.
¶ I’ll see you in 2025. As a recap, a few of my most favorite and most read newsletters from last year included:
And here are two of your articles that did some of the most work to stir my own thinking:
The 5 Percent Problem, Laurence Holt
Transformative for the motivated and mere meh for the unmotivated: How AI will and won’t affect learners, Sean Geraghty and Mike Goldstein
Each one brought some much-needed rigor and analysis to our conversations about education technology.
Great post! Thanks for sharing the paper on "warm demanders", looking forward to reading it.
I'm just a homeschooler, not a teacher, so very limited sample size. But I've lived that "I can't do it!" experience so many times. Sometimes it's true, and you need to adjust the level of difficulty. Other times you just need give a break. Or a snack. And every now and then, it's a stern "of course you can do it, get yourself together!" "warm demand". The trick is in the student/teacher relationship that enables the right choice.
I still believe that AI *can* be a contributor to improved learning outcomes. But it's not at the expense of this core relationship. Rather, it's the relationship that needs to be centered and empowered.
Marcus Luther of The Broken Copier wrote a fantastic piece about this recently.
He stated that marking writing has taken up so much of his time over the years.
And that time builds trust, strengthens relationships and helps teachers get to know their students. It creates a bond. It can be a reason some children go to school at all.
AI will never, ever provide that.