Khanmigo Doesn't Love Kids
Products express opinions about the people who use them. Khanmigo does not think much of students.
Products express opinions about the people who use them. This is well understood and easy to illustrate.
This spiked bench does not love homeless people, for example.
The bench does not advertise its disdain in words. Everyone knows better than that. But everyone also understands.
These brackets do not love skateboarders.
By contrast, these curb cuts love wheelchair users, people pushing strollers, etc.
In software, the concept of “hostile user design” is well understood. Software that doesn’t understand or value its users. Software designed to capture more value than it creates or to create value for people who are not the end user.
I would not presume to know what lives in the heart of Sal Khan or anyone else working at Khan Academy, but I know that Khanmigo does not love students. This is easy to illustrate.
Head to this screen which asks students to find the equation of a line.
Type in gibberish. “y = 2345x + 0000” or the like.
Ask Khanmigo to “Help me solve this.”
Here is Khanmigo’s response:
Sure! The goal here is to find the line’s rule. Do you know what the slope is?
Great. You can almost imagine the prompt behind this response.
You are a plucky and positive tutor. You help kids. You break ideas down into step-by-step instructions. You ask questions. You never give the answer. Your responses are a maximum of three sentences. Etc.
But head to the same screen and type in something much more thoughtful than gibberish. Type in an offering—a sincere offering—of value from yourself to Khanmigo. Get the y-intercept right perhaps but make the common error of reversing the change in y and the change in x.
Ask Khanmigo to “Help me solve this.”
Your offering is rejected.
Sure! We need to find the line's equation. Do you know what the slope-intercept form of a line is?
Obviously I do. I had to give my answer in the slope-intercept form of a line. And I mostly did great! Why do you not see what I did here, Khanmigo? Why are you ignoring the resources—the value—I have brought to our interaction?
I refreshed and repeated this same interaction with Khanmigo a dozen times. Every time, Khanmigo treated me as though I knew absolutely nothing of value about the equation of a line.
Sure! We need to find the line's equation. The form is y = mx + b. Do you know what m and b stand for in this form?
Sure! We need to find the line's equation. Do you know what the slope-intercept form of a line is?
Sure thing! We need to find the line's equation. What's the first step to do this?
Sure! We need to find the line's equation. What's the first step to do this?
Sure! Let's start. The goal is to find the line's equation. Do you know the first step?
Sure! We want to find the line's slope and y-intercept. Can you tell me the y-intercept from the graph?
That last one hurt especially. “Can you tell me the y-intercept from the graph?” Yes! I told it to you already. It’s the one part of my answer that’s fully correct!
Do you want to know what it looks like to love students in this kind of interaction? Just ask a teacher.
I went on Twitter and asked teachers there what feedback they’d give that same student. Every one of dozens of responses started with what the kid knew. They started with a recognition of the kid’s value.
Your y-int is correct, however I think you flipped your slope as you rose 2 and went right 3. Remember, slope is RISE over RUN and your fraction is 3 over 2. Go up 3 (from y-int) and then right 2. 😊 @sevaerc3
You're almost there. You got your y-intercept correct. Check your slope. What is the equation for slope again? Rise over run? How can you verify that your slope is correct? @bpcaverhill
Nice job on your y intercept. Check your slope. How do you calculate slope? @EAwaah
Good job using the constant value to identify the y-intercept (4)! Shall we plug some values in to check if the slope of the line is accurate? @radhikazahedi
This would not be difficult to fix, of course. We can fix it right now. It’s clear that however Khan Academy is prompting ChatGPT, they are not injecting the student’s answer into the prompt. The model has no appreciation or even awareness of the student’s thinking here.
So I went to ChatGPT1 with the following prompt:
You are a math tutor who helps students of all levels understand and solve mathematical problems. You seek first to understand what [the problem] is asking and also to understand how the student has tried to answer [the problem] with [the student’s answer]. When you respond, you can provide step-by-step guidance from the start of the problem if the student hasn’t answered and [the student’s answer] = None. But if [the student’s answer] exists, tell the student the parts of their answer that are correct, even if it’s wrong, before offering guidance to help them connect their answer to [the actual answer]. Never tell the student the correct answer in your feedback. Stop periodically to ask them questions about their thinking that might help guide them to new realizations. Your responses shouldn't be longer than three sentences.
Then I added the context:
[the problem] Find the equation of the line. Use exact numbers.
[the actual answer] y = 2/3x + 4.
[the student's answer] y = 3/2x + 4
Give the student feedback on their answer.
ChatGPT’s response is significantly less sophisticated than the average response from teachers on Twitter but significantly better than Khanmigo if only because it knows I did something of value:
You correctly identified the y-intercept as 4. Now, let's focus on the slope. What does the slope represent in the equation of a line? How does it affect the steepness or direction of the line? How can you adjust your slope to match the slope in the correct answer?
Maybe Khanmigo will evolve to value student thinking in its practice sets. Maybe it won’t. I only know that no one expects the park bench spikes to soften for more peaceful rest. No one expects the deterrence brackets to turn into some slick surface for an easier grind. It isn’t in their nature.
I have no idea what the makers of any particular product love (except our own) but I do know that love turns into process and process turns into product faster than you can possibly imagine. And then it’s over. You can read the rest in the commit history.
Math teachers and math students love our products because our products communicate a love for teaching, math, and students. Get all of us together in a classroom and watch something different happen.
What does your product love about teaching? What does it love about math? What does it love about students? What does it love about technology? What does it love about administrators? What does it love about schooling, about schools, the buildings, their relationship with caregivers, and on and on?
Whether you and your team have answered these questions for yourselves, your products are communicating answers about them constantly, everyday to everyone who uses them.
2024 Apr 18. Some time after I published this criticism, Khan Academy made some changes to the way Khanmigo handles student thinking. Check out my review of those product changes.
Special Invitation
My team and I at Amplify have been working on a brief professional development activity to help middle school math teachers give better feedback to students on their math thinking, not unlike what I’m describing here. The activity uses generative AI in a novel way and we’re looking for feedback. If you’re a practicing middle school math teacher and are willing to participate, let us know. (I know this crowd would do it for the love of the game, but we have a $25 gift card for you anyway.)
Odds & Ends
“An ‘education legend’ has created an AI that will change your mind about AI.” Washington Post. Quoted: Sal Khan, Indiana’s Secretary of Education, an Indiana superintendent. Not quoted: any teachers or students. The rapturous enthusiasm for AI in education might be warranted, in which case I will perform penance of some sort. But it’s hard not to notice how much of that enthusiasm is driven top-down from VCs, policymakers, CIOs, CTOs, district staff, tech coaches, all of whom must be under enormous stress to have some kind of AI play to talk about with other people in their same role and above.
AI education tools are helping students learn, grow and gain confidence. CBS News. A report on Khanmigo in Newark, NJ. (Sponsored by Microsoft and AI for All.) Here you’ll find commentary from a teacher who seems genuinely enthusiastic, which is great. Teachers could use more things to be genuinely enthusiastic about. When I watch these kinds of segments, I like to ask myself, “Does this look like a program that is tightly embedded into the fabric of the class’s learning? Or something that gets fired up once every couple of weeks or whenever cameras are rolling.” Always pause and enjoy a close look at what students are doing on their screens with the software.
We Tested an AI Tutor for Kids. It Struggled With Basic Math. Wall Street Journal. More on Khanmigo. Hallucinations account for maybe 5% of my tepid reaction to the technology in K12 contexts, so I wasn’t particularly moved here. But I do love to see people laying down concrete predictions. “Khanmigo is now being piloted by about 65,000 students in 44 school districts. Khan said he expects ‘a million or two million’ students to be using it by next school year at a price to schools of $35 a student.” Let’s revisit this in a year, shall we!
Another caregiver driven to madness by their kid’s math homework. We need to be way more agnostic about how students answer their homework questions. Leave new methods and representations for classwork. See also: This Math Homework Is Driving Caregivers Crazy.
I’ll be giving a few talks in the next few weeks.
March 2. Middle School Principal’s Association of New York. Transforming Your School’s Relationship to Mathematics.
March 12. Amplify’s 2024 Math Symposium. How to Invite Students into More Effective Math Learning. (Free, online, and co-presented with my phenomenal colleague Stephanie Blair.)
April 18. California Math Council’s virtual learning series.
ChatGPT 3.5, not 4. I pay for Khanmigo but I’ve gotta drawn the line somewhere.
Dan, I loved how you asked ChatGPT to "find the kernel of truth" in the student's answer and build on that! I agree, it is what many/most/all human teachers do, and it helps build understanding from what the student already knows.
What you've described with Khanmigo doesn't recognize what the student has done well, so the student doesn't feel seen. The message is "all of this is wrong, we need to start at the first step," instead of noticing what is right and support the student's learning from there.
Khanmigo is basically saying, "It's ok if the wire-mesh monkey mother is not comforting, it's still able to feed the little baby monkey". The problem is that we've run this experiment and we know the outcome. The baby monkey lives but that's about it.