Teaching Kids Not to Get Scammed

Five takes on a lesson I taught a few days ago.

I taught a seventh grade class here in Oakland, CA, a few days ago. We learned about increase and decrease by percentages in this Amplify Desmos Math activity. Here are five short takes on What Happened and What It All Means.

1. Students want answers to three questions before they learn.

These kids and I were new to each other and I got the sense they wanted answers to three questions before much learning could begin:

• Do you like me?

• Do you like what you teach?

• How much do you expect of me?

I think every teacher answers those three questions in their first thirty minutes with a new class even if no one says anything out loud.

2. When will you ever use this? Right now!

It’s one thing to tell students, “you might use this math someday.” It’s another to bring that future into the present. I told the kids, “Look, if someone thinks you don’t know the score, they might decide to give you less than they owe you.”

The move: “Which of these prices seem like you’re definitely getting scammed? Which ones seem like you’re maybe getting scammed?”

3. Humans Giving Chatbots a Wedgie (Part 1,000,000)

A kid was looking at this problem. 

In order to make more money, DesWorst Granola bars are now 10% shorter.  If the original bar was 15 centimeters long, how long is the new granola bar?

He was staring at the problem. No work. What’s your move? If you’re a chatbot, you start chatting and you ask the student to chat back. You’re a hammer and the kid’s a nail.

However, if you’re a human with eyes and hands, with ideas about math and human cognition, you start gesturing and ask the student to gesture back, especially to gesture where they think 10% is and isn’t.

Tutoring 101: figure out what kids know, however they know it, and start from there. I’d love to see a chatbot tutor manage it.

4. Multimedia Decision Fatigue

This was a multimedia classroom and we picked our spots pretty carefully, especially for digital technology:

• Digital technology. Interactive screens for experimentation. Text-based check-in screens I could draw on for instruction. A digital card sort where our platform would tell me the most challenging card for the class.

• Teacher presentation. Communicating & revoicing ideas, creating collective effervescence, setting the stakes.

• Paper. Nearly any kind of operational math work.

“Which of these modalities should I use and when? How do I balance the time cost of switching modalities with a kid’s need for variety?” Challenging questions IMO!

5. What makes discussions work?

Discussion questions are like swimming pools—accessible to fewer kids the deeper they go.

When I’m trying to figure out the question, “How well can these kids swim?” I stay away from deep-end-of-the-pool questions like, “How would you analyze the mathematical validity of this conjecture?” (a fine question for strong swimmers) and stick instead with questions that rely on visual perception like “Which words do you see in both answers?” That question hit. It didn’t access the full depths of the mathematics, but a huge fraction of the class had something they wanted to share.

Odds & Ends

¶ EdSurge’s Daniel Mollenkamp asks, “Is There a Problem With ‘Mathbots’?” and I offer a few answers.

Human teachers will be able to look at a sketch, ask for a couple words about a part of it, and then respond, using the context of the classroom, Meyer says. For a chatbot to have comparable knowledge would require a student to type pages and pages to try to communicate in words what they communicate so effectively and simply with multimedia, Meyer adds. Plus, even then, the human teacher could have incorporated facial cues from the student as clues to their understanding, which would have eluded a chatbot.

¶ Education Week on How Students Are Dodging Cellphone Restrictions. There is a species of parent that simply baffles me.

Parents fight the phone policy saying their student has the right to use their phone for classwork and communication because they bought the phone and gave it to their child.

¶ Kristin DiCerbo, Khan Academy’s Chief Academic Officer, offered some fairly candid thoughts on what they’re seeing in student interactions with Khanmigo, their AI chatbot:

Transcripts of student chats reveal some terrific tutoring interactions. But there are also many cases where students give one- and two-word responses or just type “idk,” which is short for “I don’t know”. They are not interacting with the AI in a meaningful way yet. There are two potential explanations for this: 1.) students are not good at formulating questions or articulating what they don’t understand or 2.) students are taking the easy way out and need more motivation to engage.

I would love for any of these large edtech chatbot platforms to report some very blunt descriptive statistics like a) the number of turns students take in the median chatbot interaction or b) the word count of the median turn or just c) monthly active students. Nobody reports these metrics—just the total people who created an account—which only makes me more curious about them.

¶ Sal Khan gave a speech in Santa Barbara, CA, and this is an interesting reflection:

You know, it’s very tempting when you see a new technology, especially if you have a background in tech. I have a background in engineering; especially if you’re a technologist, it’s very tempting to think that’s really cool technology. I want to use it somehow — which is always the wrong way to do it. You should never have a solution looking for a problem. You should say, ‘Well, what are the problems we’re trying to solve in the world?’ And then we should try to solve them in the simplest possible way. The problem that I’ve thought about a lot, and I realize Khan Academy is there to hopefully help folks solve, is how do we personalize education for and how do we give more people access to world class learning, and how do we support those supporting the students more? How do we support teachers, and how do we support parents more? And if the technology is pencil and paper, so be it. But maybe AI can be part of that solution.

It probably goes without saying that “maybe AI can be part of that solution” is light-years from “I think we're at the cusp of using AI for probably the biggest positive transformation that education has ever seen.” Have the chatbots lost Khan?

Education Week: Teacher AI Use Hasn’t Budged in a Year

Here’s why and why not.

In December 2023, Education Week found low uptake of AI tools among teachers. Just 2% of teachers said “I use them a lot” and 37% of teachers said “I have never used them and don’t plan to start.” If you wanted reasons for optimism, you might note that 33% of teachers in that survey said they were using AI tools a little, some, or a lot.

When I wrote about those findings, I heard from AI boosters that a) well, we just need better AI models, b) well, teachers just need more training, c) well, the technology is still new to most people.

Well, a year later, we have seen the release of new AI models from Meta, Anthropic, OpenAI, and Google, models which improve significantly on their predecessors. And according to Education Week’s newest poll, “43% of teachers said they have received at least one training session on AI,” an increase of nearly 50% from their previous survey.

Time, technology, and training. I was told these were the three major headwinds on AI adoption among teachers and we have made significant progress on each. So let me just take a big sip of coffee and check the latest Education Week poll and review what must be significant increases in AI usage:

Let’s put it plainly: the usage numbers haven’t budged in a year.

In spite of the release of more sophisticated AI models, in spite of increased training in AI, in spite of a summer to slow down, regroup, and take an undistracted look at this new technology, Education Week found that teachers are using AI at lower rates in October 2024 than in December 2023—from 33% to 32%.

What doesn’t explain this lack of usage?

Over on LinkedIn, the people who offer edtech training for schools and districts have attributed the low usage here to the need for … more edtech training for schools and districts, which by the way they offer.

But the share of teachers who said they weren’t using AI for lack of AI training decreased over the last year. 43% of America’s teachers have received training on AI in just two years, a massive and rapid rollout by the standard of schools and districts. America’s edtech consultants have received what can only be described as a Great Society-style welfare program. And yet AI usage has decreased.

What does explain the lack of usage?

There was only one reason for not using AI that increased from December 2023 to October 2024.

6% more teachers said they don’t think the technology is applicable to their subject matter or grade. 

Many more teachers have learned about AI over the last year and, among the ones who don’t use it, more of them are saying “because it ain’t for me.” I would devour an interview with every one of those teachers right now, and if you build tools for teachers, I hope you would as well.

The opportunity.

Edtech needs many more people to obsess over three questions.

• What do teachers and students need now?

• What can technology do to meet those needs now, not in some hypothetical future where multi-modal, empathetic, humanoid AI robots facilitate discussions among groups of students without any hallucinations?

• What can’t technology do to meet those needs?

We should use AI as a provocation to understand more deeply the complex, social, and cognitive work of teaching and learning. Right now, too many people are too dismissive of those questions and too settled on their answers. They are wasting this opportunity.

Featured Comment

SteveB, modeling democracy in his classroom :

I teach a Quantitative Reasoning class using Carnegie's Quantway College curriculum. Up on the screen I've got a google doc of the questions in the lesson, large font, one question per page. Students work in groups, I drop in and ask a student to tell me about their work, often I'll say "I like how you put that, could you please add that to the doc?" And then they go up to the front of the class and type in what they said. Or I might say, "I like how you included the units in your answer, be sure you do that when you put this into the doc" and they go up and do that.

And it's a small class, so I do my best to remember the name of each student who contributed, and to use their names when we get to whole-class discussion, "Now let's compare Jesse's graph with the equation that Gabrielle gave us." But I want to say each student's name to the class at least once, and be aware when I fall short of that.

Odds & Ends

¶ MagicSchool, I am begging you not to be sleazy. In a recent blog post, MagicSchool describes Aurora Public Schools and their “28% improvement in students meeting grade level expectations for literacy using MagicSchool,” strongly implying those results accrued to all 38,000 Aurora Public School students. If you watch the attached video, however, it becomes a little clearer that the results were only in one class taught by one teacher. Of course, there was zero attempt to control for the effect of MagicSchool in that one teacher’s class, but I’m unbothered by an n = 1 case study so long as it’s explicitly named as such, which it really isn’t here.

¶ "An AI tutor for every kid": Promise and reality. A good summary of the state of AI edtech from Axios.

Torney told Axios that there was a lot of enthusiasm in the mid-2010s, with people talking about "blended learning, hybrid learning, voice and choice in the curriculum, personalized pathways, and student-centered instruction."

Those promises haven't panned out, Torney says. "Some people have argued that the last technology that was adopted at scale in the American education system was the chalkboard."

¶ Here is a thread from Michael Pershan saying things about AI that I largely agree with.

¶ EdSurge interviews Spokane, WA, school staff about their smartphone ban. Key quote:

“We absolutely have lost some power of the opportunity that those devices provide, whether that's, ‘I can really quickly look something up,’ or ‘I can quickly participate in a class poll’ or ‘I can tune my music instrument,’” [Superintendent Adam Swinyard] told EdSurge. “But I think where we landed in our community, for our schools and for our kids, is what we gain in their level of engagement and ability to focus far outweighs what we're losing in a device being a powerful pedagogical tool inside of the classroom. But I think it's important to acknowledge.”

I’m very interested in coverage of local and statewide bans on cell phone usage. It feels like classroom technology needs to bring much more value to students and teachers now than it did pre-COVID.

¶

Noah Mackert

in “What People Are For,” a thoughtful post about teaching and AI that’s old to the world but new to me:

I would much rather join the enthusiasm. It is a marvel that ChatGPT and other programs can even begin to answer the questions we ask. Every accurate or even nearly-accurate answer should thrill us. The visuals and films are astounding, too, though derivative, as a rule. And the new, more interactive programs are only going to become more sophisticated. 

But when it comes to the kind of tutoring that can change a student’s trajectory, the technology is not there. The theory of those boosting AI-based tutors is that what students need, above all, is easier access to knowledge. That is certainly a useful thing to have. But in my experience, when tutoring is really needed, it is because there are obstacles in the way of learning that current AI-driven programs still cannot perceive, much less alleviate.

¶ AI is Evolving, but Teacher Prep is Lagging: A First Look at Teacher Preparation Program Responses to AI. I resent the edtech industry for making me defend teacher preparation programs, which, yes, often focus on theoretical knowledge at the expense of the practical and, yes, often lag in their adoption of new technologies. But when I hear that teacher preparation programs are not spending their extremely limited syllabi teaching their preservice teachers about technology that is, as of now, unproven and not particularly desired by inservice teachers, I say: good.

You Are Practicing Your Politics in Your Classroom Every Day

You can't opt out.

I have not gone a waking hour this week without thinking about this class. Just look at it. Early on, nearly every student answered a question correctly. Later, nearly everyone answered this question incorrectly:

There is one approach to teaching which says, “Well, wrong is wrong and right is right. Students did something of value early in the lesson and little of value later.”

Another approach to teaching says, “Student ideas are valuable whether they’re right or wrong. Their value is unrelated to their correctness.” Those teachers might look more closely at the student answers on that later question.

Those teachers might notice that these answers are all incorrect but …

• they are more alike than different, 

• they contain a great deal that is correct, 

• they are valuable either way simply because people earnestly thought and expressed them. 

The teacher can decide to share or not share that value with the class. There isn’t a third choice. And depending on the teacher’s choice here, their students are going to learn something very different about math and something very different about their own value.

This is politics.

People frequently disagree whether or not politics belongs in teaching. I believe this disagreement results from mismatching definitions of “politics,” one of which is probably better named “partisanship.”

Politics isn’t partisanship. It isn’t which party you belong to, who you voted for yesterday, or how you feel about who won. Instead, politics is the way a society defines which people have value and which people get value. Seen through that lens, teachers are unavoidably practicing politics in their classrooms every day.

In classrooms right now, teachers are defining who is valuable, whose ideas are valuable, whose ideas are worth sharing with the rest of the class, who gets which opportunities to learn, who gets to feel like they contributed to their classroom.

Given the infinite ideas students bring to their class and the finite time available to teachers, the teacher has to decide, “Will I ask questions that invite or suppress all of those ideas?” That’s a political question. If the teacher allows those ideas to reveal themselves, the teacher then has to decide, “What will I say about all of them? Which ones do I think are valuable? Which ones will I share?” Those are political questions too. 

Unless you possess infinite time and energy, you cannot opt out of these questions. You are practicing politics—defining and sharing value with your students—every day.

I’m not of the mind that your lesson plan will save democracy. Reject that burden. Schools are far, far downstream from where we have concentrated power as a society. They are much more a passenger on the ship of state than the rudder.

But you are practicing your politics in your classroom every day. If you seek a more generous politics in your country, a more generous recognition and distribution of value, you can practice those politics in your classroom. You must. If you don’t, your odds of contributing to a different politics nationally are no better than chance, like kids who imagine they’ll play a meaningful role in a championship game four years from now without doing the daily practice when they had the opportunity.

Teachers: “These AI Resources Are Not Classroom-Ready.”

The "80-20 rule" is more accurately a "20-80 rule."

In my recent article about AI’s “delivery problem,” I claimed that AI-generated resources like lesson plans, assessments, and presentations, leave significant work to teachers, not just to customize them to their local context, but to make them even minimally viable for classroom use.

For example, new AI teacher “copilot” tools can easily produce paragraphs of text and call it a lesson plan, but students often benefit from images, videos, and interactives when they learn new ideas, all of which are absent from those lesson plans. Many ideas, like all of geometry, are inseparable from their visual presentation.

The resources I have generated in my testing generally conceive of teaching as the transmission of information and rarely detour from what seems like a Wikipedia-style bulleted summary of a topic.

An AI-generated lesson plan.

The 80-20 Rule

Many teacher copilot developers agree that the resources they generate are not completely classroom-ready. They frequently cite an “80-20 rule.” MagicSchool is a well-regarded teacher copilot tool, for example, and they write in their FAQ:

‍The 80-20 Approach: Use AI for initial work, but make sure to add your final touch, review for bias and accuracy, and contextualize appropriately for the last 20%.

As often as it is cited, I can find no empirical basis for the 80-20 rule. How was it determined? For example, did these copilot tools survey their users to ask them “How close is this resource to being ready for your use?”

I asked MagicSchool how they derived the 80-20 rule. They responded that “the 80/20 approach at MagicSchool advocates for a collaborative partnership between AI and educators, leveraging the efficiency of AI for initial content generation while recognizing the crucial role of human educators in refining, contextualizing, and ensuring the quality and appropriateness of educational materials.”

Their response describes their intent for human intervention but not how they measured that intervention at only 20% of the total versus 10% or 90%.

In the absence of any empirical basis for the 80-20 rule, I decided to survey educators on Twitter and LinkedIn and other places where educators spend time online. I shared with them four AI generated resources on the same idea—deciding whether two ratios are equivalent—and asked them, “How close is this to being ready for classroom use by a teacher?” with options for 0, 20, …, 100%.

I received 104 responses, 42 of which identified themselves as math teachers. The math teachers indicated these resources are quite far from 80% classroom-ready. 

With an overall readiness rating of 40%, it’d be more accurate to name the rule “20-80” than “80-20.”

Teachers described the inadequacy of the lesson plan:

This is a “telling lesson” not a “teaching lesson.” Suddenly during guided practice it tells me to encourage students to use cross products to determine if they are equivalent.  Where did that come from? Did I demonstrate that earlier in the lesson? Will 6th graders know this method? Have they learned about setting up proportions or solving for x yet? Why is this the preferred method for these students and this content?  

And the presentations:

I thought the presentation was really dry and would lose most of the class to boredom. It lacked the interactive examples a teacher could include using any type of manipulatives.

The presentation made me laugh.  It was a bulleted list.  No visuals, no animations, no basic clip art to even pretend that it wouldn’t be a nightmare to sit through this presentation. If reading a bulleted list was an effective way to teach students math, students wouldn’t struggle with this content, and my job wouldn’t be nearly as complex or rewarding.  

Discussion

No, of course this survey is not representative of every math teacher. It is possible that my respondents are more skeptical of AI than the median teacher, but I’m afraid the median teacher is still quite skeptical of AI’s utility. I suspect the respondents also have more experience and capacity than the median teacher, given they are participating in social networking and professional development experiences outside of their contract hours.

Perhaps novice teachers would find these resources more useful and closer to classroom-ready than veteran teachers. This would be in keeping with other research finding that generative AI benefitted lower-performing workers more than higher-performing ones.

Perhaps 40% is still better than 0%. If I’m walking along from point A to point B on a road, I would not turn down a ride 40% of the way just because it is not 100% or even 80%. But it seems equally possible that these teacher copilot tools are taking teachers 40% of the way towards point C, some alternate destination that is undesired by teachers or students. Two of the respondents spoke to this concern, saying these resources would create more work for them, not less.

The slides were utterly useless. It’s more work for me to extract something from those than to just create the slides on my own that would actually be engaging, have visual representations, and support understanding.

None of those AI components understand what it takes to "connect to students" and "engage students" in the lesson. I would still need to "craft" the lesson and the worksheets would need to be fixed for mistakes and clarity; therefore, not saving me any time in my planning. AI is great at creating documentation to check that box for lesson planning but thoughtful lesson planning is not about the “documentation.”

The needs in education are clear. Teachers clearly need help. Students clearly need effective resources. The public clearly needs a strong understanding of the sophistication of effective teaching.

I have conducted a survey here, one with a small sample and one that is likely unrepresentative of the population of teachers in important ways. But it is a unique survey, and contributes a uniquely useful insight. While most surveys about generative AI still ask teachers, “have you even used generative AI ever?” this survey has tried to understand how well generative AI is meeting those clear needs of teachers, students, and the public. The results indicate that generative AI is still leaving much of that need unmet.

Upcoming Presentations

Come hang out and learn together!

• Council of School Supervisors and Administrators 2024 Leadership Conference.

  • Solving the Math Problem

  • November 2, 2024. 8:55 AM.

  • New York City, NY.

• California Math Council - South Conference

  • The Teacher’s Superpower: How to Build and Put It to Work

  • November 8, 2024. 3:30 PM.

  • Palm Springs, CA.

Creating "Just Try It" Classrooms

As opposed to "Just Do It" classrooms.

Here is a twenty-second video compilation of several different teachers each conducting an uncommon energy inside their math classrooms.

Try it. What’s gonna happen when I say try it? Try it out. I’m gonna press try it. Okay, yeah, try it. Let’s see. Try it. Okay? So try it out. Let’s just try it. Try it. Try it one more time. You should try it? Try it. Did it work when you pressed try it? Try it. Let’s try it. Let’s try it!

What does it take to create a “Just Try It” math class?

In many math classes, students experience a “Just Do It” energy. “I showed you what to do. Just do it. You didn’t do it right. Do it again, but right this time.”

What does it take to create a classroom where students feel, rightly, that they can interact with mathematics safely, that they can experiment with mathematics? This is an open question for the curriculum and instruction nerds among my subscribers, but I’ll add several early thoughts under the categories of beliefs, pedagogy, and curriculum.

Beliefs

Do teachers believe that every question in math class has one correct route to one correct answer? If so, then they should probably expect students to just do it. By contrast, if they believe there are as many ways to be smart as there are humans in the world, then they have one of the necessary beliefs for a “Just Try It” classroom.

What other beliefs are essential to a “Just Try It” classroom?

Pedagogy

In a “Just Try It” classroom, the teacher draws ideas out of students that are not always maximally efficient or even initially correct. The teacher will follow the implications of those ideas, regardless, understanding that they’ll generate learning for the class. When a student suggests solving 5x < 15 by dividing by 3, the teacher says, “Let’s try it,” producing 5x/3 < 5 and some valuable learning, rather than using their hard and soft power to get the student back onto the most efficient path. 

What other pedagogies are essential to a “Just Try It” classroom?

Curriculum

A teacher who values diverse mathematical thinking might fight against a curriculum that does not. With digital curricula, especially, you’ll hear of students who are nervous to try out an idea for fear of losing their streak of correctness and having to start over. 

At lots of moments in our curriculum, we show students a giant “Try It” button and send them information, rather than evaluation, on their thinking, whatever that thinking was, whatever they tried.

What other features of a curriculum are essential to a “Just Try It” classroom?

Featured Comment

Mike McGibbon:

I love your first/last mile analogy, but as a tutor, I don't think this Tutor CoPilot solves those problems for me. I couldn't quite tell from your example, but it seems like this is a chat that is only visible to the tutor, and the tutor is supposed to keep one eye on the chatbot's hints while also interacting with the student. I am not eager to bring this distraction into my tutoring sessions. And while these hints might help novice tutors in the short run, I wonder if they would hamper the tutors' long term development, training them to seek answers from a bot rather than build their own teaching skills.

Odds & Ends

¶ I’m writing this an hour after a visit to a local charter school, the sort of school where kids build tiny homes and apply to community internships in their junior year. I was tagging along with a delegation of education officials from a country I won’t name out of respect for their hospitality. They were there to assess AI’s impact on education and every group they interviewed—students, teachers, and administrators—said roughly the same thing: “It’s neat, but ….”

• Students: “It’s neat, but the employers at our internship are more interested in how we work together.”

• Teachers: “It’s neat, but I’m trying to get my students to see each other as resources.”

• Administrator: “It’s neat, but we draw a socioeconomically and racially diverse student body and we’re focusing more on developing empathy and discussion across our differences.”

This was an administrator who attended ASU+GSV, the annual edtech confab in San Diego, CA. These were teachers who studied teaching at elite Bay Area institutions. These were students whose grown-ups entered them into a lottery for enrollment into this school. This is a forward-thinking school that’s an hour’s drive from Google’s headquarters. 

None of them could muster more than a limp “🥴 it’s neat I guess” when asked about artificial intelligence, all of which I offer to you as data in case it helps you re-weight your mental model of teaching, learning, and technology.