33 Comments
User's avatar
Mike G's avatar

What he said.

Was intro'd to a true believer this week and agreed to a chat.

Me: "So um yeah, I organized this tiny Harvard AI conference last spring - "

"Amazing! Finally someone on our side, there so many naysayers, we have to tell people how AI is really transforming already..."

Me: "Oh wait. No this was I guess a naysayer-ish conference. Convince us you see a path here around the 5% rule, and nobody could."

We had a good laugh.

Expand full comment
Josh Watson's avatar

"They invariably redefine math to mean “becoming an absolute demon at math exercises.”

Part of the problem is that some folks believe that math exercises ARE math and that anything past that is just unnecessary fluff.

I often hear this argument from parents/teachers: "You don't need to understand every part of the car to successfully drive it. You just need to be able to drive the car." In other words, why do we need all that fancy understanding of (place math concept here) anyway? Aside from the edtech/marketing bias issue of targeting only the population for which their product is useful and also defining "useful" as the thing that their product does... Aside from that, we still have many people in our country who genuinely don't see the value of mathematics past computation and math exercises. I think this is often (maybe always?) because they haven't experienced that value for themselves.

My response to that argument tends to be something like, "The point is that humans can't learn to drive a car safely if they are taught isolated concepts with only academic connection to the actual experience of driving a car. It wouldn't be very helpful to have a student drill a bunch of blinker signal situations without context "signal left, now signal right...". Driving safely requires understanding speed limits, gas and brake pedal regulation principles, how to safely distance from other vehicles, traffic patterns, etc, but more importantly, driving safely requires that you can apply principles in real contexts.

Expand full comment
Bryan Frances's avatar

Former prof of philosophy & logic here, who started out in physics.

I like to compare the development of one's mind with the development of one's body. If you want real strength in either one, you have to go through training which is extremely challenging. Looking for intellectual shortcuts is like hiring someone to lift weights for you in the gym.

Expand full comment
Justin Reidy's avatar

I homeschool my two kids right now, and in my experience, rapid learning as described in these X posts *is* achievable – but not because of any online platform.

Yes, you really can outpace a classroom learning environment with 45-60 minutes a day of individually tailored 1:1 instruction. But that has nothing to do with some magic online tool. It's similar to comparing a group fitness class to a personal trainer. More personalized instruction is more efficient and meaningful!

For this kind of thing to work in a school, you'd need to reduce classroom ratios to 4 or 5 students per teacher (or perhaps para-educator). That's the opposite approach of "replace all the teachers with tech".

Yes, new tech can bring all sorts of improvements to teacher efficiency and student experience. As a super-powering enhancement, not replacement.

Expand full comment
TT's avatar

What's left out is that Math Academy started small as an experiment with in-person instruction providing advanced math to 5th or 6th grade students in Pasadena Unified. I have no insight into why and how they moved to a technology platform though I have watched from afar as they have done so. The original approach was never meant to educate all students.

If you believe that students should have an opportunity to learn advanced math at an earlier age or more advanced math than what's set out by grade-level standards, which I know is not always the case, two issues with the original in-person approach are equity and scale. Who gets to participate and who is left out? And, assuming you can mitigate or live with the equity issues, how can you reach more students given the current systems and resources available?

Math Academy seems to have seen technology as the way to address at least the latter. I am not sure they have considered the former. Obviously, their platform and approach leaves much to be desired.

However, I wonder often what are better options that are more equitable and can reach more students who are ready for advanced math?

I ask as a parent in a district where there is no longer tracking in math until high school. Some teachers, like the one we currently have, allow for one 30 minute session of "advanced math" per week for selected students with a volunteer, in this case me, a former teacher. There are other students who would benefit from participating in this group but don't. I have a feeling its related to many of the issues we all know as to why they are not offered this opportunity given there are other kids who are seem quite capable of doing advanced math from what I witness and hear from the students I work with. I wish these kids had the opportunity to do more.

And yet, what I do weekly is not sufficient for most of the kids in the pull-out group. Some of the kids I work with, including my own, are utterly bored in class because of the slow pace and low level of the concepts. This doesn't seem fair to them either.

Expand full comment
datasciencemom's avatar

The selection bias in the examples you noted is key. And knowing how to get the derivative (procedural math) can be orthogonal from conceptual understanding, as you cleverly demonstrated with your kids. Anything worth learning takes work for the vast majority of learners.

Expand full comment
Dan Meyer's avatar

While teaching them this little stupid mathy trick, I literally had the thought that "this is going to make actually understanding the derivative harder for them later."

Expand full comment
SteveB's avatar

It's like we have a gym class where the kids are lifting weights, and some clever fellow says, "Look! I invented a machine that can lift the weights in half the time!" Uh, yeah, the goal wasn't really to get this set of weights from h=0 to h=3.

Expand full comment
Connor Wagner's avatar

I use a weight-lifting/exercise analogy often when kids ask the perennial question, "when are we going to need to know this anyway?"

I'm not a professional athlete, but I know exercise is good for my health. You might not be in a career that uses ________ (insert math concept), but math works out your brain in ways that other subjects might not, and the goal in this class is to become a stronger thinker.

Expand full comment
Josh Watson's avatar

I have found myself using similar analogies too -- I even used to play up a big "Mr Miagi" sort of story that was essentially master Roshi from Dragonball putting heavy turtle shells on his student's back so they could jump 10 feet after all the weight training. It was helpful in some situations, and fun!

That said, nowadays I push back on this kind of idea if that's the ONLY reason we are learning particular math concepts. I don't want to diminish the benefits of this kind of learning (it's definitely not useless), but why would we do math that isn't likely to be directly used for the majority of our students when we can do math that IS likely to be directly useful AND builds those same kinds of "brain muscles"?

I'm a pretty big fan of the "Invigorating High School Math" book. One of the authors, Steve Leinwand, will ask you in his lovely enthusiastic language, "Why in the world are we teaching kids stuff they don't need to know when there's so many things they do need to know and don't know yet?"

Expand full comment
datasciencemom's avatar

I can see that! I also have a kiddo around your kids’ age. I haven’t taught them derivates yet :), but we’re learning multi-digit arithmetic through a concrete-pictorial-abstract method. I am the tutor, not AI. At this age, it’s so important to have human-led learning. I do see some of the merits of AI and apps down the road when they’re older—but I’m not planning to outsource this process right now! Plus I enjoy teaching math, so that helps.

Expand full comment
Jonathan Brownson's avatar

"But on some level everyone knows you get the big results (in education) through selection effects. The only question is whether you are paid to not know this."

So true...and unfortunately "private" education is literally paid to not know this...and public education is under attack as it seeks to confront it...

Expand full comment
Dan Meyer's avatar

🎯 Several commenters noted after the recent NAEP results that students in Catholic schools scored higher and at lower cost than students in public schools. I, someone who is not paid to not know what is going on, might suggest that selection effects are at play here. Others, paid to not know what is going on, have implied that we should turn over the education of our nation's kids to the US Conference of Catholic Bishops, who seem to have figured out the secret to better, more efficient education! C'mon, folks!

Expand full comment
SM McCarthy's avatar

Maybe the difference is the teacher's union?

Expand full comment
Matt Griswold's avatar

I've encountered this framing several times lately, so I want to understand the argument better. Is the point that "public education underperforms because it accepts everyone and so there's a high variance in the student population?"

Expand full comment
Kent Haines's avatar

Yes. Selective private schools (or public magnet schools) have academic requirements for admission, so their achievement data looks great. Is it because of the teachers? Or is it because their student population is already more likely to academically succeed? Clearly, in my view, the latter is the answer.

If they set a height requirement for admission to Harvard, they'd be able to brag about how tall their students were, but would that mean they had a meaningful impact on the height of their students? Not necessarily.

The single most important, mostly ignored, aspect of all education data is the impact of selection effects.

Expand full comment
Matt Griswold's avatar

Obviously, any school that must serve the average of every possible variable would be suboptimal for any given individual (on the criteria that are most consequential to them). That is the usual criticism of the public system, so it's weird that the defense is a restatement of the same.

Our kids have been in many kinds of schools (each different because they are so different), and their success varies by the environment and approach – the same kid. That experience makes me doubt the height analogy. It's more like the difference between cafeterias and restaurants (except cafeterias don't blame the food quality on their patrons, haha – they are designed to serve everyone, so of course they are inferior to a specialist restaurant as it relates to that style).

I appreciate your explanation, even though (especially though) our views may start from different ends.

(FWIW I buy the selection effects argument more as the scope becomes more granular – tools vs. schools. Noting since the original post was more at the tools level.)

Expand full comment
Jonathan Brownson's avatar

As a former board member of Vanderbilt Charter Academy in Holland, MI, I encouraged the board and leadership to engage in progress monitoring, rather than outcome monitoring...this I believe provides more accurate information. Economic and social factors of the student body also impact outcome measurements, ESL, family stability etc.

Expand full comment
Connor Wagner's avatar

I am curious if anyone else has this tool in their state:

https://pvaas.sas.com/e/scatterplot.html?as=a&aj=a&w4=83#/report?populationType=1&yearId=2024&testId=3&subjectId=8&modelId=1&gradeId=13&predefinedAxesId=23-24

It is a scatterplot that graphs PSSA achievement on the y-axis and growth on the x-axis. It is a great way to see what schools are accomplishing with varying degrees of resources/achievement. (It is also particularly helpful if you are familiar with Pennsylvania school districts and know which ones generally serve affluent areas vs. the ones that serve students with higher needs.)

Expand full comment
SM McCarthy's avatar

I think it is very important for people to look at data such as this. I would suggest, specifically, one looks at achievement/growth plots and how the shape changes through the upper grades (6-8th grade). These are KEY formative years for learning and understanding math concepts and the plots reveal, regardless of income, where students "fall off the rails". Another data point-in PA the state went from plotting success in Algebra I in the grade you took the class versus after the pandemic plotting a students success in the test during the 4 years of high school...produces a great deal more "success" per student as each student is bound to pass the Algebra I Keystone after 3-5 attempts. People might better understand very weird plot for years spanning the change on the portal the public sees.

Expand full comment
Sovereign's avatar

Most of schooling is wrote learning. Following a rule set is easier than true understanding. Is this interesting to point out?

Expand full comment
salvatore's avatar

highkey i was only good at math as a kid because i was happy to follow strange and arbitrary instructions (e.g. copy the small number at the top towards the front and minus one from it and keep the x) but what he said — this does not constitute learning, but instruction-following, which is what education has highkey become (but certainly should not be)

thanks for sharing mate!!

Expand full comment
SteveB's avatar

Social media encourages a certain pose, sitting in front of your computer, passing judgement (and it's NEVER good) on life's passing parade: "Hey, how come those California firefighters are takin' so long to put out those fires? Why, if I was doin' it..." Try it yourself, it's fun and literally ANYONE can do it!

Needless to say, these people are never going to put out a fire, and they're probably never going to try to teach a kid - even their own kids - math. But this "one weird trick" nonsense so beloved on the internet can help them develop a dismissive and contemptuous attitude towards the people who actually do these difficult jobs.

Expand full comment
SteveB's avatar

If there was a job where people just sat at a desk and solved equations all day long, then I'd say that purely procedural knowledge would be just fine. But there is no job like that, so presumably we're teaching math and these specific mathy procedures for some other reason? And that is a problem, we may say our goal is "critical thinking skills" but what we're assessing isn't that, it's fluency with procedures.

Or maybe it's that learning how to learn is important, so it's good to have experience learning lots of different things, and math is one of those things? Assessing your math skills is just the one way we have of measuring your learning skills, indirectly.

Expand full comment
Phoenix Koh's avatar

Yes! Fluency with procedures that the top 5% can do super well. What about "hey class, let's ALL go on this ride where we take a break from trying to see who can do the most and fastest, and instead try to learn about what it really is we are working on, together, as a class, yes, all of us- [where everybody has something to learn whether they like it or not]"

Expand full comment
Phoenix Koh's avatar

Steve B I'm going to ?borrow? the final sentence of this post's first paragraph for my "critical thinking skills" 3-page paper for my American College of Education education class, where I'm studying with the nursing profession and the business profession, about what i think of Benjamin Bloom's educational classification levels, where I have to figure out how many different ways can I say, that metacognition & Greek "kritikos" of thought is required for step 1+ all the others, of any lesson, instead of the presumed 3rd+ levels. Thank you. And if I happen to be at a conference with you please excuse me if my hideous inability of socialization prevents me from speaking to you about this!

Expand full comment
Kathy A. Johnson, M. Ed.'s avatar

Good Morning Dan,

Thanks for this explanation of what good teachers do. I thought you were going to talk about home teachers who teach only computation. I am sure that is a problem, too. Talking about math is the real value. Being able to defend our answers, in any arena, shows learning- understanding.

This takes some guided practice and time to demonstrate (and practice) student learnings.

Teach On,

Mrs. J.

Expand full comment
Matt Griswold's avatar

Noting that selection effects can "curate their student body so the students have fewer disabilities, fewer special education and language needs" doesn't invalidate the premise that it can work. It's unlikely anything works for everyone, especially in the realm of varied disabilities and special education needs. One teacher won't come close to stretching across every variety of student either, but you would not conclude they're ineffective because a rigorous application process biases their school population.

Expand full comment
Dan Meyer's avatar

Sure, Math Academy and platforms like it can work for SOME students and for SOME definition of "work." But that isn't their claim. Their claim isn't "this can work." Their claim is this works 4x faster than "a traditional math class." But traditional math classes aren't required to work for SOME students. They're required to provide a free and appropriate education to EVERY student. That's the fallacy here.

Expand full comment
Matt Griswold's avatar

Most private schools offer "a traditional math class" too.

Expand full comment
SM McCarthy's avatar

I always appreciate your insights. I suppose, before we have anyone trying to "redefine math learning" we should, most importantly, define math learning. The challenges associated with reading through your state's curricula, matching your learning objectives and assessments with the conceptual underpinnings of the math at hand and then clearly targeting your "state's goals" is difficult for an experienced teacher with ultimate control in their classroom and probably impossible for most inexperienced teachers. If we are going to "win at math", we will first need some better definitions, hopefully with the guidance of these experienced teachers.

When I homeschooled my boys for one year during Covid, one was in Algebra the other in Geometry. I knew Algebra had a state test and I wanted to make sure my son learned all he should. I used my state standards, but they were unclear. I used a state specific textbook (it was great and aligned well with state standards), I used "StudyIsland", it was good because teachers from each state had aligned it with state standards and I used a well known publisher's online platform (I had used this publishers college platform and LOVED it when I taught). The "well known publisher" had standards NO WHERE NEAR my states standards-the concepts labelled Algebra I for the state of PA included about 40% of the content slated for Algebra II here in PA. The amount of time I had to spend "picking and choosing" content made the online platform valueless-StudyIsland was 25x more helpful. Now, the best aligned product, StudyIsland, is no longer available for families or individuals so if your district doesn't adopt it, a teacher must "cobble together" a curriculum that might align with state standards or receive a stale set of guidelines created long ago.

We really could use a national set of standards that guarantee a certain level of math understanding for each grade. We should be in the habit of helping teachers and families understand the definition of "math learning" for a grade and working together to not only meet but exceed that goal. Until we define math learning, it will consistently be a victim of being undefined which will allow companies to constantly redefine the objectives to suit their desired results.

Expand full comment
Elizabeth K. Baker's avatar

Thank you Dan for so clearly spelling out WHY NOT.

Expand full comment
shaeda's avatar

Complete neutral here as a non-user*, but did you try the site?

The blog seems to end abruptly without any kind of insight, opinion or conclusion etc?

*I have been following MA and engaging with users but not yet signed up.

Expand full comment