"Alpha School students must be “functioning within a typical range of ability and independence,” according to their application."
It feels like that's the whole game right there. Can we select out just typical performers? Which doesn't feel like an ideal way for a school to operate if it's going to change education more broadly
Does anyone want to join me in putting down a marker on when it turns out the 'AI' is a bunch of starving adjuncts mechanical turking, some cooked numbers and/or attrition rates get leaked, or the whole thing goes up in a private equity bustout?
Of course it's a selection effect. Get one of these folks drunk or on a safe-seeming podcast and they'll probably be proud of it. You can just smell the hustle- if not quite a scam, then the aggressively marketed messianic pitch that characterizes our current slot machine economy. The narrowly successful STEM kids turned tech adults with the naked hostility towards teachers as useful people and school as a developmental playground, the MOOC-like conviction that all anyone ever needs to do is lay a runway and cheerlead- we've done all this before and the result is a few people made some money before their toys turned into wastelands for coding certifications and/or targets of class action lawsuits. If the magic sauce is supposed to be that there are LLMs now, well, that's how I know the parties involved haven't actually tried to salvage a student that's gotten real friendly with one.
I have no doubt that there are circumstances where average students can make above average gains. At 75K a year they could be paying for multiple hours of one-on-one instruction per student every day of the school year, and then I have no doubt, per Bloom's two-sigma 'problem', that you'd get quite capable kids out the other side. Cooler me suspicious that with all that money floating around that the teachers aren't, in fact, occasionally teaching when little Jimmy melts down that his Math Academy module doesn't make any sense but he still needs to be at the top of the leaderboard. If you can afford 75K a year for school you can probably scare up substantial external support on your end anyways to get your grades where they need to be to justify your 75K expenditure. But if you open a school that says 'we charge 75K a year to sort out bad apples and for every student to get a private instructor', you don't get fawning press coverage about how the robots (which are supposed to make everything cheap, I thought) are doing a Future Thing. I've been in the halo of a lot of private schools and for every one that uses their money to do extraordinary education there are two that count on the money to make it *look* that way.
It's just the usual oogling at the wacky shit rich people get up to. They are so High Tech! They paid A Lot of Money! CEO Said a Thing! Gimme 75K to hire a decent grad student to follow a kid around and I bet they go twice as fast up a testing ladder too.
> Get one of these folks drunk or on a safe-seeming podcast and they'll probably be proud of it.
This IMO is admissions director Cara Wilkinson admitting she lets kids opt out of Alpha School without parents knowing.
> If you can afford 75K a year for school you can probably scare up substantial external support on your end anyways to get your grades where they need to be to justify your 75K expenditure.
I have reliable sourcing internal to Alpha that they hired elite tutors in the run-up to Advanced Placement testing to ensure the kinds of AP scores parents expected. Math Academy & TimeBack weren't going to cut it.
I mean, of course on both counts. It couldn't be otherwise. The effect sizes stank, the lived experience of anyone who had every touched an LMS stank, the money stank- if I was a parent paying enough each year for a sports car and my kids was underperforming because they'd been farting around on a laptop all year and they *didn't* get a decent tutor at crunch time I'd burn down the building.
It makes me furious, the whole milieu. The entrenched hostility in tech land to teachers is so strange and deep- is it just because it's a care profession? Because it has soft skills they don't like? Why is that the job that they're so keen to put on the chopping block? So they do the whole 10x disruptor song and dance about how they're gonna change the world, school in two hours- but then they define 'school' to mean 'math ' (in which case two hours isn't very surprising), toss the humanities in the dumpster in favor of 'life skills', and then charge winnowed rich kids enough to incidentally educate them the old fashioned way, which evidently they do, and then make headlines. Nothing startling has occurred here.
This is the best critique of alphas model I’ve read. Thanks for digging in on this.
I hope Mackenzie and team respond and give some insight into what happened at Unbound. I’m also very curious on what the yoy performance changes look like at unbound. Are there results on the map test changes from the fall to the spring at the school? That’s would be the best data to see how the cohort improved over the year controlling for selection effects (ie if the unbound kids started lower than the Arizona general population)
Some additional thoughts on selection:
- alpha definitely selects but it is pretty light. Much less selective than other private schools.
- …but WAY more than public schools. I think the biggest difference is not the “receives feedback” etc you mention. But it is the “not disrupting others” and “can focus for 10 minutes at a time”. That’s a low bar but it still cuts off the left hand side of the distribution
- I think the best way to evaluate alpha right now is how they do vs other private schools that charge $10k+. Then the question is whether they can achieve better results with non-selective methods. Recall that it took almost a decade for alpha to “crack the code” on their current model. I expect it will take some iteration to get it to work for a completely non-selective group of students with 30:1 teacher:student ratios
Edward - I appreciated your review over at Scott Alexander's newsletter. It changed my priors—student selection seemed a bit less important, parent selection more. Having parents who will get on the phone, who are dialed into grades, who will ensure attendance—all add massive value to a student's learning.
I'd be happy to see Alpha leadership critique other private schools and try their best to get Dalton to care even a little about MAP and then share their results. (Never gonna happen.) Maybe Alpha is offering the best elite education around. That isn't my project.
My project is public education, which bears the same resemblance to elite education as the lives of the poor bear to the lives of the elite. Price & Liemandt can critique public schools as broken, but to me it sounds like the freshman golf team saying "we're better golfers" to the varsity football team. The golf team can even spend a billion dollars trying to crack the code on their putt, and it will not prepare them to play varsity football. We're playing different games.
I found this line pretty curious from one of their board meetings.
Academic Performance: Approximately 30% of students are currently on track for "2x learning" growth, up from 24–25% at the prior reporting. The school expects further gains as end-of-year promotion discussions create additional motivation for families and students.
Dan, thanks for writing this balanced piece. Most articles about Alpha School hug the poles of the argument: Alpha School is great or it’s a hoax. The truth is somewhere in between. Your best point is in the concluding paragraph. It’s okay to begin testing your idea with great students, but the true measure of success is how you fare with the largest group of students, or even better, with the most disadvantaged students. Let’s see if Alpha is willing to shift their schedule a bit to meet the needs of a broader spectrum of students.
"Unbound is our online, virtual charter, which nobody at Alpha thinks is the ideal model. (We were hoping to open brick-and-mortor schools, but were stopped by cowards in love with the status quo.) These result show just how important everything else we do at Alpha is -- incentivizing students, having AlphaSquadLeaders hover around, putting kids in a physical space, keeping them motivated with the afternoon incentives. Frankly, we aren't shocked by any of this, though we are hoping to grow our online charter segment so that we can continue to speed up development of the Timeback platform and get that into the hands of consumers and public school kids. As Timeback improves, we expect even our virtual schools to impress you with their scores."
EEEEEWWWWW.....Alpha School gives me the creeps. I don't care what anyone says, the 2 hour learning thing in the am is ridiculous!!! When they talk "learning" what do they mean? Do they mean the children run into school excited about what they are to discover that day both from each other and their teachers? Do They mean that children love to talk and discuss about what they are learning so that they can both grow their own learning and their classmates'? Do they impart the understanding of the magic of Leonardo da Vinci? or the beauty found in the science and math of a sea shell? I don't think so.
So they only take self motivated kindergarteners? And they tell children they can make decisions without including their parents (leaving the process wide open for abuse). I have taught for 50 years and I know that throughout that time I "did good" for my students. I have attended PhD challenges; I have watched some students grow in the entertainment world; I have seen some start their own businesses; I have seen some embark in the political arena, and I know that many are intent on changing this world for the better and feel competent and happy. None of this took only 2 hrs a day in setting them on their road and none had me deciding the course they would take by the program they would be indoctrinated with in the afternoons. I do not want a world where children spend their time in rote learning in front of a computer. I do not want a world where children are fractured from their families. As if a 5 year old can make a reasonable decision about their education! This whole discussion makes me almost physically ill.
Throughout my teaching experience I have had classes that tried to outsmart me and get away with something. One wanted to take a non educational field trip. When I asked them where, they said Sausalito. This was 1969 and they were all farm kids. Another time when I had my kids for 3 years, I asked what they wanted to do to start the 3rd year. They said study mythological creatures. This turned into the most amazing study of scientific observation imaginable and at the same time they loved every moment. Yes I had computers in my classroom. they were used as tools to add to our learning or for some to make writing easier (pencils were not their friends neither was spelling). But learning happened and grew at morning meeting when we could discuss whatever was on our minds at the moment or begin to solve a problem that was tugging at us.
So count me out of Alpha School. It will fail like all the others that forgot there were human children and their families involved.
The kid who ends up at an Alpha school is the sort that would typically attend a "high performing" suburban school. Many of these schools pride themselves on their workloads which tend to be disproportionate to their academic results, kind of a "the workload is the point" environment. A few years ago I looked at a private school in the greater Boston area for my kids where they claimed students could expect 4 hours of homework every day. They never mentioned their academic outcomes. Basically there are many parents who believe that a school's reputation for a heavy workload is what will get their kid into an elite university.
As someone who has spent enough years with this nonsense and the detrimental effect it has had on my kids, I understand where the Alpha guys are coming from while not necessarily agreeing with their approach. Given this context, I believe they can get these kids to a higher level with less effort just by dropping the hours of busywork. The question is whether this applies generally, are schools with large numbers of struggling students overloading their students with busywork to please Harvard? Probably not.
I feel like so much of this comes down to
"Alpha School students must be “functioning within a typical range of ability and independence,” according to their application."
It feels like that's the whole game right there. Can we select out just typical performers? Which doesn't feel like an ideal way for a school to operate if it's going to change education more broadly
Does anyone want to join me in putting down a marker on when it turns out the 'AI' is a bunch of starving adjuncts mechanical turking, some cooked numbers and/or attrition rates get leaked, or the whole thing goes up in a private equity bustout?
Of course it's a selection effect. Get one of these folks drunk or on a safe-seeming podcast and they'll probably be proud of it. You can just smell the hustle- if not quite a scam, then the aggressively marketed messianic pitch that characterizes our current slot machine economy. The narrowly successful STEM kids turned tech adults with the naked hostility towards teachers as useful people and school as a developmental playground, the MOOC-like conviction that all anyone ever needs to do is lay a runway and cheerlead- we've done all this before and the result is a few people made some money before their toys turned into wastelands for coding certifications and/or targets of class action lawsuits. If the magic sauce is supposed to be that there are LLMs now, well, that's how I know the parties involved haven't actually tried to salvage a student that's gotten real friendly with one.
I have no doubt that there are circumstances where average students can make above average gains. At 75K a year they could be paying for multiple hours of one-on-one instruction per student every day of the school year, and then I have no doubt, per Bloom's two-sigma 'problem', that you'd get quite capable kids out the other side. Cooler me suspicious that with all that money floating around that the teachers aren't, in fact, occasionally teaching when little Jimmy melts down that his Math Academy module doesn't make any sense but he still needs to be at the top of the leaderboard. If you can afford 75K a year for school you can probably scare up substantial external support on your end anyways to get your grades where they need to be to justify your 75K expenditure. But if you open a school that says 'we charge 75K a year to sort out bad apples and for every student to get a private instructor', you don't get fawning press coverage about how the robots (which are supposed to make everything cheap, I thought) are doing a Future Thing. I've been in the halo of a lot of private schools and for every one that uses their money to do extraordinary education there are two that count on the money to make it *look* that way.
It's just the usual oogling at the wacky shit rich people get up to. They are so High Tech! They paid A Lot of Money! CEO Said a Thing! Gimme 75K to hire a decent grad student to follow a kid around and I bet they go twice as fast up a testing ladder too.
> Get one of these folks drunk or on a safe-seeming podcast and they'll probably be proud of it.
This IMO is admissions director Cara Wilkinson admitting she lets kids opt out of Alpha School without parents knowing.
> If you can afford 75K a year for school you can probably scare up substantial external support on your end anyways to get your grades where they need to be to justify your 75K expenditure.
I have reliable sourcing internal to Alpha that they hired elite tutors in the run-up to Advanced Placement testing to ensure the kinds of AP scores parents expected. Math Academy & TimeBack weren't going to cut it.
I mean, of course on both counts. It couldn't be otherwise. The effect sizes stank, the lived experience of anyone who had every touched an LMS stank, the money stank- if I was a parent paying enough each year for a sports car and my kids was underperforming because they'd been farting around on a laptop all year and they *didn't* get a decent tutor at crunch time I'd burn down the building.
It makes me furious, the whole milieu. The entrenched hostility in tech land to teachers is so strange and deep- is it just because it's a care profession? Because it has soft skills they don't like? Why is that the job that they're so keen to put on the chopping block? So they do the whole 10x disruptor song and dance about how they're gonna change the world, school in two hours- but then they define 'school' to mean 'math ' (in which case two hours isn't very surprising), toss the humanities in the dumpster in favor of 'life skills', and then charge winnowed rich kids enough to incidentally educate them the old fashioned way, which evidently they do, and then make headlines. Nothing startling has occurred here.
This is the best critique of alphas model I’ve read. Thanks for digging in on this.
I hope Mackenzie and team respond and give some insight into what happened at Unbound. I’m also very curious on what the yoy performance changes look like at unbound. Are there results on the map test changes from the fall to the spring at the school? That’s would be the best data to see how the cohort improved over the year controlling for selection effects (ie if the unbound kids started lower than the Arizona general population)
Some additional thoughts on selection:
- alpha definitely selects but it is pretty light. Much less selective than other private schools.
- …but WAY more than public schools. I think the biggest difference is not the “receives feedback” etc you mention. But it is the “not disrupting others” and “can focus for 10 minutes at a time”. That’s a low bar but it still cuts off the left hand side of the distribution
- I think the best way to evaluate alpha right now is how they do vs other private schools that charge $10k+. Then the question is whether they can achieve better results with non-selective methods. Recall that it took almost a decade for alpha to “crack the code” on their current model. I expect it will take some iteration to get it to work for a completely non-selective group of students with 30:1 teacher:student ratios
Edward - I appreciated your review over at Scott Alexander's newsletter. It changed my priors—student selection seemed a bit less important, parent selection more. Having parents who will get on the phone, who are dialed into grades, who will ensure attendance—all add massive value to a student's learning.
I'd be happy to see Alpha leadership critique other private schools and try their best to get Dalton to care even a little about MAP and then share their results. (Never gonna happen.) Maybe Alpha is offering the best elite education around. That isn't my project.
My project is public education, which bears the same resemblance to elite education as the lives of the poor bear to the lives of the elite. Price & Liemandt can critique public schools as broken, but to me it sounds like the freshman golf team saying "we're better golfers" to the varsity football team. The golf team can even spend a billion dollars trying to crack the code on their putt, and it will not prepare them to play varsity football. We're playing different games.
The same slide deck Dan linked to has a line from May saying: "MAP - 1.8x average growth".
Another slide shows that ~50% of students were "Off Track" in 2 hr learning for most of the year, but that number went down to 19% at the end of May.
@Dan - I saw that. The Off Track category falls from 61% to 19% in one week. You're feeling good about this metric?
Definitely made me curious about what the incentive was that drove that change!
I found this line pretty curious from one of their board meetings.
Academic Performance: Approximately 30% of students are currently on track for "2x learning" growth, up from 24–25% at the prior reporting. The school expects further gains as end-of-year promotion discussions create additional motivation for families and students.
https://drive.google.com/file/d/1XHiBcdxkVzZLl9ljz7rMsE8AnJsRJ1tm/view
Dan, thanks for writing this balanced piece. Most articles about Alpha School hug the poles of the argument: Alpha School is great or it’s a hoax. The truth is somewhere in between. Your best point is in the concluding paragraph. It’s okay to begin testing your idea with great students, but the true measure of success is how you fare with the largest group of students, or even better, with the most disadvantaged students. Let’s see if Alpha is willing to shift their schedule a bit to meet the needs of a broader spectrum of students.
Unbelievably good work here Dan!
100%. Great sleuthing.
Let me see how good I am at corporate ping-pong:
"Unbound is our online, virtual charter, which nobody at Alpha thinks is the ideal model. (We were hoping to open brick-and-mortor schools, but were stopped by cowards in love with the status quo.) These result show just how important everything else we do at Alpha is -- incentivizing students, having AlphaSquadLeaders hover around, putting kids in a physical space, keeping them motivated with the afternoon incentives. Frankly, we aren't shocked by any of this, though we are hoping to grow our online charter segment so that we can continue to speed up development of the Timeback platform and get that into the hands of consumers and public school kids. As Timeback improves, we expect even our virtual schools to impress you with their scores."
EEEEEWWWWW.....Alpha School gives me the creeps. I don't care what anyone says, the 2 hour learning thing in the am is ridiculous!!! When they talk "learning" what do they mean? Do they mean the children run into school excited about what they are to discover that day both from each other and their teachers? Do They mean that children love to talk and discuss about what they are learning so that they can both grow their own learning and their classmates'? Do they impart the understanding of the magic of Leonardo da Vinci? or the beauty found in the science and math of a sea shell? I don't think so.
So they only take self motivated kindergarteners? And they tell children they can make decisions without including their parents (leaving the process wide open for abuse). I have taught for 50 years and I know that throughout that time I "did good" for my students. I have attended PhD challenges; I have watched some students grow in the entertainment world; I have seen some start their own businesses; I have seen some embark in the political arena, and I know that many are intent on changing this world for the better and feel competent and happy. None of this took only 2 hrs a day in setting them on their road and none had me deciding the course they would take by the program they would be indoctrinated with in the afternoons. I do not want a world where children spend their time in rote learning in front of a computer. I do not want a world where children are fractured from their families. As if a 5 year old can make a reasonable decision about their education! This whole discussion makes me almost physically ill.
Throughout my teaching experience I have had classes that tried to outsmart me and get away with something. One wanted to take a non educational field trip. When I asked them where, they said Sausalito. This was 1969 and they were all farm kids. Another time when I had my kids for 3 years, I asked what they wanted to do to start the 3rd year. They said study mythological creatures. This turned into the most amazing study of scientific observation imaginable and at the same time they loved every moment. Yes I had computers in my classroom. they were used as tools to add to our learning or for some to make writing easier (pencils were not their friends neither was spelling). But learning happened and grew at morning meeting when we could discuss whatever was on our minds at the moment or begin to solve a problem that was tugging at us.
So count me out of Alpha School. It will fail like all the others that forgot there were human children and their families involved.
The kid who ends up at an Alpha school is the sort that would typically attend a "high performing" suburban school. Many of these schools pride themselves on their workloads which tend to be disproportionate to their academic results, kind of a "the workload is the point" environment. A few years ago I looked at a private school in the greater Boston area for my kids where they claimed students could expect 4 hours of homework every day. They never mentioned their academic outcomes. Basically there are many parents who believe that a school's reputation for a heavy workload is what will get their kid into an elite university.
As someone who has spent enough years with this nonsense and the detrimental effect it has had on my kids, I understand where the Alpha guys are coming from while not necessarily agreeing with their approach. Given this context, I believe they can get these kids to a higher level with less effort just by dropping the hours of busywork. The question is whether this applies generally, are schools with large numbers of struggling students overloading their students with busywork to please Harvard? Probably not.
Holy shit. Thank you for writing this.
Did you find that PA denied Unbound Academy from forming a charter in their state? Link is in this note: https://substack.com/@playfuldiligence/note/c-295844784
I don't need evidence, data, or facts, I already made my mind up.
Although, TBH, that's just me being sarcastic.
Really interesting article.