Andrej Karpathy Is in Trouble

Karpathy helped build ChatGPT and now he's taking on the much larger challenge of helping people learn stuff.

Andrej Karpathy, one of OpenAI’s co-founders, left OpenAI in February and has now set his sights on education. In July, he announced the launch of a new AI edtech startup called Eureka Labs.

This Teacher + AI symbiosis could run an entire curriculum of courses on a common platform. If we are successful, it will be easy for anyone to learn anything, expanding education in both reach (a large number of people learning something) and extent (any one person learning a large amount of subjects, beyond what may be possible today unassisted).

Karpathy helped create one of the most advanced computing technologies the world has ever seen and he will need to bring every bit of that determination and creativity to his new, much harder task of helping people learn stuff. It isn’t too late for him to retreat from that task into something simpler. Cold fusion maybe. Since he seems quite committed to this venture, however, I am going to use this newsletter to help him avoid the mistakes of his predecessors in the world of online education.

Add more verbs than “watch” and “read.”

Eight months ago, Karpathy uploaded a video to YouTube explaining large language models. It is a very popular video with over two million views at the time of this writing, a fantastic accomplishment for a YouTuber but an incredible liability for an edtech founder.

If you come to edtech through schooling and other venues of compulsory face-to-face learning, you understand quite well that just because a student’s body is at their desk that doesn’t mean their spirit or mind is present there as well. 

If you come to edtech through YouTube, it is easy to convince yourself that the people who are watching your videos are also learning from your videos, that they are also enjoying learning from your videos, that they would also enjoy learning from more of your videos. Certainly, all of that is true for some percentage of Karpathy’s two million views, but far less than 100%. I would anchor our predictions at roughly 5%.

Karpathy is now jumping into the top of a flume ride full of people who found early success explaining stuff on YouTube and later struggled to fulfill their greater ambitions for education, people like:

• Sebastian Thrun, who created a very popular YouTube playlist on artificial intelligence, and later turned that playlist into Udacity, and later fell far short of his ambitions in K-16 schooling, and later pivoted to friendlier terrain in corporate education, and later sold his company altogether for what is widely assumed to be a steep discount.

• Andrew Ng, who put videos of his Stanford lectures online, and later created Coursera with Daphne Koeller to surround those videos with a learning management system, and later saw course completion rates hover around 10%, and later saw the stock price of his company decline 85% from its initial public offering.

• Sal Khan, a popular YouTuber whose videos were first watched by family members and then by millions, who later scaled his ideas about learning into Khan Academy, a platform that has demonstrated significant learning gains in studies where researchers first throw out 95% of the students in the study.

A large reason why these edtech startups do not work well for the majority of students is that the majority of students are not particularly interested in reading academic text or watching sequences of explanatory videos. If they were, we would have solved mass education many centuries ago with the printing press. Thomas Edison would have been correct in 1922 that “the motion picture is destined to revolutionize our educational system and that in a few years it will supplant largely, if not entirely, the use of textbooks.”

Whether generative media will over- or underperform static media here remains to be studied, but let’s just say I have my suspicions.

Whenever Karpathy launches his first course, I will be curious if he asks students to engage in more verbs than just read and watch. Will he also ask students to do the work that has been common in online courseware for at least forty years, work like selecting multiple choice responses, typing in numbers, writing text-based responses? Probably. He will likely have students write code as well, given the nature of his first planned course on AI. But it is an open question for me if he will ask students to engage in the kinds of concrete, intuitive, and sensory work that is accessible and engaging to many more learners, verbs like sketch, estimate, argue, describe, notice, wonder, explore, etc.

Make thinking matter.

A student says, “yes, I believe I have learned something here,” so they send a number, a sketch, some text, some representation of their learning, into Karpathy’s learning platform. The student will likely wonder two questions here:

“Is my thinking right?” is certainly one.

“Does my thinking matter?” is another.

When students receive a ✅ or ❌ on their thinking, that feedback answers their first question about correctness, but not their second question about significance. 

A satisfying aspect of learning to program a computer is the compiler never responds to your work with just a ✅ or an ❌. If you have an error, you receive information about which line of code had what kind of error. If you don’t have an error, you’ll see something happen. It may not be what you wanted to see happen, though, which is generally a very interesting and productive moment of learning.

The challenge, one that we have taken on at Amplify, is to make student thinking matter in disciplines like math, science, and language arts. This has required lots of creativity from talented teams implementing designs we call “responsive feedback” where visuals respond in meaningful ways to student thinking. For example, where:

• A sheep responds to an inequality.

• A turtle responds to a function.

• A robot responds to a point.

• And so on.

Add people.

People often answer the question, “Does my thinking matter?” by sharing their thinking with other people. This kind of sharing is commonplace in K-16 classrooms worldwide. It shows up in partner conversations, whole-class discussion, small study groups, and so on.

But the edtech graveyard is stacked several caskets deep with resources created by autodidacts—people who are highly self-motivated and highly self-directed, people who are just fine learning by themselves, thank you—for autodidacts. We might estimate the population of autodidacts at about (well look at that) 5% of the overall population.

If Karpathy seeks to broaden the appeal and efficacy of his platform beyond 5% of learners, he’ll need to add community. He’ll need students to understand that their work has an audience, that they can learn from other people and other people can learn from them.

Create connections between people.

Community cuts two ways, though. As the community grows larger, social cohesion between learners grows, but only up to a certain point past which it’s easy to feel lost in the community. It’s easy to feel overwhelmed by the endless scroll of responses and invisible in the crowd. Who are these people? Who am I among them? For decades, online educators have tried to resolve this contradiction by telling students to “go respond to three other students” with no meaningful effect on belonging or learning.

Very few of Karpathy’s two million viewers expected to interact with one another. But if two million people sign up for a course, Karpathy will need to group them in ways that feel cohesive but not overwhelming, in ways that take advantage of the particularities of their thinking, helping students learn across their similarities and benefit from their differences.

I don’t have any idea what Karpathy means when he describes a “Teacher + AI symbiosis” but the task of analyzing student thinking and using that analysis to group students in productive ways is a use case for AI that I find very interesting—less about generative content and more about generative connection.

👋 Best of luck!

Anyway, I’m wishing Karpathy our Mathworlds best.

The world does not need another learning management system that shows students videos and text, that sends them ✅’s and ❌’s in response to their multiple choice responses, that isolates them from the brilliance of their classmates. It doesn’t need another LMS like that even if someone bolts an AI chatbot on top of it, generating more text that students aren’t interested in reading.

If that is what Karpathy is building, he will likely fall short of his considerable ambitions here. But the good news for him is that if helping people learn stuff proves too challenging he can likely fall back on the easier task of helping computers learn stuff instead.