I am incredibly grateful for the learning gains produced by generative AI tools like chatGPT and Khanmigo. Not learning gains for *students, *of course. (Those are largely hypothetical at this point.) Rather learning gains for *me*. I’m grateful for the ways these tools have helped so many of us *learn about learning*. Whenever someone suggests that generative AI could improve some part of learning, I read it as an invitation to think more deeply about that part—its nature and why we do it the way we do it.

David Wiley, the Chief Academic Officer of Lumen Learning, invites us in a recent post to think more deeply about the nature of *curriculum materials* in the process of learning. He describes what he calls a “generative textbook”:

Earlier this week I started wondering – what if, in the future, educators didn’t write textbooks at all? What if, instead, we only wrote structured collections of highly crafted prompts? Instead of reading a static textbook in a linear fashion, the learner would use the prompts to interact with a large language model.

I have spent most of my career developing curriculum materials for math learners so I was happy for the opportunity to wonder **what advantages would a generative textbook have over a traditional textbook?**

### Creating a Generative Textbook

I looked at Matthew Boelkins *Active Prelude to Calculus, *an open source math curriculum that includes “Motivating Questions” throughout, questions which might help us create prompts like Wiley describes. For example, the first chapter teaches students how to describe relationships using variables.

So I asked chatGPT:

How can algebraic notation help us understand how different quantities are changing in a scenario? Use an example from solid geometry.

ChatGPT offered this response. Don’t worry about reading it. Just *look* at it.

Then I asked chatGPT for an assessment:

Give me some example questions to help me see if I understand what you mean.

ChatGPT gave me this question among others:

Consider a cube with a side length of "x" units. Write an algebraic expression for its volume, V, and another expression for its surface area, A.

I responded and chatGPT assessed me for correctness.

### Analysis

“Is the generative textbook better or worse than a traditional textbook *right now*?” isn’t all that interesting a question to me any more. I think it’s very hard to conduct an impartial analysis of the two options and conclude anything other than **the generative textbook is ****right now**** worse than the traditional textbook in almost every way**.

Math typography? Worse. Page layout and illustrations? Worse. Mathematical correctness? I have been told that hallucinations of the kind I highlighted above will be resolved any day now for many days now.

The more interesting question to me is, “*Will* the generative textbook be better or worse?” What will the generative textbook look like after a month of improvements to the generative model that created it? What about a year? Ten years? I do not understand the technical complexity here but I can at least *imagine* a generative textbook that’s rendered with mathematical typography, that features supportive illustrations, that offers accurate explanations and feedback.

But there are *other* shortcomings of generative textbooks and I struggle to imagine how technological advancements will resolve them.

#### Generative textbooks lack a distinctive voice.

I am not the first person to point out that generative AI produces prose that is competent but indistinct. This owes to the probabilistic way it constructs sentences and paragraphs. It’s cheerful and dull. Certainly, the generative textbook shares this feature with many of the *worst *textbooks. But the *best* textbooks feature a distinctive voice and it isn’t clear to me how even the *best* generative textbooks can break free from this fundamental constraint.

#### Generative textbooks lack editorial oversight.

Right now, different states require very different learning outcomes for their students. Sometimes those outcomes are mutually contradictory, especially as they relate to this country’s history of exploitation and violence towards indigenous populations and Black people. For example, different state standards answer the question “What was the effect of slavery on Black people?” very differently.

Which points of view will generative textbooks present? How can we hold these models, which researchers argue inherit the biases of their training data, accountable for ideas that are incorrect or dehumanizing to the kids who read them? Generative textbooks, by definition, lack an author or an editor, and many of the most popular generative AI models lack transparency into the ways they respond to these sensitive questions.

#### Generative textbooks lack coherence.

Here’s *Active Prelude to Calculus *with a very useful pedagogical move:

We have already establishedthat any exponential function of the formf(t) = a*b^twhereaandbare positive real numbers withb ≠ 1is always concave up and is either always increasing or always decreasing.We next introduceprecise language to describe the behavior of an exponential function’s value astgets bigger and bigger.

The textbook references *previous* learning before it introduces *new *learning. That useful pedagogical move is only possible because the author, Matthew Boelkins, *knows* what he wrote previously. When you start a chatGPT session in *Week 5* of the semester, will your generative textbook remember what you learned in *Week 1* of the semester and use it as a foundation for new learning? Will a deductive proof in the fifth chapter of your generative logic textbook know which theorems have been proven in previous chapters?

Perhaps this is technologically possible, but generative tools are, for now, built for much shorter learning interactions and much less context than is common in traditional textbooks. We shouldn’t leave this coherence to chance.

#### Generative textbooks weaken the cognitive and social bonds between students and teachers.

I opened up a new chatGPT window and asked, “Can you give me five problems to help me know if I understand algebraic functions in geometric contexts?” It gave me five problems. I immediately opened up another chatGPT window and asked the same question again. The five problems were different in both windows.

This means that if I don’t understand problem #3, I can’t message a friend in class and ask, “Have you done #3 in the problem set yet? I have a question about it.” #3 is different in both of our problem sets. Moreover, the teacher can’t do what teachers often do when their students encounter the same difficult problem in a problem set: review it together. Everyone’s problems are different.

#### How is the best generative textbook *better* than the best traditional textbook?

I’ll leave this as an exercise for the reader. I very much appreciated Wiley’s generative prompt. But it isn’t obvious to me that his concept of a generative textbook is superior to traditional textbooks. In the areas of typography, page layout, illustrations, and correctness, the generative textbook *right now* seems worse to me. In the areas of voice, editorial oversight, coherence, and social interaction, the generative textbook seems worse for reasons that are perhaps irreducible.

### What Else?

I interviewed edtech investor

**Jennifer Carolan**about AI in education in the most recent Math Teacher Lounge podcast. We talked about the kinds of generative AI applications that do and don’t excite her. She’s a former teacher and we share many of the same pedagogical convictions though she has a pragmatic perspective that I think balanced my ideological perspective in a useful way.A recent

**Walton Foundation**survey found that more teachers than students have used AI in their work. 63% v 42%. That offers an interesting contrast with other technological innovations like social media, where the usage patterns are reversed.**Derek Newton**in Forbes: Instructure, Khan Academy Announce Major Partnership On AI Tutoring, Teaching. IMO higher education needs to get its pedagogical house in order if it wants me to get agitated about university-level partnerships like this. I’m categorizing this one as “probably not gonna hurt anybody IMO.”There’s a free conference on AI in education this weekend. It’s led by students and features some interesting speakers.

Some of my recent chatGPT queries:

What should my two friends and I do if we have 12 hours to spend in Portland, Oregon?

What are some interesting stories from the game modding community?

What are some interesting stories from the fan fiction community?

When I first read this, my gut response was, "No, it can't be. It's a chat program that's been trained on available data." I decided to toy around with your prompt and asked a whole bunch of follow-up questions to refine the results. It was then I understood that because I know my field and my audience, which is middle school students, I knew the right questions to ask. This suggests that to properly use this tool, one needs specific knowledge and experience.

The generative nature of GPT is beneficial, but only if the user has the knowledge to separate the good from the bad.

So, despite the lack of a human touch, it's not quite ready to outshine traditional writing created by someone with a strong grasp of pedagogy and content knowledge.

Oh and btw, test this one out to see if you think it’s really adopting your voice ;)

[Assume the role of a mathematician and a seasoned math educator at the K-12 level. Adopt the voice of Dan Meyer in math education to answer this question:

“How can algebraic notation help us understand how different quantities are changing in a scenario? Use an example from solid geometry.”]

P.S. Use GPT4.0 instead of 3.5!

Generative textbooks are worse than traditional textbooks, it's true, along traditional metrics, in the same way that most YouTube is worse than traditional TV in terms of production values, editorial oversight, since budgets are far lower and production skills are usually far worse. But YouTube has the unique advantage of allowing individual voices to shine, and that is enough to let it overtake traditional media. I suspect, like you say, that "generative textbook" is a "horseless carriage" linguistic misfire based on history, just as Google Search is a fundamentally different beast from Yahoo! indexing. The interesting question to me is whether people will put in the active effort to use tools like ChatGPT to sharpen their learning. I love the idea of active learners participating in their learning, but I think it will take more than the existence of ChatGPT to make this happen.