What Is a Generative AI Textbook and How Does It Compare to a Traditional Textbook?

I am incredibly grateful for the learning gains produced by generative AI tools like chatGPT and Khanmigo. Not learning gains for students, of course. (Those are largely hypothetical at this point.) Rather learning gains for me. I’m grateful for the ways these tools have helped so many of us learn about learning. Whenever someone suggests that generative AI could improve some part of learning, I read it as an invitation to think more deeply about that part—its nature and why we do it the way we do it.

David Wiley, the Chief Academic Officer of Lumen Learning, invites us in a recent post to think more deeply about the nature of curriculum materials in the process of learning. He describes what he calls a “generative textbook”:

Earlier this week I started wondering – what if, in the future, educators didn’t write textbooks at all? What if, instead, we only wrote structured collections of highly crafted prompts? Instead of reading a static textbook in a linear fashion, the learner would use the prompts to interact with a large language model.

I have spent most of my career developing curriculum materials for math learners so I was happy for the opportunity to wonder what advantages would a generative textbook have over a traditional textbook?

Creating a Generative Textbook

I looked at Matthew Boelkins Active Prelude to Calculus, an open source math curriculum that includes “Motivating Questions” throughout, questions which might help us create prompts like Wiley describes. For example, the first chapter teaches students how to describe relationships using variables.

So I asked chatGPT:

How can algebraic notation help us understand how different quantities are changing in a scenario? Use an example from solid geometry.

ChatGPT offered this response. Don’t worry about reading it. Just look at it.

Then I asked chatGPT for an assessment:

Give me some example questions to help me see if I understand what you mean.

ChatGPT gave me this question among others:

Consider a cube with a side length of "x" units. Write an algebraic expression for its volume, V, and another expression for its surface area, A.

I responded and chatGPT assessed me for correctness.

Analysis

“Is the generative textbook better or worse than a traditional textbook right now?” isn’t all that interesting a question to me any more. I think it’s very hard to conduct an impartial analysis of the two options and conclude anything other than the generative textbook is right now worse than the traditional textbook in almost every way.

Math typography? Worse. Page layout and illustrations? Worse. Mathematical correctness? I have been told that hallucinations of the kind I highlighted above will be resolved any day now for many days now.

The more interesting question to me is, “Will the generative textbook be better or worse?” What will the generative textbook look like after a month of improvements to the generative model that created it? What about a year? Ten years? I do not understand the technical complexity here but I can at least imagine a generative textbook that’s rendered with mathematical typography, that features supportive illustrations, that offers accurate explanations and feedback.

But there are other shortcomings of generative textbooks and I struggle to imagine how technological advancements will resolve them.

Generative textbooks lack a distinctive voice. 

I am not the first person to point out that generative AI produces prose that is competent but indistinct. This owes to the probabilistic way it constructs sentences and paragraphs. It’s cheerful and dull. Certainly, the generative textbook shares this feature with many of the worst textbooks. But the best textbooks feature a distinctive voice and it isn’t clear to me how even the best generative textbooks can break free from this fundamental constraint.

Generative textbooks lack editorial oversight.

Right now, different states require very different learning outcomes for their students. Sometimes those outcomes are mutually contradictory, especially as they relate to this country’s history of exploitation and violence towards indigenous populations and Black people. For example, different state standards answer the question “What was the effect of slavery on Black people?” very differently. 

Which points of view will generative textbooks present? How can we hold these models, which researchers argue inherit the biases of their training data, accountable for ideas that are incorrect or dehumanizing to the kids who read them? Generative textbooks, by definition, lack an author or an editor, and many of the most popular generative AI models lack transparency into the ways they respond to these sensitive questions.

Generative textbooks lack coherence.

Here’s Active Prelude to Calculus with a very useful pedagogical move:

We have already established that any exponential function of the form f(t) = a*b^t where a and b are positive real numbers with b ≠ 1 is always concave up and is either always increasing or always decreasing. We next introduce precise language to describe the behavior of an exponential function’s value as t gets bigger and bigger.

The textbook references previous learning before it introduces new learning. That useful pedagogical move is only possible because the author, Matthew Boelkins, knows what he wrote previously. When you start a chatGPT session in Week 5 of the semester, will your generative textbook remember what you learned in Week 1 of the semester and use it as a foundation for new learning? Will a deductive proof in the fifth chapter of your generative logic textbook know which theorems have been proven in previous chapters?

Perhaps this is technologically possible, but generative tools are, for now, built for much shorter learning interactions and much less context than is common in traditional textbooks. We shouldn’t leave this coherence to chance.

Generative textbooks weaken the cognitive and social bonds between students and teachers.

I opened up a new chatGPT window and asked, “Can you give me five problems to help me know if I understand algebraic functions in geometric contexts?” It gave me five problems. I immediately opened up another chatGPT window and asked the same question again. The five problems were different in both windows.

This means that if I don’t understand problem #3, I can’t message a friend in class and ask, “Have you done #3 in the problem set yet? I have a question about it.” #3 is different in both of our problem sets. Moreover, the teacher can’t do what teachers often do when their students encounter the same difficult problem in a problem set: review it together. Everyone’s problems are different.

How is the best generative textbook better than the best traditional textbook?

I’ll leave this as an exercise for the reader. I very much appreciated Wiley’s generative prompt. But it isn’t obvious to me that his concept of a generative textbook is superior to traditional textbooks. In the areas of typography, page layout, illustrations, and correctness, the generative textbook right now seems worse to me. In the areas of voice, editorial oversight, coherence, and social interaction, the generative textbook seems worse for reasons that are perhaps irreducible.

What Else?

• I interviewed edtech investor Jennifer Carolan about AI in education in the most recent Math Teacher Lounge podcast. We talked about the kinds of generative AI applications that do and don’t excite her. She’s a former teacher and we share many of the same pedagogical convictions though she has a pragmatic perspective that I think balanced my ideological perspective in a useful way.

• A recent Walton Foundation survey found that more teachers than students have used AI in their work. 63% v 42%. That offers an interesting contrast with other technological innovations like social media, where the usage patterns are reversed.

• Derek Newton in Forbes: Instructure, Khan Academy Announce Major Partnership On AI Tutoring, Teaching. IMO higher education needs to get its pedagogical house in order if it wants me to get agitated about university-level partnerships like this. I’m categorizing this one as “probably not gonna hurt anybody IMO.”

• There’s a free conference on AI in education this weekend. It’s led by students and features some interesting speakers.

• Some of my recent chatGPT queries:

  • What should my two friends and I do if we have 12 hours to spend in Portland, Oregon?

  • What are some interesting stories from the game modding community?

  • What are some interesting stories from the fan fiction community?