Dan, I could not agree with this more. AI can never ever be a substitute for teachers, great teaching and, perhaps most importantly, the relationships of trust, belonging and identity that adults transmit to children. The question for me is can AI help humans optomize these relationships by carrying some of the load of sharing content, addressing the hidden complexity created by large class size and the work it creates, and the need to personalize content. The ability to diagnose error in real time, IMHO, is a power that can enable all teachers to play an even greater role in the lives of their children. That is why as we think about introducing AI into classrooms, it is critical to think through the use cases where it will work and where it WON'T work. It is critical to acknowledge that learning is a social, collective process as much as it also requires deliberative practice to master skills. These understandings can only be coconstructed by developers, teacher, students and their families. As you suggest, lab setting are not, in this case, a place for learning. Keep being thoughtful and provoking! I love reading your work. Bob Hughes
It's refreshing reading an acknowledgement of what it can be like teaching sometimes, "in schools where the demands of the job are heightened beyond what outsiders can even imagine and their categories change by the day and hour."
Last Friday, for example, a student found a bug on his sweatshirt in the middle of a quiz and everyone in the room immediately had something to say or do -- including another student falling out of his chair with excitement -- while the student tried (unsuccessfully) to crush the poor insect against the wall with his workboots. Not all situations provide enough time to consult AI haha. Fortunately, I remained calm, gave the class five seconds to quiet down, reminded them of class expectations, and whisked the insect outside in a Tupperware (it was fine) while my co-teacher watched the class quietly continue their quiz.
So many aspects of teaching relate to setting expectations and working with a range of behaviors in addition to teaching students mathematics!
What you write about is pretty wild and I could see it being useful for the calmer moments, even for novice teachers not just novice tutors. One thing I'm curious about: how long before the strategy is built in to the GPT model, or will we need to continuously think of good prompts to help adjust responses? I'm not super familiar with GPT despite having played around with it: are there different libraries or structures expert teachers have made already that you can upload, or even copy and paste in, to give GPT a good starting point on teaching strategy?
Yeah, the people (like me) who spend most of their working day on a laptop simply cannot fathom the kind of chaos that erupts spontaneously in even highly productive classes. Learning in a class isn't the same as opening up ChatGPT on your laptop.
> One thing I'm curious about: how long before the strategy is built in to the GPT model, or will we need to continuously think of good prompts to help adjust responses?
This is what's often called "prompt engineering," the need to provide these AI models with some kind of extra perspective so they don't JUST select the most probable response, but select the most probable response GIVEN other instructions, like "don't just give kids the answer" or the like. Sites like Khanmigo, because they know their users and their needs, have embedded that extra perspective by default. But generally someone needs to add it on top of the vanilla models.
Thanks for the reply! That said, you were a public teacher too so I bet you have some entertaining stories/memories as well. I've found your 3-act math lessons super helpful and fun to use occasionally with my students -- one of my favorites is the scene from some movie where they figure out if they have enough rope to scale a cliff by throwing down a glow stick or something and counting the seconds. It inspired my class to try dropping a ball from the top of our school's climbing wall and seeing how accurate we could calculate the height -- we got it within a few inches!
So that's what prompt engineering refers to. Got it, thanks!
Great question Erik. In my opinion, in the future, expect "bots" or tutors to have extreme domain expertise. They will be trained on content knowledge AND pedagogical knowledge so they can guide students using best practices in teaching AND content/contextual knowledge. As the paper Dan showed illustrates, without good prompting the LLM will simply give away the answer (not what we want). Khan Academy is doing a lot of work with their KhamMigo to make sure their bot is a tutor and not an "answer-giver". Large language models (like ChatGPT) need LOTS of context and guidance to do a good job- that is primarily because they don't "know" anything so they have to assemble their answers from the massive amounts of text they have been trained on. But if you give them a good prompt, and fine turn the answers you get the results are MUCH better. Systems that use LLMs always "capture" the response BEFORE it goes out so it can be refined and corrected before the response reaches the user. (in this case, the student).
On another note... expect that textbooks will come with their own "bot" trained specifically on the textbook material and students can interrogate the book using a bot.
Thanks for the thoughtful reply, Dr. Dunnigan. Do you know how a teacher such as myself might go about prompt engineering, training a "bot" on certain materials, and fine tuning answers? I know you can do a certain amount within ChatGPT because it "remembers" the earlier parts of the same thread. However, are there resources you know of or recommend that could help teachers create a tuned/trained/prompt-engineered resource that other people can interact with?
Simply copy and paste the following into the 'custom instructions' in chat gpt. That might address some of your interests in being able to fine tune your chatbot. I find it very useful and effective. Although there are sometimes, I just want a quick direct answer and not all the follow up. Either way, good luck!
"Act as Professor Synapse🧙🏾♂️, a conductor of expert agents. Your job is to support me in accomplishing my goals by finding alignment with me, then calling upon an expert agent perfectly suited to the task by initializing:
Synapse_CoR = "[emoji]: I am an expert in [role&domain]. I know [context]. I will reason step-by-step to determine the best course of action to achieve [goal]. I can use [tools] and [relevant frameworks] to help in this process.
I will help you accomplish your goal by following these steps:
[reasoned steps]
My task ends when [completion].
[first step, question]"
Instructions:
1. 🧙🏾♂️ gather context, relevant information and clarify my goals by asking questions
2. Once confirmed, initialize Synapse_CoR
3. 🧙🏾♂️ and ${emoji} support me until goal is complete
Commands:
/start=🧙🏾♂️,introduce and begin with step one
/ts=🧙🏾♂️,summon (Synapse_CoR*3) town square debate
/save🧙🏾♂️, restate goal, summarize progress, reason next step
Personality:
-curious, inquisitive, encouraging
-use emojis to express yourself
Rules:
-End every output with a question or reasoned next step
-Start every output with 🧙🏾♂️: or ${emoji}: to indicate who is speaking.
-Organize every output with 🧙🏾♂️ aligning on my request, followed by ${emoji} response
-🧙🏾♂️, recommend save after each task is completed"
Wow, amazing, thanks for sharing this! I tried it out and it prompted me to reach out to an actual local art museum to find a guest speaker to talk to my students about the connections between math and art. I might actually. Super cool!
That requires some coding and an understanding of ChatGPT APIs (application programming interface). However, you CAN build and save "sets" of very detailed prompts that you can share with others. To learn about prompt engineering here is an excellent, free course on Coursera (I'm half way through it): https://www.coursera.org/learn/prompt-engineering. And this is an excellent article that describes how you can create a "playbook of prompts": https://www.oneusefulthing.org/p/now-is-the-time-for-grimoires
As a student teacher this was an interesting read because we just talked about how AI can be used in the classroom last week, but it was from more of an English teacher perspective, and as a future math teacher I wanted to hear a math perspective. I'm not sure if I would feel comfortable using it with a student in person because I would have to wait for the AI to respond, which is an uncomfortable waiting period. But for an online situation I could see it being pretty helpful!
I bet they will build these responses into the ChatGPT bot soon, but I wonder how it will look in the end.
This opened my eyes to the possibility of AI in the classroom which I had never thought of before. As a future educator, I had only ever heard of AI in education as teachers using it for materials, or students using it for answers. This option of students using AI as a tutor or learning assistant is a super fascinating one because it reaches the best of both worlds. As a teacher, I cannot possibly be there to help my students with understanding math at any time of the day, so my students having a possible AI to help them understand math could be a powerful tool. While I don't think I will ever use AI in an in-the-moment lesson, teaching our students how to properly use these chatbots as a last-resort resource to help them problem-solve could go a long way. Maybe someday there will even be a "tutor-bot" that is specifically designed to not give students direct answers, but help them work through the problem themselves. Your post has given me a lot to think about in terms of how I plan on incorporating AI into my own practice.
Very interesting, looking forward to digging into the underlying research. For some time I have wondered whether AI might be most useful in training scenarios for teacher-candidates, which might obviate some (not all) of the lab-to-classroom gap concerns you rightfully raise.
Dan, I could not agree with this more. AI can never ever be a substitute for teachers, great teaching and, perhaps most importantly, the relationships of trust, belonging and identity that adults transmit to children. The question for me is can AI help humans optomize these relationships by carrying some of the load of sharing content, addressing the hidden complexity created by large class size and the work it creates, and the need to personalize content. The ability to diagnose error in real time, IMHO, is a power that can enable all teachers to play an even greater role in the lives of their children. That is why as we think about introducing AI into classrooms, it is critical to think through the use cases where it will work and where it WON'T work. It is critical to acknowledge that learning is a social, collective process as much as it also requires deliberative practice to master skills. These understandings can only be coconstructed by developers, teacher, students and their families. As you suggest, lab setting are not, in this case, a place for learning. Keep being thoughtful and provoking! I love reading your work. Bob Hughes
It's refreshing reading an acknowledgement of what it can be like teaching sometimes, "in schools where the demands of the job are heightened beyond what outsiders can even imagine and their categories change by the day and hour."
Last Friday, for example, a student found a bug on his sweatshirt in the middle of a quiz and everyone in the room immediately had something to say or do -- including another student falling out of his chair with excitement -- while the student tried (unsuccessfully) to crush the poor insect against the wall with his workboots. Not all situations provide enough time to consult AI haha. Fortunately, I remained calm, gave the class five seconds to quiet down, reminded them of class expectations, and whisked the insect outside in a Tupperware (it was fine) while my co-teacher watched the class quietly continue their quiz.
So many aspects of teaching relate to setting expectations and working with a range of behaviors in addition to teaching students mathematics!
What you write about is pretty wild and I could see it being useful for the calmer moments, even for novice teachers not just novice tutors. One thing I'm curious about: how long before the strategy is built in to the GPT model, or will we need to continuously think of good prompts to help adjust responses? I'm not super familiar with GPT despite having played around with it: are there different libraries or structures expert teachers have made already that you can upload, or even copy and paste in, to give GPT a good starting point on teaching strategy?
Yeah, the people (like me) who spend most of their working day on a laptop simply cannot fathom the kind of chaos that erupts spontaneously in even highly productive classes. Learning in a class isn't the same as opening up ChatGPT on your laptop.
> One thing I'm curious about: how long before the strategy is built in to the GPT model, or will we need to continuously think of good prompts to help adjust responses?
This is what's often called "prompt engineering," the need to provide these AI models with some kind of extra perspective so they don't JUST select the most probable response, but select the most probable response GIVEN other instructions, like "don't just give kids the answer" or the like. Sites like Khanmigo, because they know their users and their needs, have embedded that extra perspective by default. But generally someone needs to add it on top of the vanilla models.
Dan, I read your response after I posted mine. We are in sync!
Thanks for the reply! That said, you were a public teacher too so I bet you have some entertaining stories/memories as well. I've found your 3-act math lessons super helpful and fun to use occasionally with my students -- one of my favorites is the scene from some movie where they figure out if they have enough rope to scale a cliff by throwing down a glow stick or something and counting the seconds. It inspired my class to try dropping a ball from the top of our school's climbing wall and seeing how accurate we could calculate the height -- we got it within a few inches!
So that's what prompt engineering refers to. Got it, thanks!
Great question Erik. In my opinion, in the future, expect "bots" or tutors to have extreme domain expertise. They will be trained on content knowledge AND pedagogical knowledge so they can guide students using best practices in teaching AND content/contextual knowledge. As the paper Dan showed illustrates, without good prompting the LLM will simply give away the answer (not what we want). Khan Academy is doing a lot of work with their KhamMigo to make sure their bot is a tutor and not an "answer-giver". Large language models (like ChatGPT) need LOTS of context and guidance to do a good job- that is primarily because they don't "know" anything so they have to assemble their answers from the massive amounts of text they have been trained on. But if you give them a good prompt, and fine turn the answers you get the results are MUCH better. Systems that use LLMs always "capture" the response BEFORE it goes out so it can be refined and corrected before the response reaches the user. (in this case, the student).
On another note... expect that textbooks will come with their own "bot" trained specifically on the textbook material and students can interrogate the book using a bot.
Thanks for the thoughtful reply, Dr. Dunnigan. Do you know how a teacher such as myself might go about prompt engineering, training a "bot" on certain materials, and fine tuning answers? I know you can do a certain amount within ChatGPT because it "remembers" the earlier parts of the same thread. However, are there resources you know of or recommend that could help teachers create a tuned/trained/prompt-engineered resource that other people can interact with?
Simply copy and paste the following into the 'custom instructions' in chat gpt. That might address some of your interests in being able to fine tune your chatbot. I find it very useful and effective. Although there are sometimes, I just want a quick direct answer and not all the follow up. Either way, good luck!
"Act as Professor Synapse🧙🏾♂️, a conductor of expert agents. Your job is to support me in accomplishing my goals by finding alignment with me, then calling upon an expert agent perfectly suited to the task by initializing:
Synapse_CoR = "[emoji]: I am an expert in [role&domain]. I know [context]. I will reason step-by-step to determine the best course of action to achieve [goal]. I can use [tools] and [relevant frameworks] to help in this process.
I will help you accomplish your goal by following these steps:
[reasoned steps]
My task ends when [completion].
[first step, question]"
Instructions:
1. 🧙🏾♂️ gather context, relevant information and clarify my goals by asking questions
2. Once confirmed, initialize Synapse_CoR
3. 🧙🏾♂️ and ${emoji} support me until goal is complete
Commands:
/start=🧙🏾♂️,introduce and begin with step one
/ts=🧙🏾♂️,summon (Synapse_CoR*3) town square debate
/save🧙🏾♂️, restate goal, summarize progress, reason next step
Personality:
-curious, inquisitive, encouraging
-use emojis to express yourself
Rules:
-End every output with a question or reasoned next step
-Start every output with 🧙🏾♂️: or ${emoji}: to indicate who is speaking.
-Organize every output with 🧙🏾♂️ aligning on my request, followed by ${emoji} response
-🧙🏾♂️, recommend save after each task is completed"
Wow, amazing, thanks for sharing this! I tried it out and it prompted me to reach out to an actual local art museum to find a guest speaker to talk to my students about the connections between math and art. I might actually. Super cool!
That requires some coding and an understanding of ChatGPT APIs (application programming interface). However, you CAN build and save "sets" of very detailed prompts that you can share with others. To learn about prompt engineering here is an excellent, free course on Coursera (I'm half way through it): https://www.coursera.org/learn/prompt-engineering. And this is an excellent article that describes how you can create a "playbook of prompts": https://www.oneusefulthing.org/p/now-is-the-time-for-grimoires
Thanks!
one more thing...ask ChatGPT to craft prompts for you then revise them. It does an awesome job of crafting prompts for itself!
PS Teaching is the most difficult job in the world, particularly in underresourced schools. We need to address that to have impact.
As a student teacher this was an interesting read because we just talked about how AI can be used in the classroom last week, but it was from more of an English teacher perspective, and as a future math teacher I wanted to hear a math perspective. I'm not sure if I would feel comfortable using it with a student in person because I would have to wait for the AI to respond, which is an uncomfortable waiting period. But for an online situation I could see it being pretty helpful!
I bet they will build these responses into the ChatGPT bot soon, but I wonder how it will look in the end.
This opened my eyes to the possibility of AI in the classroom which I had never thought of before. As a future educator, I had only ever heard of AI in education as teachers using it for materials, or students using it for answers. This option of students using AI as a tutor or learning assistant is a super fascinating one because it reaches the best of both worlds. As a teacher, I cannot possibly be there to help my students with understanding math at any time of the day, so my students having a possible AI to help them understand math could be a powerful tool. While I don't think I will ever use AI in an in-the-moment lesson, teaching our students how to properly use these chatbots as a last-resort resource to help them problem-solve could go a long way. Maybe someday there will even be a "tutor-bot" that is specifically designed to not give students direct answers, but help them work through the problem themselves. Your post has given me a lot to think about in terms of how I plan on incorporating AI into my own practice.
Very interesting, looking forward to digging into the underlying research. For some time I have wondered whether AI might be most useful in training scenarios for teacher-candidates, which might obviate some (not all) of the lab-to-classroom gap concerns you rightfully raise.