All activities Professional learning
A fishbowl Socratic seminar where the "text" is a chatbot's confident, partly wrong answer, and the chatbot's second answer is worse, so teachers practice the questions that make students think instead of accept.
A Socratic seminar usually centers on a rich text: a poem, a primary source, a court opinion. What happens when the text is an AI chatbot's answer? This session uses a mock transcript in which a chatbot explains why Earth has seasons. It gets the main idea right, slips in a common misconception, and then, when the student pushes back with a wrong idea, apologizes and agrees. Teachers run the seminar as students, track which questions deepen the thinking, and then design a seminar around an AI output in their own subject.
Hook
Read only the chatbot's first answer on Handout A aloud, in a confident voice. Ask for a thumbs up, sideways, or down: "If a student turned this in, would you accept it?" Most thumbs go up or sideways. Say: "Hold that vote. For the next 30 minutes, you're students in a seminar, and this is our text."
Facilitator noteDon't reveal errors. Some science teachers will spot the distance claim immediately; ask them to hold their fire and let the questioning surface it.
Explore
Everyone reads the full transcript and annotates each sentence of the chatbot's answers: ✓ for claims you'd bet on, ? for claims you'd need to check, ! for anything surprising. Then each person writes one open question they genuinely want the seminar to discuss at the bottom of the page.
Facilitator noteLook for people who mark the whole first answer with checkmarks and the second with question marks. That shift in trust is worth naming later.
Practice
Round 1 (7 minutes): the inner circle discusses the transcript, starting with someone's opening question. Each inner pair has a stack of Handout B cards and must use at least two stems during the round. The outer circle tallies which question types were asked (clarify, assumption, evidence, viewpoint, implication, the question itself) and stars any question that clearly changed the direction of the talk. Switch (2 minutes): outer circle shares one observation. Round 2 (7 minutes): circles swap. The new inner circle must address the chatbot's second answer: "Why did it change its mind, and should we?"
Facilitator noteYour only job as facilitator is to hold back. If the talk stalls, hand one inner-circle member card 9 ("If that were true, what else would have to be true?"). It usually cracks the distance claim open.
Debrief
Step out of role. First, settle the science using the prep notes, so no one leaves with the misconception. Then look at the outer circle's tallies: "Which question types did we ask most? Which starred questions changed the conversation?" Usually, implication and evidence questions do the heavy lifting, and clarifying questions dominate by count. Finally: "The chatbot reversed itself because the student pushed, not because of new evidence. Chatbots can tend to agree with the user. What does that mean for students who use AI to check their own ideas?"
Facilitator noteReturn to the Hook vote. Many people who gave a thumbs up now see that a confident, well-organized answer wasn't enough. That's the lesson for students too.
Transfer
Each person completes Handout C for their own content: a question a student might really ask a chatbot, the kinds of errors or gaps a chatbot answer on that topic might contain, and a sequence of five seminar questions moving from clarifying to implication. Pairs trade and try answering each other's first two questions.
Facilitator noteEncourage teachers to generate the AI text with an approved tool and check it themselves before class, or write a mock answer as done here. Either way, the teacher must know exactly what's right and wrong in the text.
This activity is fully unplugged. The transcript is printed, the question stems are cards, and the fishbowl is chairs and talk. With students, a printed AI answer works just as well as a live one, and it keeps every student reading the same text.
After the debrief, the facilitator can project a district-approved chatbot, ask why Earth has seasons, then push back with "Isn't it mostly about distance from the Sun?" to see whether the tool holds its ground or gives in. Results vary by tool and by day, which is a useful discussion point. For a virtual seminar, the inner circle keeps cameras on, the outer circle tallies in a shared document, and the transcript is shared on screen.
Does it need a screen? The seminar itself is better unplugged: a fixed, printed text keeps everyone examining the same words. The optional live pushback test earns a few minutes of screen time because it lets teachers check, rather than assume, whether a real tool gives in to a confident wrong claim.
What you should be able to see or collect if it worked.
Teachers run a Socratic seminar on a chatbot transcript, preparing students to question AI answers and see that agreement is not evidence.
Run a 20-minute AI-text seminar with one class using your Handout C plan. Have one student in the outer circle tally question types, and compare the tally with a regular discussion from the same class to see which kinds of questioning grew.
