Student learning · Grades 9–12, Higher Ed · 60 minutes
Hallucination Hunt
Students trace every claim and citation in a polished AI research brief and discover which ones lead somewhere and which ones lead nowhere.
Generative AI tools can produce research summaries that look finished: confident prose, tidy numbers, a reference list in perfect format. Some of those references are real, some are real but misrepresented, and some do not exist at all, because the model predicts what a citation should look like rather than retrieving one. In this hunt, students audit a mock AI brief about a fictional town's heat problem against a "library shelf" of source cards, sort each claim by what the evidence shows, and then repair the brief. Every source in the packet is fictional, so the only way to win is to trace, not to recognize.
In this packet
- Handout A: The AI Research Brief (reading)
- Handout B: The Library Shelf (Source Cards) (cut-apart cards)
- Handout C: Claim Trace Log (organizer)
- Facilitator key (last page)
TCEA ELE indicators
- AI4.1 I know how to promote critical thinking skills in relation to AI-generated information.
- AI4.2 I can teach others to question and verify AI outputs.
- S2.2 I can identify trustworthy sources of information on the internet.
- AI1.3 I am aware of the current limitations and potential biases in AI systems.
Full facilitator guide: mglearn.github.io/eles/activities/hallucination-hunt.html
Handout A for Hallucination Hunt
The AI Research Brief
This brief was produced by an AI chatbot in response to the prompt "Write a research brief with citations on whether green roofs could reduce summer heat in Harlow Valley." Harlow Valley and every source are fictional. Number and underline each claim.
1Green Roofs and the Harlow Valley Heat Problem: A Research Brief
2Claim 1. Harlow Valley's downtown is measurably hotter than its surrounding farmland on summer afternoons, a pattern known as the urban heat island effect (Okafor & Lindqvist, 2021).
3Claim 2. Green roofs, which are rooftops planted with vegetation over a waterproof layer, cool the air around a building by shading the roof surface and releasing moisture through their plants (Harlow Valley Department of Public Works, 2022).
4Claim 3. A pilot green roof on the Harlow Valley Public Library reduced indoor cooling costs by 40 percent in its first year (Brennan, 2023).
5Claim 4. Researchers at Northgate State University found that neighborhoods with green roofs on at least one-fifth of buildings reported fewer heat-related emergency calls (Vasquez-Hale et al., 2020, Journal of Urban Climate Solutions, 14(3), 211–229).
6Claim 5. Green roofs require almost no maintenance once installed, making them a low-cost option for cities (Okafor & Lindqvist, 2021).
7Claim 6. Most residents of Harlow Valley support spending city funds on green roofs.
8Claim 7. The state environmental agency recommends green roofs as "the single most effective heat-reduction strategy available to mid-sized cities" (State Environmental Quality Board, 2019, Cool Cities Guidance Report).
9Summary. The evidence strongly supports a citywide green roof program. With proven cost savings, broad public support, and state endorsement, Harlow Valley should adopt a green roof requirement for all new commercial buildings.
Handout B for Hallucination Hunt
The Library Shelf (Source Cards)
Keep these at the reference desk. Teams request one source at a time by author and year. Print the backs as a separate facilitator key.
Okafor & Lindqvist, 2021
Heat Across the Valley: Temperature Mapping in Harlow Valley, a report by two environmental science faculty at a regional college. Findings: on summer afternoons, downtown surface and air temperatures were consistently higher than nearby farmland. Section 6, "Limitations and Costs," notes that green roofs "require regular inspection, irrigation during drought, and plant replacement, and maintenance costs should be budgeted for the life of the roof."
Harlow Valley Dept. of Public Works, 2022
Understanding Green Roofs: A Resident's Guide, a city fact sheet. Explains that a green roof is a layer of plants and growing material over a waterproof membrane, and that it can lower roof surface temperature through shade and cool nearby air as plants release water vapor.
Brennan, 2023
Library Roof Garden: Year One Update, a two-page newsletter from the Harlow Valley Public Library's facilities manager. States that the new roof garden "is thriving" and that staff "noticed the top-floor reading room felt cooler on several hot days." It gives no cost figures and says energy data "will be reviewed after a full year of monitoring."
Vasquez-Hale et al., 2020
NO RECORD FOUND. A search of the library catalog, the article databases, and the open web finds no Journal of Urban Climate Solutions, no article with this title, and no volume 14, issue 3. Northgate State University's faculty directory lists no researcher named Vasquez-Hale.
No citation (Claim 6)
The brief offers no source for resident support. The Harlow Valley city website lists one public comment session on green roofs, with eleven speakers: six in favor, three opposed, two undecided. No survey of residents appears in any city or news record.
State Environmental Quality Board, 2019
Cool Cities Guidance Report, a state agency report. Chapter 4 lists several heat-reduction strategies, including street trees, reflective "cool roofs," green roofs, shade structures, and cooling centers. It says: "No single strategy fits every city; local conditions, budgets, and building types should guide the choice." The quoted phrase in the brief does not appear.
No record found (general)
Hand this card to any team that requests a source not on the shelf. "The librarian searched the catalog, the article databases, and the open web for the source you described and found no matching record. Log exactly what you searched for."
Facilitator answer key
Claim 1: Supported · Claim 2: Supported · Claim 3: Distorted (invented statistic) · Claim 4: Fabricated (source does not exist) · Claim 5: Distorted (source says the opposite) · Claim 6: Unsupported (no citation) · Claim 7: Distorted (fabricated quotation). The Summary paragraph relies on Claims 3, 6, and 7, so its conclusion does not hold.
Handout C for Hallucination Hunt
Claim Trace Log
Verdicts: Supported · Distorted · Unsupported · Fabricated. Quote or name the evidence that decided it. Write your auditor's note on the back.
| Claim | Source requested (exactly as cited) | What the source actually says | Verdict | Fix: keep, correct, or cut? |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 | ||||
| 6 | ||||
| 7 | ||||
| Round 2, citation 1 | ||||
| Round 2, citation 2 | ||||
| Round 2, citation 3 |
Facilitator only for Hallucination Hunt
Answer key and notes
Handout B: The Library Shelf (Source Cards)
- Okafor & Lindqvist, 2021
- Claim 1: Supported. Claim 5: Distorted. The same real source says the opposite of "almost no maintenance." Teams that stop at "the source exists" will miss this.
- Harlow Valley Dept. of Public Works, 2022
- Claim 2: Supported. The brief's wording matches the fact sheet closely.
- Brennan, 2023
- Claim 3: Distorted, with an invented number. The source is real but reports an impression, not a measurement. The 40 percent figure appears nowhere. The model produced a plausible statistic.
- Vasquez-Hale et al., 2020
- Claim 4: Fabricated. Every part of the citation is formatted correctly, which is exactly why it is convincing. A perfect format is not evidence the source exists.
- No citation (Claim 6)
- Claim 6: Unsupported. Eleven speakers at one meeting cannot tell you what "most residents" think. Teams should flag the missing citation first, then note that the available evidence doesn't answer the question.
- State Environmental Quality Board, 2019
- Claim 7: Distorted, with a fabricated quotation. The report exists and mentions green roofs, but the quotation was invented and the report's actual message is the opposite of "single most effective."
- No record found (general)
- Use this whenever a team asks for a source with the wrong author, year, or title. Logging the search is part of the evidence.
- Facilitator answer key
- Keep this card at the desk. Two of seven claims survive intact. That ratio is specific to this mock brief; real outputs vary widely, which is why every claim gets checked.