All activities Professional learning
Leaders take apart eight mock dashboard panels, including an AI-generated "insight," to practice asking what the data shows, what it can't show, and who could be harmed by a hasty conclusion.
Dashboards arrive with confidence: colors, arrows, and increasingly, automatically generated "insights" written in plain English. Leaders are expected to act on them. In this session, teams read eight fictional dashboard panels with the same critical eye they'd teach students to bring to an infographic: checking axes, baselines, sample sizes, what the metric actually measures, and what the data can't tell you. They also examine what personal data each panel exposes and to whom. Teams finish by redrawing one misleading panel honestly and writing a three-sentence summary a staff meeting could trust.
Hook
Sketch Panel 1 on chart paper exactly as described: a bar chart where the y-axis starts at 80%. Ask: "If this were on the screen at a board meeting, what would people conclude?" Then redraw the axis starting at 0% right next to it. Ask: "Same data. What changed?"
Facilitator notePeople usually say attendance "collapsed." The actual drop is about three percentage points. Both charts are accurate; only one is honest about scale.
Explore
Teams rotate through the eight panels (or work through them at their table), spending about two minutes each. For each panel they fill a row of Handout B: the claim someone might make, what the panel actually shows, what it can't tell us, and the next question or data needed. They also mark any panel that exposes personal information to people who may not need it.
Facilitator notePanel 5, the AI-generated insight, tends to spark the biggest debate. Ask: "What data could the software actually have used to decide students are 'disengaged'?"
Debrief
Build a class list on chart paper titled "Questions to ask any dashboard" from what teams found: What's the baseline? How many students is this? What is this metric actually counting? Could something else explain it? Who can see this, and do they need to? Compare with the facilitator key on the card backs.
Facilitator noteConnect to media literacy: these are the same questions students should ask about a viral infographic. Leaders who model them teach them.
Create
Each team chooses one misleading panel and (1) redraws it honestly on paper or in a spreadsheet, and (2) writes a three-sentence summary for a staff meeting: what the data shows, what it can't tell us, and what we'll look at next. Post the redrawn charts.
Facilitator noteLook for summaries that include uncertainty without drowning in it. "We don't know yet, and here's how we'll find out" is a leadership sentence.
Reflect
Each person completes Handout C for a dashboard they actually use. Pairs compare and each names one access or display change they'll ask about, for example who can see individual student names, or whether an AI-generated label appears in a place families or other students can see it.
Facilitator noteUnder FERPA, access to student records generally should be limited to school officials with a legitimate educational interest. Your district's policy details matter; check them rather than guessing.
Transfer
Each person writes the one question from the class list they'll ask at their next data meeting, and who they'll ask it of.
Facilitator noteSuggest leaders post the "Questions to ask any dashboard" list in the room where data meetings happen.
All eight panels are described in words on printed cards, and teams redraw charts on graph paper. The facilitator can sketch the key panels on chart paper. No device is needed to learn to question a chart.
Give teams the Panel 1 numbers in a spreadsheet and have them create two charts, one with a truncated axis and one honest, to see how easily defaults mislead. If your campus has a demo account or anonymized view of a real dashboard, use it for the Reflect step. In a virtual session, share panels as slides and have teams complete Handout B in a shared document.
Does it need a screen? Building the misleading chart and the honest one from the same numbers in a spreadsheet shows, in under a minute, how a single axis setting changes the story. Paper can explain that; the spreadsheet lets leaders feel it.
What you should be able to see or collect if it worked.
Leaders spot truncated axes, small samples, and proxy metrics, then redraw a chart honestly, the same data-representation and privacy skills they want students to apply.
At your next data meeting, ask one question from the class list before any decision is made, and present one chart with its "what this can't tell us" note beside it. Review who has access to your campus dashboards with your data privacy contact this month.
