Teaching AI and Your Data
Not fear, not fatalism. Teach data privacy as a set of ordinary, learnable habits — and one rule that covers most of the risk.
Why this is hard to teach.
The room splits between “I don’t care, take my data’ and “everything is surveillance, why bother.” Both end the conversation. The useful middle is that the risk is real, specific, and mostly manageable with a couple of habits — no doom required.
The second difficulty is that the harm is delayed and abstract. Typing a health question into a chatbot feels harmless because nothing happens right then. The cost, if it comes, arrives months later through a breach, a lawsuit, or a policy change — which makes it easy to discount. Concrete cases (the 20-million-chat discovery order, the 23andMe bankruptcy) do the persuading that abstractions can’t.
Finally, keep it non-partisan and non-alarmist. The goal is capable, calm users who know where their data goes — not converts to a position.
Target misconceptions.
Misconception: “I have nothing to hide, so it doesn't matter.”
Reframe: The risk isn't embarrassment — it's that data held by a company is subject to breach, legal discovery, and policy change, none of which require you to have done anything wrong. In one 2025 case a court ordered 20 million ordinary ChatGPT conversations handed over in a lawsuit those users had nothing to do with.
Misconception: “Deleting the chat deletes the data.”
Reframe: Not necessarily. Retention, backups, training use, and legal holds can all outlive the visible conversation. 'Delete' in the interface is not the same as 'gone from the company.'
Misconception: “It's anonymized, so it's safe.”
Reframe: 'Anonymized' rarely means anonymous. A widely cited study estimated 99.98% of Americans could be re-identified from just 15 demographic attributes once datasets are combined.
Two classroom-ready activities.
Have everyone run the Data-Exposure Audit privately (it stores nothing). Then discuss only the aggregate patterns: which categories showed up most, how many people used a personal account for something sensitive, what surprised them. The privacy of the reflection is what makes people honest.
Facilitation:never ask anyone to share a specific item. You’re surfacing the shared pattern (“most of us have done this”), not auditing individuals.
In pairs, open the official data pages for two AI tools students actually use and answer the guide’s four questions: is it used for training? how long is it kept? do humans read it? does paying change it? The point is the skill of finding the answer on the primary source — not memorizing a policy that will change next quarter.
Facilitation:insist on the vendor’s own page, not a tech-news summary. Notice how hard some answers are to find — that’s part of the lesson.
Discussion prompts.
Ordered concrete to open-ended.
- 1What's something you've typed into an AI tool that you wouldn't put in an email to a stranger? What made the chat feel different?
- 2A company promises never to sell your data. Then it goes bankrupt. What happens to the promise? (This is roughly the 23andMe story.)
- 3Where should the line be between 'my data' and 'the model's training data' — and who should get to decide?
- 4Your school or employer gives you an enterprise AI account and a personal one. Which should you use for what, and why?
- 5If there's no comprehensive federal privacy law, should the protections you get depend on which state you live in? What's the argument each way?
- 6What would you want a company to tell you, in plain language, before you type something sensitive?
Seeing whether it landed.
- Ask each student to write their own three-line 'AI data rule' for themselves. You're assessing whether they can turn the lesson into a habit they'd actually keep — the enterprise-account rule is the tell.
- Have them find and quote one thing from a real vendor's policy page, with the URL. You're assessing whether they can locate a primary answer, not recall a fact.
- One-paragraph reflection: 'something I'll stop putting into a consumer AI tool, and why.'
When a student asks “is this actually true?”
The court cases and the COPPA rule are matters of public record — cite them plainly. Be precise about vendor practices: they change often, so the honest line is “here’s what the policy says today; check it yourself, because it may have moved.” That habit — verifying against the primary source — is the actual skill.
The re-identification statistic (99.98%) is from a real, peer-reviewed study, but it’s a modeled estimate, not a guarantee for every dataset. Say so; the nuance is part of the point.
Where this leads
Teaching this is a different skill than knowing it.
Teaching AI Well is CPAI's train-the-trainer curriculum for educators — ten lessons on how to teach AI honestly, including the material on this page.